The Huge Big Data Paradigm Shift

The only human being who wants change is a tod­dler who needs a new dia­per. This bon mot is espe­cial­ly true if a change is fun­da­men­tal, if some­thing sub­stan­tial is chang­ing, if a par­a­digm shifts, such as the huge Big Data par­a­digm shift.

Now and then, things, tech­nolo­gies, per­spec­tives change dra­mat­i­cal­ly. If you hap­pen to work in an indus­try expe­ri­enc­ing such a change, con­sid­er your­self for­tu­nate if you can accept or even wel­come it. Many, if not most peo­ple try to pre­vent changes, either being fully aware or as an act of push­ing some­thing to the back of their minds.

The main dif­fer­ence between a change in terms of a devel­op­ment process, such as an evo­lu­tion or even a rev­o­lu­tion, and a par­a­digm shift, is: a change takes time. It might sur­prise peo­ple, but it can. be sep­a­rat­ed into dif­fer­ent steps. Com­pared with that, a par­a­digm shift just hap­pens. It sud­den­ly occurs to peo­ple that some­thing should be seen in a fun­da­men­tal­ly dif­fer­ent way.

Hav­ing been active in the field of Big Data for a few years, and talk­ing about the par­a­digm shift in that area; i.e. hypotheses-driven sta­tis­tics being removed by correlation-gathering data sci­ence, I sud­den­ly real­ized exact­ly the same par­a­digm shift in total­ly dif­fer­ent aspects of life: in health­care and pol­i­tics. You might agree that there are some con­nec­tions between health­care or pol­i­tics and Big Data, but what fas­ci­nates me is that in all three there is the same par­a­digm shift.

The Pre­dis­po­si­tion Of The Data Collector

Start­ing with Big Data: before we began col­lect­ing all data we could get our hands-on, we threw some data away and aggre­gat­ed most of the rest before ana­lyz­ing it. We hypoth­e­sized and came to con­clu­sions based on that lim­it­ed think­ing. That was ‘sta­tis­tics’. Today, we dis­pas­sion­ate­ly col­lect all data, store it, add some lay­ers to make data man­age­able, bom­bard it with algo­rithms and ana­lyt­ics to find cor­re­la­tions, and then tell the sto­ries behind the data. The par­a­digm shift lies in los­ing inter­est in pre-fabricated hypothe­ses: if you ever worked with data guys edu­cat­ed in the old days of sta­tis­tics you know how dif­fi­cult it is to over­come this archa­ic way of thinking.

Hypoth­e­sis Vs Explo­ration In Healthcare

Now — let’s move to health­care. In mod­ern ortho­dox med­i­cine, health experts are strong believ­ers in sci­en­tif­ic pro­ce­dures exam­in­ing patients and find­ing new drugs. If you hap­pen to be taken to a hos­pi­tal’s emer­gency depart­ment, nobody knows you, there is no data at all about you and if you’re lucky, you your­self or a rel­a­tive can tell the doc­tors what hap­pened to you. Then, the doc­tor looks at you, exam­ines you, and looks at her com­put­er screen for sim­i­lar symp­toms. The most appro­pri­ate match will be cho­sen and will define your ill­ness. What we have here: one or more hypothe­ses, very small data, and a quick­ly made con­clu­sion based on that lim­it­ed information.

If you visit your doc­tor and explain your ill­ness’s symp­toms to her, she will try to “under­stand” your ill­ness by com­par­ing what you tell her with what she knows or she finds search­ing in her data­base. In the best case, she has seen you before and she has at least some data about you. But what she does not have is a con­sis­tent or com­plete pic­ture of you. She does­n’t know any­thing about your lifestyle, your per­ma­nent or recent expe­ri­ences — unless you tell her. And that’s only anec­do­tal evi­dence. How dif­fer­ent your doc­tor’s visit will look when you share your body activ­i­ty data with her on a reg­u­lar basis. When she is con­sis­tent­ly informed about you, your lifestyle, and your phys­i­cal and men­tal health. She knows every­thing even before you visit her. She lets her sys­tem ana­lyze your data and com­pare it with other data of peo­ple with the same lifestyle and liv­ing con­di­tions as you. She might even ask you to see her because your data sug­gest so — before you your­self become aware of an impend­ing health prob­lem. In this case she does not hypoth­e­size, or rack her brains but she will be informed by her alarm sys­tem about your health issue and will then decide how to sup­port you.

In health­care, as in tech­nol­o­gy, the hypothe­ses and exper­tise of the expert or expert cen­ter will be replaced by dis­pas­sion­ate man­age­ment and ana­lyt­ics of Big Data, pro­vid­ed by your­self, your doc­tor’s offices and offices, and hos­pi­tals all over the world. It works, because the data already exists, it just has to be man­aged well. And health experts should real­ize that they need to use, under­stand, and instru­men­tal­ize data.
By match­ing indi­vid­ual patien­t’s data with other inter­nal and exter­nal data, data has become a fac­tor of pro­duc­tion — data itself adds value to the pro­duc­tion process.

Hypoth­e­sis Vs Explo­ration In Politics

Now, let’s move over to pol­i­tics. The first nations, or his­tor­i­cal­ly con­sti­tut­ed, sta­ble com­mu­ni­ties of peo­ple, are defined to be Eng­land and the Dutch Repub­lic — in the early 17th cen­tu­ry. Mod­ern soci­ol­o­gists speak of civic nations such as France, and eth­nic nations such as Ger­many. With nations come bor­ders, with bor­ders come bor­der con­trols. Liv­ing on a quite frag­ment­ed con­ti­nent, Euro­peans cheered to the sign­ing of the Schen­gen treaty in 1985 that allowed inhab­i­tants of Bel­gium, France, (West) Ger­many, Lux­em­bourg and the Nether­lands to trav­el with­out any bor­der con­trols with­in their shared area. Today, the Schen­gen area con­sists of 26 Euro­pean coun­tries with a pop­u­la­tion of over 400 mil­lion peo­ple and no bor­der con­trols with­in that area.

With more and more refugees immi­grat­ing from Syria, Iraq, and other coun­tries to our rich West-European coun­tries, gov­ern­ments get under pres­sure. When Ger­man chan­cel­lor Angela Merkel declared to wel­come all refugees inde­pen­dent­ly of cur­rent Euro­pean reg­u­la­tions, other Euro­pean heads of state were upset. They asked Ger­many to stick to the rules, but Angela Merkel was adamant about wel­com­ing all refugees with­out con­di­tion. This behav­ior led to ten­sions, not only between Euro­pean states but espe­cial­ly with­in Ger­many itself when the typ­i­cal share of 30% of a pop­u­la­tion start­ed to express nation­al­is­tic ideas that have been echoed and ampli­fied by politi­cians with the same mind­set. Rich Euro­peans fear that their health and wel­fare sys­tems will implode fac­ing too many newly arrived recip­i­ents of trans­fer benefits.

Where­as a con­cept of nations allows gov­ern­ments to plan wel­fare sys­tems based on hypothe­ses about sta­ble com­mu­ni­ties, this kind of plan­ning becomes imprac­ti­ca­ble when these com­mu­ni­ties face huge flows of immi­grants. While a gov­ern­ment knows quite a lot about each cit­i­zen of its coun­try, it knows noth­ing about a refugee. While being able to influ­ence their respec­tive economies and leg­is­la­tions, they have no means to pre­vent civil wars and eco­nom­ic dis­as­ters abroad which forces peo­ple to flee these coun­tries and immi­grate to safer and rich­er areas. In other words: even gov­ern­ments in rich coun­tries don’t have enough infor­ma­tion to guar­an­tee well-informed decision-making and a sub­se­quent sta­ble envi­ron­ment and liv­ing conditions.

The Huge Big Data Par­a­digm Shift

In the case of nations it becomes clear that lim­it­ed access to infor­ma­tion paired with a belief in hypoth­e­siz­ing based on poor infor­ma­tion leads to ill-informed deci­sions by gov­ern­ments and ques­tion­able opin­ions of cit­i­zens. Typ­i­cal­ly, gov­ern­ments then are griev­ous­ly mis­tak­en by try­ing to sta­bi­liz­ing their old sys­tems, i.e. for­ti­fy­ing their bor­ders by adding to exist­ing con­trols or build­ing new ones. The his­to­ry of migra­tions shows that these efforts of pre­serv­ing old sys­tems won’t be successful.
As dis­cussed regard­ing Big Data tech­nol­o­gy and health­care, the par­a­digm shift from hypoth­e­siz­ing based on poor infor­ma­tion to col­lect­ing as much data as pos­si­ble, ana­lyz­ing it, find­ing cor­re­la­tions, and draw­ing con­clu­sions from it takes place in pol­i­tics, too. Old-fashioned politi­cians and gov­ern­ments van­ish and are replaced by open-minded gov­ern­ments and lead­ers who take advan­tage of that par­a­digm shift. The gov­ern­ment of Esto­nia with its Prime Min­is­ter Taavi Rõi­vas is a good exam­ple of a group of lead­ers who pro­vide their cit­i­zens with data-driven tech­nolo­gies and, sub­se­quent­ly, with more polit­i­cal and social auton­o­my than their Euro­pean neigh­bors’ experience.

Human beings don’t want change (besides the tod­dlers). But we see this par­a­digm shift from indi­vid­ual experts or cen­tral­ized insti­tu­tions act­ing, hypoth­e­siz­ing and decid­ing on the basis of lim­it­ed data and poor infor­ma­tion, to decen­tral­ized autonomous enti­ties gath­er­ing as much data as pos­si­ble, decid­ing on cor­re­la­tions rather than per­ceived causal­i­ties, and steadi­ly updat­ing their deci­sions with each new and dif­fer­ent data set. There are areas where peo­ple need more time to adapt to that new par­a­digm than in other areas. But I expect the world-changing irrev­o­ca­bly, and def­i­nite­ly for the bet­ter, thanks to this par­a­digm shift.

Leave a Reply


The Mind­ful Rev­o­lu­tion, Michael Reuter

Die Acht­same Rev­o­lu­tion, Michael Reuter

What‘s our prob­lem?, Tim Urban

Rebel Ideas — The Power of Diverse Think­ing, Matthew Syed

Die Macht unser­er Gene, Daniel Wallerstorfer

Jel­ly­fish Age Back­wards, Nick­las Brendborg

The Expec­ta­tion Effect, David Robson

Breathe, James Nestor

The Idea of the Brain, Matthew Cobb

The Great Men­tal Mod­els I, Shane Parrish

Sim­ple Rules, Don­ald Sull, Kath­leen M. Eisenhardt

Mit Igno­ran­ten sprechen, Peter Modler

The Secret Lan­guage of Cells, Jon Lieff

Evo­lu­tion of Desire: A Life of René Girard, Cyn­thia L. Haven

Grasp: The Sci­ence Trans­form­ing How We Learn, San­jay Sara

Rewire Your Brain , John B. Arden

The Wim Hof Method, Wim Hof

The Way of the Ice­man, Koen de Jong

Soft Wired — How The New Sci­ence of Brain Plas­tic­i­ty Can Change Your Life, Michael Merzenich

The Brain That Changes Itself, Nor­man Doidge

Lifes­pan, David Sinclair

Out­live — The Sci­ence and Art of Longevi­ty, Peter Attia

Younger You — Reduce Your Bioage And Live Longer, Kara N. Fitzgerald

What Does­n’t Kill Us, Scott Carney

Suc­cess­ful Aging, Daniel Levithin

Der Ernährungskom­pass, Bas Kast

The Way We Eat Now, Bee Wilson

Dein Gehirn weiss mehr als Du denkst, Niels Birbaumer

Denken: Wie das Gehirn Bewusst­sein schafft, Stanis­las Dehaene

Mind­ful­ness, Ellen J. Langer

100 Plus: How The Com­ing Age of Longevi­ty Will Change Every­thing, Sonia Arrison

Think­ing Like A Plant, Craig Holdredge

Das Geheime Wis­sen unser­er Zellen, Son­dra Barret

The Code of the Extra­or­di­nary Mind, Vishen Lakhiani

Altered Traits, Daniel Cole­man, Richard Davidson

The Brain’s Way Of Heal­ing, Nor­man Doidge

The Last Best Cure, Donna Jack­son Nakazawa

The Inner Game of Ten­nis, W. Tim­o­thy Gallway

Run­ning Lean, Ash Maurya

Sleep — Schlafen wie die Profis, Nick Littlehales