The only human being who wants change is a toddler who needs a new diaper. This bon mot is especially true if a change is fundamental, if something substantial is changing, if a paradigm shifts, such as the huge Big Data paradigm shift.
Now and then, things, technologies, perspectives change dramatically. If you happen to work in an industry experiencing such a change, consider yourself fortunate if you can accept or even welcome it. Many, if not most people try to prevent changes, either being fully aware or as an act of pushing something to the back of their minds.
The main difference between a change in terms of a development process, such as an evolution or even a revolution, and a paradigm shift, is: a change takes time. It might surprise people, but it can. be separated into different steps. Compared with that, a paradigm shift just happens. It suddenly occurs to people that something should be seen in a fundamentally different way.
Having been active in the field of Big Data for a few years, and talking about the paradigm shift in that area; i.e. hypotheses-driven statistics being removed by correlation-gathering data science, I suddenly realized exactly the same paradigm shift in totally different aspects of life: in healthcare and politics. You might agree that there are some connections between healthcare or politics and Big Data, but what fascinates me is that in all three there is the same paradigm shift.
The Predisposition Of The Data Collector
Starting with Big Data: before we began collecting all data we could get our hands-on, we threw some data away and aggregated most of the rest before analyzing it. We hypothesized and came to conclusions based on that limited thinking. That was ‘statistics’. Today, we dispassionately collect all data, store it, add some layers to make data manageable, bombard it with algorithms and analytics to find correlations, and then tell the stories behind the data. The paradigm shift lies in losing interest in pre-fabricated hypotheses: if you ever worked with data guys educated in the old days of statistics you know how difficult it is to overcome this archaic way of thinking.
Hypothesis Vs Exploration In Healthcare
Now — let’s move to healthcare. In modern orthodox medicine, health experts are strong believers in scientific procedures examining patients and finding new drugs. If you happen to be taken to a hospital’s emergency department, nobody knows you, there is no data at all about you and if you’re lucky, you yourself or a relative can tell the doctors what happened to you. Then, the doctor looks at you, examines you, and looks at her computer screen for similar symptoms. The most appropriate match will be chosen and will define your illness. What we have here: one or more hypotheses, very small data, and a quickly made conclusion based on that limited information.
If you visit your doctor and explain your illness’s symptoms to her, she will try to “understand” your illness by comparing what you tell her with what she knows or she finds searching in her database. In the best case, she has seen you before and she has at least some data about you. But what she does not have is a consistent or complete picture of you. She doesn’t know anything about your lifestyle, your permanent or recent experiences — unless you tell her. And that’s only anecdotal evidence. How different your doctor’s visit will look when you share your body activity data with her on a regular basis. When she is consistently informed about you, your lifestyle, and your physical and mental health. She knows everything even before you visit her. She lets her system analyze your data and compare it with other data of people with the same lifestyle and living conditions as you. She might even ask you to see her because your data suggest so — before you yourself become aware of an impending health problem. In this case she does not hypothesize, or rack her brains but she will be informed by her alarm system about your health issue and will then decide how to support you.
In healthcare, as in technology, the hypotheses and expertise of the expert or expert center will be replaced by dispassionate management and analytics of Big Data, provided by yourself, your doctor’s offices and offices, and hospitals all over the world. It works, because the data already exists, it just has to be managed well. And health experts should realize that they need to use, understand, and instrumentalize data.
By matching individual patient’s data with other internal and external data, data has become a factor of production — data itself adds value to the production process.
Hypothesis Vs Exploration In Politics
Now, let’s move over to politics. The first nations, or historically constituted, stable communities of people, are defined to be England and the Dutch Republic — in the early 17th century. Modern sociologists speak of civic nations such as France, and ethnic nations such as Germany. With nations come borders, with borders come border controls. Living on a quite fragmented continent, Europeans cheered to the signing of the Schengen treaty in 1985 that allowed inhabitants of Belgium, France, (West) Germany, Luxembourg and the Netherlands to travel without any border controls within their shared area. Today, the Schengen area consists of 26 European countries with a population of over 400 million people and no border controls within that area.
With more and more refugees immigrating from Syria, Iraq, and other countries to our rich West-European countries, governments get under pressure. When German chancellor Angela Merkel declared to welcome all refugees independently of current European regulations, other European heads of state were upset. They asked Germany to stick to the rules, but Angela Merkel was adamant about welcoming all refugees without condition. This behavior led to tensions, not only between European states but especially within Germany itself when the typical share of 30% of a population started to express nationalistic ideas that have been echoed and amplified by politicians with the same mindset. Rich Europeans fear that their health and welfare systems will implode facing too many newly arrived recipients of transfer benefits.
Whereas a concept of nations allows governments to plan welfare systems based on hypotheses about stable communities, this kind of planning becomes impracticable when these communities face huge flows of immigrants. While a government knows quite a lot about each citizen of its country, it knows nothing about a refugee. While being able to influence their respective economies and legislations, they have no means to prevent civil wars and economic disasters abroad which forces people to flee these countries and immigrate to safer and richer areas. In other words: even governments in rich countries don’t have enough information to guarantee well-informed decision-making and a subsequent stable environment and living conditions.
The Huge Big Data Paradigm Shift
In the case of nations it becomes clear that limited access to information paired with a belief in hypothesizing based on poor information leads to ill-informed decisions by governments and questionable opinions of citizens. Typically, governments then are grievously mistaken by trying to stabilizing their old systems, i.e. fortifying their borders by adding to existing controls or building new ones. The history of migrations shows that these efforts of preserving old systems won’t be successful.
As discussed regarding Big Data technology and healthcare, the paradigm shift from hypothesizing based on poor information to collecting as much data as possible, analyzing it, finding correlations, and drawing conclusions from it takes place in politics, too. Old-fashioned politicians and governments vanish and are replaced by open-minded governments and leaders who take advantage of that paradigm shift. The government of Estonia with its Prime Minister Taavi Rõivas is a good example of a group of leaders who provide their citizens with data-driven technologies and, subsequently, with more political and social autonomy than their European neighbors’ experience.
Human beings don’t want change (besides the toddlers). But we see this paradigm shift from individual experts or centralized institutions acting, hypothesizing and deciding on the basis of limited data and poor information, to decentralized autonomous entities gathering as much data as possible, deciding on correlations rather than perceived causalities, and steadily updating their decisions with each new and different data set. There are areas where people need more time to adapt to that new paradigm than in other areas. But I expect the world-changing irrevocably, and definitely for the better, thanks to this paradigm shift.