May 27, 2013

The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

Recently I came across a article in Wired with very interesting point of view about science and application of big data and analytics in it. The author suggests that the theories we build cannot be tested any more that is making further development of science impossible and new approach has to be applied among other things and I do recommend it for all of you interested in science, statistics and modelling.

The article The End of Theory: The Data Deluge Makes the Scientific Method Obsolete is published in Wired few year ago but is even more relevant today due to the latest development in Big Data. The author is exploring the weakness of modern science to go deeper in the secrets of nature and calls for a new approach with application of Big Data methods to find evidence of correlation and causation and build our view on the world. I really like the nice analogy of the science theories as models of reality and increasing costs of building and testing these models. I have always thought of my models as a piece of science. For many applications the statistical approach is maybe good. In majority of the cases, all we need for a good decision is well confirmed facts and theories for the relations between them are not necessary. This is sort of a black box - when you do A, B happens, why exactly is of no interest. The progress has advanced in this way for ages. However, the are applications that require identification of the true connections between facts and establish the causality chains. Statistical approach could extract correlations but this is not enough when the causality is important. When I think of it, there are considerable portion of models where the establishing correlation is not enough. Just for an example I would point the medical research where the true causes and relations are of a great matter (well, at least should be). All decision and modelling situations where the true reasons are important cannot be treated in a pure statistical way, no matter how complicated the statistics. There are situations when the status quo is changed - then all the data and statistical tricks cannot answer the questions about what happens next. For example, how would car market change in Arabic countries if women get the right to drive? The statistics and data we have exclude this group of buyers, so statistics cannot answer all the questions.

It is very nice and though-provoking article and I believe it would be interesting for everyone interested in science, statistics and modelling.

No comments:

Post a Comment