Forecasting with Maximum Entropy

The principle of maximum entropy simply states that the probability distribution that best represents the current state of knowledge is the one with the highest entropy as it makes the fewest assumptions about the true distribution of the data. It is regarded as a foundation of statistical inference due to its broad applicability, conceptual simplicity and secure mathematical underpinning. A key tool in mathematical and applied physics it is now being applied to construct both descriptive and predictive models of complex systems and networks in a wide range of sciences from biology to structural engineering.
This book aims at providing a unifying framework, based on Information Entropy and its maximization, to connect the phenomenology of evolutionary biology, community ecology, financial economics, and statistical physics. This more comprehensive view, besides providing further insight into problems, enables problem-solving strategies by applying proven methods in one discipline to formally similar problems in other areas. The book also proposes a forecasting method for important practical problems in these disciplines and is directed to researchers, students and practitioners working on modelling the dynamics of complex systems.
The common thread is how the flux of information both controls and serves to predict the dynamics of complex systems. The basis for information theory was laid down by Claude Shannon who defined information as a reduction in uncertainty or entropy. Subsequentially E. T. Jaynes proposed the Maximum Entropy principle as a general method to make the least biased inferences compatible with available information. It is shown how maximizing the Shannon information entropy allows one to infer a central object controlling the dynamics of complex systems, such as ecosystems or markets. The resulting models, which are known as pairwise maximum-entropy models, can be used to infer interactions from data in a wide variety of systems. Here, two examples are analysed in detail. The first is an application to conservation ecology, namely the issue of providing early warning indicators of population crashes of species of trees in tropical forests. The second is about forecasting the market values of firms through evolutionary economics. An interesting lesson is that PME modelling often produces accurate predictions despite not incorporating explicit interaction mechanisms.