Introduction

In the world that surrounds us, a lot of events have a random (nondeterministic) structure. At molecular and subatomic levels, all natural phenomena are random. Movement of particles in the surrounding environment is accidental. Numerical characteristics of cosmic radiation and the results of monitoring the effect of ionizing radiation are random. The majority of economic factors surrounding asset prices on financial markets vary randomly. Despite efforts to mitigate risk and randomness, they cannot be completely eliminated. Moreover, in complex systems, it is often easier to reach an equilibrium state when they are not too tightly controlled. Summing-up, chance manifests itself in almost everything that surrounds us, and these manifestations vary over time. Anyone can simulate time-varying randomness by tossing a coin or rolling a dice repeatedly and recording the results of successive experiments. (If a physical random number is unavailable, one of the numerous computer algorithms to generate random numbers can be used.) In view of this ubiquity of randomness, the theory of probability and stochastic processes has a long history, despite the fact that the rigorous mathematical notion of probability was introduced less than a century ago. Let us speak more on this history.

People have perceived randomness since ancient times, for example, gambling already existed in ancient Egypt before 3000 BC. It is difficult to tell exactly when systematic attempts to understand randomness began. Probably, the most notable were those made by the prominent ancient Greek philosopher Epicurus (341-270 BC). Although his views were heavily influenced by Democritus, he attacked Democritus' materialism, which was fully deterministic. Epicurus insisted that all atoms experience some random perturbations in their dynamics. Although modern physics confirms these ideas, Epicurus himself attributed the randomness to the free will of atoms. The phenomenon of random detours of atoms was called clinamen (cognate to inclination) by the Roman poet Lucretius, who had brilliantly exposed Epicurus' philosophy in his poem On the Nature of Things.

Moving closer to present times, let us speak of the times where there was no theory of stochastic processes, physics was already a well-developed subject, but there wasn't any equipment suitable to study objects in sufficiently small microscopic detail. In 1825, botanist Robert Brown first observed a phenomenon, later called Brownian motion, which consisted of a chaotic movement of a pollen particle in a vessel. He could not come up with a model of this system, so just stated that the behavior is random.

A suitable model for the phenomenon arose only several decades later, in a very different problem, concerned with the pricing of financial assets traded on a stock exchange. A French mathematician Louis Bachelier (1870-1946), who aimed to find a mathematical description of stochastic fluctuations of stock prices, provided a mathematical model in his thesis "Théorie de la spéculation" [BAC 95], which was defended at the University of Paris in 1900. The model is, in modern terms, a stochastic process, which is characterized by the fact that its increments in time, in a certain statistical sense, are proportional to the square root of the time change; this "square root" phenomenon had also be observed earlier in physics; Bachelier was the first to provide a model for it. Loosely speaking, according to Bachelier, the asset price St at time t is modeled by

where a, b are constant coefficients, and ? is a random variable having Gaussian distribution.

The work of Bachelier was undervalued, probably due to the fact that applied mathematics was virtually absent at the time, as well as concise probability theory. Bachelier spent his further life teaching in different universities in France and never returned to the topic of his thesis. It was only brought to the spotlight 50 years after its publication, after the death of Bachelier. Now, Bachelier is considered a precursor of mathematical finance, and the principal organization in this subject bears his name: Bachelier Finance Society.

Other works which furthered understanding towards Brownian motion were made by prominent physicists, Albert Einstein (1879-1955) and Marian Smoluchowski (1872-1917). Their articles [EIN 05] and [VON 06] explained the phenomenon of Brownian motion by thermal motion of atoms and molecules. According to this theory, the molecules of a gas are constantly moving with different speeds in different directions. If we put a particle, say of pollen which has a small surface area, inside the gas, then the forces from impacts with different molecules do not compensate each other. As a result, this Brownian particle will experience a chaotic movement with velocity and direction changing approximately 1014 times per second. This gave a physical explanation to the phenomenon observed by the botanist. It also turned out that a kinetic theory of thermal motion required a stochastic process Bt. Einstein and Smoluchowski not only described this stochastic process, but also found its important probabilistic characteristics.

Only a quarter of a century later, in 1931, Andrey Kolmogorov (1903-1987) laid the groundwork for probability theory in his pioneering works About the Analytical Methods of Probability Theory and Foundations of the Theory of Probability [KOL 31, KOL 77]. This allowed his fellow researcher Aleksandr Khinchin (1894-1859) to give a definition of stochastic process in his article [KHI 34].

There is an anecdote related to the role of Khinchin in defining a stochastic process and the origins of the "stochastic" as a synonym for randomness (the original Greek word means "guessing" and "predicting"). They say that when Khinchin defined the term "random process", it did not go well with the Soviet authorities. The reason is that the notion of random process used by Khinchin contradicted dialectical materialism (diamat). In diamat, similarly to Democritus' materialism, all processes in nature are characterized by totally deterministic development, transformation, etc., so the phrase "random process" itself sounded paradoxical. As a result, to avoid dire consequences (we recall that 1934 was the apogee of Stalin's Great Terror), Khinchin had to change the name. After some research, he came up with the term "stochastic", from sto?ast??? t????, the Greek title of Ars Conjectandi, a celebrated book by Jacob Bernoulli (1655-1705) published in 1713, which contains many classic results. Being popularized later by William Feller [FEL 49] and Joseph Doob [DOO 53], this became a standard notion in English and German literature. Perhaps paradoxically, in Russian literature, the term "stochastic processes" did not live for long. The 1956 Russian translation of Doob's monograph [DOO 53] of this name was entitled Probabilistic processes, and now the standard name is random process.

An alternative explanation, given, for example, in [DEL 17], attributes the term "stochastic" to Ladislaus Wladyslaw Bortkiewicz (1868-1931), Russian economist and statistician, who in his paper, Die Iterationen [BOR 17], defined the term "stochastic" as "the investigation of empirical varieties, which is based on probability theory, and, therefore, on the law of large numbers. But stochastic is not simply probability theory, but above all probability theory and applications". This meaning correlates with the one given in Ars Conjectandi by Jacob Bernoulli, so the true origin of the term probably is somewhere between these two stories. It is also worth mentioning that Bortkiewicz is known for proving the Poisson approximation theorem about the convergence of binomial distributions with small parameters to the Poisson distribution, which he called the law of small numbers.

This historical discussion would be incomplete without mentioning Paul Lévy (1886-1971), a French mathematician who made many important contributions to the theory of stochastic processes. Many objects and theorems now bear his name: Lévy processes, Lévy-Khinchin representation, Lévy representation, etc. Among other things, he wrote the first extensive monograph on the (mathematical model of) Brownian motion [LÉV 65].

Further important progress in probability theory is related to Norbert Wiener (1894-1964). He was a jack of all trades: a philosopher, a journalist, but the most important legacy that he left was as a mathematician. In mathematics, his interest was very broad, from number theory and real analysis, to probability theory and statistics. Besides many other important contributions, he defined an integral (of a deterministic function) with respect to the mathematical model of Brownian motion, which now bears his name: a Wiener process (and the corresponding integral is called a Wiener integral).

The ideas of Wiener were developed by Kiyoshi Itô (1915-2008), who introduced an integral of random functions with respect to the Wiener process in [ITÔ 44]. This lead to the emergence of a broad field of stochastic analysis, a probabilistic counterpart to real integro-differential calculus. In particular, he defined stochastic differential equations (the name is self-explanatory), which allowed us to...