1
Information Processing

1.1. Background

Since the beginning, humankind has forever created and used various techniques to process, transmit and memorize information, a vast domain summed up by the term "information processing". This concept is very different from that of intelligence, and the reader should take care not to confuse these two ideas. This is a conflation commonly witnessed today owing to the buzz around the idea of artificial intelligence, which often only covers Big Data processing. Misuse of this term has seen the concepts of "intelligence" and "processing power" come to be erroneously employed interchangeably.

Intelligence is a much more complex matter that is beyond the scope of this book, referring instead to adaptation and imagination capacity. Moreover, Einstein is credited with the maxim: "The true sign of intelligence is not knowledge, but imagination."

Generally speaking, information processing can be broken down into several phases, set out in Figure 1.1.

Let us elaborate on this figure a little. One might say that for a piece of information (a sign, sound, color, etc.) existing in the real world, the first operation necessary before any processing is its acquisition through encoding.

Figure 1.1. Block diagram of information processing systems. For a color version of this figure, see www.iste.co.uk/cappy/neuro.zip

Our senses, such as our vision and hearing, use sensors, in this case the retina and the cochlea, whose role it is to encode visual and auditory information so that they can be processed by the brain. If this same visual and auditory information from the real world is captured by a camera and a microphone, the encoding will be different and may be analog or binary. Once encoded, the information can be processed. Processing machines and their internal organization can be very diverse, ranging from the brain to computers, but all possess a device enabling information memorization and communication, whether this be within the machine itself, or with a similar machine. After processing, the information, which exists in the machine's encoding system, will need to be decoded in order for it to be interpretable in the real world. Decoding is, for example, the role of the computer screen displaying a computation result.

Let us briefly develop on these basic concepts.

1.1.1. Encoding

All processing requires a prior information encoding operation. To illustrate this concept, let us imagine that the information in question is the temperature, T, of a physical environment.

Figure 1.2. Different information encoding types

Encoding can be analog (Figure 1.2a), in which case, the "temperature" magnitude will be represented after encoding by a continuous value1 x (e.g. voltage or current), such that:

where is the encoding transfer function. If, for example, is a linear function, we will have x = a.T + ß, where a and ß are real constants.

When encoding is analog, the information processing must also be analog, in which case we speak of analog computers.

Another way to process this information is to code it in binary. In this case, we choose a pitch ?T, and T will be represented by a number, M, of N bits such that:

One of the N bits (Figure 1.2b) will only be able to take on two values, arbitrarily noted 0 and 1. Binary coding offers numerous advantages: it enables the use of Boolean algebra and the set of associated algorithms for processing encoded information, and can easily be implemented with physical systems in two entirely distinct states. The switch, in particular, is either open (state 0) or closed (state 1).

Other encoding methods also exist. In the brain, for example, the organ focused on in this book, information is both time and frequency encoded by electrical impulses known as "action potential", also referred to below as "spikes" (Figure 1.2c). In the case of frequency encoding, for example, the average frequency, < F >, of the impulses will be linked to the magnitude to be encoded by an expression of the type:

Where is the encoding transfer function. In the case of a linear transfer function, the frequency of the impulses will be given, for example, by < F > = ?.T + d, where ? and d are real constants.

In summary, encoding can be analog, i.e. represented by a continuous value, and can also consist of an alphabet, i.e. limited to a set of signs such as binary code {0,1} of Boolean algebra or the four bases of DNA {A, T, G, C}, which encode the genome. Lastly, it can be event-based and linked to the appearance of a temporally-determined phenomenon.

There is a close link between the choice of encoding and that of the technology of the processing machine that is to process the encoded information: analog or binary encoding for electronic processing, and pulse encoding for the brain.

1.1.2. Memorization

The artist who, over 30,000 years ago, painted the "Horses panel" in France's Chauvet-Pont-d'Arc Cave (Chauvet 1994) was not only processing the information of the world around him, but also enabling it to be analogously memorized. The memorization technique used, rock painting, is simple but of the highest quality, because the message has been conserved right up to the present day!

Throughout history, information memorization has experienced two major revolutions: printing in the 15th Century and digital technology in the mid-20th Century. We can thus observe three characteristic periods. Before Gutenberg and the development of printing, a great number of techniques were used to store information, among them paint, clay, wax and paper. The printed book would come to dominate information memorization for almost five centuries, with this domination today replaced by digital technology. Nevertheless, in the digital world of today, there is a tendency to equate "memory" with "digital memory", yet there are still many analog memories in existence, including not only painting and photography, but also sound on a vinyl record.

Regardless of the technology used, the memorization process consists of the same three phases: writing information on the memorization medium, storing it, and retrieving or reading it. The performance of a particular information memorization technology is characterized by quality criteria such as read and write speeds, storage durability, etc. Each of these performance elements is closely linked to both the quality of the encoding process and the machine processing the memorized information.

1.2. Information processing machines

In this section, we will turn our attention to artifacts, i.e. human-made machines that are used for information processing. This section focuses on dominant technology composed of binary coding, John von Neumann's processing architecture and semiconductor devices for their material implementation.

1.2.1. The Turing machine

Alan Turing, the brilliant 20th-Century mathematician, used abstract and theory to demonstrate that any computing machine could be reduced to three elements (Figure 1.3):

• - a tape of infinite length divided into squares, each containing a symbol (e.g. 0 or 1 in binary language) or a "blank" where the square has not yet been written. The tape represents the input/output peripheral device;
• - a read/write head, which reads or writes the symbols and can move one square to the right or left;
• - a state register, which memorizes the current state of the machine. There is a finite number of possible states. A "start state" is the initial state of the machine before executing a program;
• - at each stage a program, or table of actions, defines whether the machine is to read or write and whether it is to move to the right or to the left, and specifies the new state.

The machine operates as follows: if the machine is in a state En, and the symbol read is s, i.e. the square beneath the read/write head contains the symbol s, then it writes the symbol e and moves in direction r (to the right) or l (to the left), and its new state is En+1.

Turing demonstrated that in binary logic, two functions simply needed to be created in order to perform any computational algorithm: one operating on one bit (the inverter) and the other on two bits (OR, AND, EXCLUSIVE OR, etc.).

Figure 1.3. The Turing machine and its practical realization (http://aturingmachine.com/)

The Turing machine was defined at a time when computers as we know them today did not yet exist. Thus, it is first and foremost an abstract tool, although there have been practical implementations of the machine, such as that shown in Figure 1.3. It is a universal machine model, which can compute anything a...