1
Overview of Source Separation

Christian Jutten1, Leonardo Tomazeli Duarte2, and Saïd Moussaoui3

1GIPSA-lab, Univ. Grenoble Alpes, CNRS, Institut Univ., de France, Grenoble, France

2School of Applied Sciences (FCA), University of Campinas, Limeira, Brazil

3LS2N, Nantes Université, Ecole Centrale Nantes, Nantes, France

1.1 Introduction

The purpose of this chapter is to give a general overview of the source separation problem and its underlying hypotheses, and of methods and algorithms for solving the problem, with an emphasis on the context of data processing in physical/chemical sensing applications.

1.1.1 Brief Introduction to Source Separation

Source separation is a very general problem in signal processing, and, more generally, in sensing. In fact, a basic task in signal processing consists in separating useful information (called signal) from non-useful one (called noise) in noisy measurements. Measurements (also called observations) are frequently obtained through sensors, sensitive to some physical or chemical properties of the object which is analyzed. However, the sensor selectivity is limited so that its output depends on various phenomena (called sources).

As a first example, the signal captured by a microphone is the superimposition of signals emitted by all the acoustic sources in the neighborhood. Similarly, the signal measured by a scalp electrode in electroencephalography (EEG) is assumed to be the superimposition of the synchronous electrical activity of neural assemblies located in various areas in the brain. In remote sensing by hyperspectral imaging, due to low spatial resolution, the measured reflectance spectrum in each pixel is an aggregation of the reflectance spectra of all physical materials present in the ground surface related to this pixel. Finally, in chemical sensing, ion-sensitive electrodes have been designed for measuring activity of specific ions; however, due to a limited selectivity, the sensor output depends on the activity of the main ion and on the activities of interfering ions which can be present in the solution.

Source separation problems are considered in a blind (unsupervised) framework, i.e., by assuming that only sensor measurements (called mixtures) are available, but neither the source signals nor the mixing process (the superimposition in the above examples) are known. The main concept for solving blind source separation (BSS) problems is based on diversity, whose simplest implementation is to use a large number of sensors, thus providing spatial diversity. Solving source separation requires first to model the observations, i.e., how the signals received by the sensors are related to the sources, and then to add some (weak) priors and hypotheses on these sources in order to ensure their separability, and therefore the separation problem becomes well-posed.

The problem of source separation has been formulated in the middle of 1980s by Jutten and Hérault [1] for modeling motion decoding in vertebrates. Then, theoretical foundations have been developed mainly in the signal processing community by different researchers like Comon [2], Cardoso and Souloumiac [3,4], Pham and Garat [5], Pham and Cardoso [6], Delfosse and Loubaton [7], and in parallel in the Neural Networks and Machine Learning communities by researchers like Bell and Sejnowski in USA [8], Hyvärinen in Finland [9], Cichocki, Amari, and their team in Japan [10].

The interest for methods of source separation is due to its strong theoretical foundations [11] (Chapters 2-14) and to its very wide application domains [11] (Chapter 16). In 2022, with the keywords source separation, Google recalled more than 584 million entries!

From the application point of view, due to expansion of both low-cost sensors and powerful computers that are able to process very fast huge data sets, the problem of source separation appears in several domains, like communications [11] (Chapters 15 and 17), audio and music processing [12, 11] (Chapter 19), biomedical engineering [11] (Chapter 18), and remote sensing and hyperspectral imaging [13]. First applications of source separation appeared in the middle of 1990s and were focused on biomedical problems: non-invasive fetal electrocardiogram (ECG) separation [14] in 1994 and ECG processing [15] in 1996. Source separation has its success stories, too. As an example, applying the spectral matching independent component analysis algorithm [16] on the data provided by the Plank space mission, Cardoso was able to extract wonderful images of the Cosmic Microwaves Background, i.e., a very early image of our universe, which is a very important material for cosmologists.

In chemical engineering, it seems that source separation has been applied first for nuclear magnetic resonance spectroscopy [17] in 1998, and a larger number of works have been published in middle of 2000s as detailed by Monakhova et al. in the review [18]. In this context, even if Monte Carlo statistical methods have been proposed for mixture analysis [19], it must be noted that source separation is strongly related to positive matrix factorization [20] and to algebraic methods of tensor factorization, popular in Chemometrics and quoted PARAFAC [21].

1.1.2 Chapter's Organization

This chapter is a general introduction to the main concepts related to source separation. It is organized as follows. Section 1.2 presents the mathematical problem of source separation and a few basic solution principles. Section 1.3 focuses on source separation methods based on mutual statistical independence. Section 1.4 gives some examples of the various applications in physics and chemistry that can be formulated in the source separation framework. Section 1.5 is an overview of approaches that can be used for solving problems of Section 1.4. Section 1.6 details the organization of the book.

1.2 The Problem of Source Separation

1.2.1 Mathematical Description

Let us first consider a unique sensor and denote its output, at sample . Due to the poor selectivity of the sensor, one can model as a function of unknown sources, denoted , :

where models the mixing operator, unknown too.

In the simplest case, i.e. if the mapping is assumed linear, one could write:

Although this model is simple, we are faced with two problems: (i) the mixing coefficients are unknown, as well as the sources (ii) even if the mixing coefficients were known, the separation of the contributions coming from the various sources remains an ill-posed problem.

The first problem can be solved through modeling or identification methods. Modeling requires to write propagation equations of signals from sources to sensors, and implicitly to know the locations of sources and sensors. Then, identification can be achieved but it requires a training set of many samples , , i.e., pairs of inputs/output, for performing supervised parameter estimation.

The second problem could be solved if we have additional information concerning the different sources. For instance, in the above example of acoustic sources recorded by a microphone, if we know that the useful source and non-useful ones are characterized by different frequency bands, we can separate them by simple spectral filtering. Subtraction methods [22] can be also used, provided that one has a reference of the background noise (i.e. non-useful sources).

As a conclusion, solving the source separation problem with a unique sensor is impossible without very strong priors.

The basic concept of methods for solving source separation problems is diversity. A simple way to enhance diversity is to use a few sensors, say , instead of only one. In that case, at each sample , instead of having a scalar observation, we have a -dimensional observation vector :

where denotes the multidimensional mapping between the sources and the sensors. Denoting the -size vector of sources, one can also write:

For instance, in the above examples, spatial diversity is obtained by using several microphones or few EEG electrodes located at different places, or several image pixels, or several ion-sensitive sensors, specific to different ions. But another diversity is required, the sample diversity (time diversity in the above examples), i.e., the shapes of functions must be actually different. This sample diversity can be ensured by assuming that sources (considered as random variables) are mutually independent, or have different spectra or different variance time courses. Note that the discrete character of sources can be used instead [23].

The problem of source separation is often said blind in the signal processing community: in this context, blind refers to the fact that we only have observations , without a precise knowledge either on the sources or on the mapping . In other words, blind must be understood as unsupervised in machine learning. In such a...