1
Explanatory Tools for Machine Learning in the Symbolic Data Analysis Framework

The aim of this chapter is mainly to give explanatory tools for the understanding of standard, complex and big data. First, we recall some basic notions in Data Science: what are complex data? What are classes and classes of complex data? Which kind of internal class variability can be considered? Then, we define "symbolic data" and "symbolic data tables", which express the within variability of classes, and we give some advantages of such kind of class description. Often in practice the classes are given. When they are not given, clustering can be used to build them by the Dynamic Clustering method (DCM) from which DCM regression, DCM canonical analysis, DCM mixture decomposition, and the like can be obtained. The description of these class yields by aggregation to a symbolic data table. We say that the description of a class is much more explanatory when it is described by symbolic variables (closer from the natural language of the users), and then by its usual analytical multidimensional description. The explanatory and characteristic power of classes can then be measured by criteria based on the symbolic data description of these classes and induce a way for comparing clustering methods by their explanatory power. These criteria are defined in a Symbolic Data Analysis framework for categorical variables, based on three random variables defined on the ground population. Tools are then given for ranking individuals, classes and their symbolic descriptive variables from the more toward the less characteristic. These characteristics are not only explanatory but can also express the concordance or the discordance of a class with the other classes. We suggest several directions of research mainly on parametric aspects of these criteria and on improving the explanatory power of Machine Learning tools. We finally present the conclusion and the wide domain of potential applications in socio demography, medicine, web security and so on.

1.1. Introduction

A "Data Scientist" is someone who is able to extract new knowledge from Standard, Big and Complex Data. Here we consider complex data as data that cannot be expressed in terms of a standard data table, where units are described by quantitative and qualitative variables. Complex data happen in case of unstructured data, unpaired samples, and multisource data (as mixture of numerical, textual, image and social networks data). The aggregation, fusion, and summarization of such data can be done into classes of row units that are considered as new units. Classes can be obtained by unsupervised learning, giving a concise and structured view on the data. In supervised learning, classes are used in order to provide efficient rules for the allocation of new units to a class. A third way is to consider classes as new units described by "symbolic" variables whose values are "symbols" as: intervals, probability distributions, weighted sequences of numbers or categories, functions, and the like, in order to express their within-class variability. For example, "Regions" express the variability of their inhabitant, "Companies" express the variability of their web intrusion, and "Species" express the variability of their specimen. One of the advantages of this approach is that unstructured data and unpaired samples at the level of row units become structured and paired at the classes' level (see section 1.2.4).

Three principles guide this chapter in conformity with the Data Science framework. First, new tools are needed to transform huge data bases intended for management to data bases usable for Data Science tools. This transformation leads to the construction of new statistical units-described by aggregated data in terms of symbols as single-valued data-are not suitable because they cannot incorporate the additional information on data structure available in symbolic data. Second, we work on the symbolic data as they are given in data bases and not as we wish that they be given. For example, if the data contain intervals, we work on them even if the within-interval uniformity is statistically not satisfactory. Moreover, by considering Min-Max intervals, we can obtain useful knowledge, complementary to the one given without the uniformity assumption. Hence, considering that the Min-Max or interquartile where the aim is to extract useful knowledge from the data and not only to infer models (even if inferring models like in standard statistics, can for sure give complementary knowledge). Third, by using marginal description of classes by vectors of univariate symbols, rather than joint symbolic description by multivariate symbols, 99% of the users would say that a joint distribution describing a class often contains too much low or 0 values and so has a poor explanatory power in comparison with marginal distributions describing the same class. For example, having 10 variables of 5 categories each, the joint multivariate distribution leads to a sparse symbolic data table where the classes are described by a unique bar chart symbolic variable value containing 510 categories and taking for each class 510 low or 0 values. On the other hand, the 10 marginal bar chart symbolic variables' values describe the classes by vectors of 10 bar charts of 5 categories each, easy to interpret and to compare between classes. Nevertheless, a compromise can be obtained by considering joints instead of marginal between the more dependent variables.

Symbolic Data Analysis (SDA) is an extension of standard data analysis and data mining to symbolic data. The theory and practice of SDA have been developed in several books [AFO 18a], [BIL 06], [BOC 00], [DID 08], many papers (see overviews in [BIL 03] and [DID 16]), and several international workshops (http://vladowiki.fmf.uni-lj.si/doku.php?id=sda:meet:pa18). Special issue related to SDA has been published, for example, in the RNTI journal, edited by Guan et al. [GUA 13] on Advances in Theory and Applications of High Dimensional and Symbolic Data Analysis; in the ADAC journal on SDA, edited by Brito et al. [BRI 16]; in IEEE Transactions on Cybernetics [SU 16].

This chapter is organized into five sections. Section 1.2 aims to define symbolic data issued from the descriptions of classes of statistical units (called "individuals") in order to take care of their internal variability. "Complex data", "classes", and "classes of complex data" are defined. The symbolic data appear in the cells of a "symbolic data table", where the rows describe classes and the columns are associated with variables of symbolic value. Some advantages of symbolic data are finally given in this section.

Section 1.3 is devoted to the case where the classes are not given, but built by a clustering process. We illustrate this case by two clustering tools: Dynamic Clustering Method (DCM) and by mixture decomposition with the Estimation-Maximization (EM) method. We present different variants of the DCM, which can lead to different kinds of clusters, depending on the kind of clusters representative: regression, canonical analysis, distributions, and so on. Then, we show how to build a symbolic data table from the results of these clustering methods. Several criteria measuring the explanatory power of a symbolic data table are suggested. In consequence, the explanatory quality of clustering methods can be compared by these criteria.

In section 1.4, our aim is to define other kinds of explanatory criteria in the case where the initial variables defined on the ground population are of categorical value. We introduce, in this case, a theoretical framework of SDA based on three random variables. From this framework, we define two kinds of bar chart. The first called "fx(c)" which assigns to each category x, its frequency in the class, and the second called "gc, E (x)" that associates its frequency to each event E containing fx(c). These functions yield the characterization of pairs (category and class) by different kinds of criteria. We show that these criteria generalize to symbolic data, the standard Tf-Idf widely used in text mining (see, for example, [ROB 04]). According to these criteria, can be placed in order: the individuals, the classes, the symbolic variables and the symbolic data tables from the more to the less characteristic power.

Finally, in section 1.5, we suggest two directions of research. First, in this SDA framework, there are different possible parameterizations of the criteria expressed in terms of concordance or discordance of a class with the other classes are given. An interesting open question is to find in which condition when a sequence of partitions converges toward a trivial partition, such parametric criteria defined on classes converges toward a parametric distribution defined on O, as it is interesting and economical to obtain from distributions on classes to the distribution on the population (in the case of concordance or discordance). Second, as explaining for understanding is complementary to discriminating for learning, we suggest a filtering process that improves on a filtered sub-population the explanatory power without degrading the discriminating power of any learning machine tool.

1.2. Introduction to Symbolic Data Analysis

1.2.1. What are complex data?

By definition, "complex data" are any data set...