Probabilistic Framework for Person Tracking and Classification in Security Assistance Systems

In the context of intelligent surveillance of public facilities, the automatic analysis of persons by distributed sensor systems increasingly gains in importance. The detection of hazardous material in busy areas as well as its assignment to a person is a challenging task that cannot be performed without technical decision support. However, the application of conventional technologies and the corresponding courses of action lead to long waiting times and pressure of work for the security personnel. This situation can be extremely relieved by security assistance systems with the ability to analyze an area automatically, based on distributed sensor systems and appropriate data fusion techniques.
This thesis makes contributions to the design and the realization of a security assistance system. The system aims at localizing a person with hazardous material in a person stream. To this end, the design stipulates two complementary types of sensors: tracking sensors and sensors to detect hazardous material. While the first type provides location data of high accuracy, but does not have any detection abilities, the second type is able to signalize the presence of a material, but is not able to localize the source. Only within a data fusion approach combining the strengths of the two types of sensor technologies, it is possible to distinguish the carrier of a material from the non-carriers in the person stream. This thesis is particularly focussed on the design of data fusion techniques to realize the described system task. In the context of this task, a probabilistic framework for person tracking and classification in well-defined areas is developed.
To realize the tracking component, a novel approach for tracking multiple persons as extended objects is developed. Within this approach ellipsoidal object extents are modeled by random matrices and treated as additional state variables to be estimated. The unknown object extent is accommodated through the use of a Wishart prior on the measurement probability density. An extended object can either be given by a single person due to high-resolution sensor data, or by a group of persons with correlated movements. Methodically considered, the new approach is derived by integrating random matrices with the framework of Probabilistic Multi-Hypothesis Tracking (PMHT). The new algorithm is called the PMHT for Extended Objects (PMHT-E). Besides the derivation and the statement of the basic PMHT-E, several useful extensions are introduced in this thesis. This includes the derivation of the Histogram PMHT-E for tracking extended objects in image sequences and the design of a track management system (TMS). Besides functions for track extraction and deletion, the TMS also includes the handling of split and merge events as they occur naturally in group object tracking. A simulation study shows that the PMHT-E outperforms conventional point tracking approaches. The full framework of Histogram PMHT-E is finally applied to track persons in two real video sequences. This also includes a comparison with a particle filter solution.
The classification component of the assistance system is essentially based on the computation of assignment probabilities between the person tracks and the sensors for hazardous material detection. In other words, the source localization task is reduced to the task of finding the best match between the tracks and a series of detection signals. The detection signals are either a sequence of concentration measurements signalizing the amount of certain gas molecules, or a sequence of count rates signalizing the strength of gamma radiation at the sensor position. The (quantized) detection signals are interpreted as classes of hazardousness so that each of the assignment probabilities represents the relevance of a class measurement with respect to the classification of a tracked person. The class measurements and their relevance probabilities are collected over a suitable time window and provide the basis for the estimation of a classification matrix, from which the final source carrier potential of each person is derived. The potential of being the carrier of the detected hazardous material is then displayed next to each person track on the screen. Methodically speaking, the algorithm results from an adaption of the PMHT with Classification Measurements (PMHT-C), which was originally designed to exploit class information for improved tracking and data association. In this thesis, the PMHT-C framework is used to exploit tracking for classification purposes.
The first version of this PMHT-C exploits kinematical similarities between the person tracks and the detection data. More technically speaking, the assignment probabilities are governed by a position and a velocity likelihood. While the position likelihood evaluates the distance between a person and a sensor, the velocity likelihood compares the sensor-relative velocity of a person with the slope of the sensor signal. As an extension to the kinematics-based version, this thesis introduces the PMHT-CR version, which is particularly designed for the localization of radioactive sources. Within this approach, the assignment probabilities are not only controlled by the position and the velocity likelihood, but also by an additional intensity likelihood. To derive the intensity likelihood, each person is considered as a fictitious source carrier. Based on the measured count rates, a novel Poisson filter is applied, to estimate the gamma emission rate (referred to as intensity) that is required on the person's trajectory to achieve the count rate measurements. However, only the trajectory of the true source carrier has a functional relation to the measured count rates that is consistent with the dispersion model of the radiation. This leads to large likelihood values for the true carrier and small likelihood values for the non-carriers.
This thesis presents an experimental assembly called HAMLeT1 to test the kinematics-based PMHT-C and the PMHT-CR with real chemical and real radioactive sources. The PMHT-C and the PMHT-CR are demonstrated in choreographed scenarios with multiple persons walking through the system. Moreover, the PMHT-CR is compared with another source localization approach based on count rate accumulation. The superiority of the PMHT-CR is additionally verified by a simulation study.