1
Extraction and Segmentation of Structures in Image Sequences

1.1 Problematics

The analysis of imaged anatomical or biological structures and of their dynamics is an important task in terms of application and therefore of diagnostics. This analysis facilitates the quantification of the shape of these structures and their possible evolution over time, whether this evolution is intrinsic to the functioning of the structure (cardiac motion for example) or indicating a transformation related to a pathology (tumor evolution).

Such an analysis involves in the first place the extraction of these structures from the acquired images according to a given modality, which corresponds, in image processing terminology, to a segmentation phase. This chapter is devoted to this problem: after a very brief overview of the existing techniques, it discusses in detail the methodology of deformable models and more specifically their more flexible form, namely variational active contours. The chapter concludes with specific examples for the application of this type of technique carried out in the field of cardiac ultrasound imaging.

1.2 Overview of segmentation methods

Segmentation is a fundamental operation in imaging and cardiac and thoracic imaging in particular. Its role consists of assigning to the parts of an image a relevant category ("muscle", "blood", "tumor", etc.) relating to the underlying medical application: detection of the presence/absence of a pathological structure (for example "tumor", "aneurysm"), evaluation of the area, the extent, the volume of organs or the pathological structures as well as their evolution over time. Due to this central role, image segmentation is a very active area of research. This can be illustrated by observing the result of a search on the Web of Science® (Figure 1.1) and based on the presence of the terms "image segmentation" in the title of articles over 15 years (1994-2009). It can be seen that more than 4,700 articles1 have been published during this period and that this number is constantly increasing.

Figure 1.1. Number of articles containing the terms "image segmentation" in the title for the period 1994-2009

A segmentation method can be schematically characterized by three main elements (see Figure 1.2): (1) the low level properties (or "image information") used to characterize the objects to detect, (2) a priori knowledge introduced to constrain the segmentation and (3) the formalism chosen to integrate these two pieces of information.

If only the "image information" aspect is considered, segmentation can be formally defined as an operation consisting of partitioning the image in related regions verifying a consistency predicate, based for example on statistical properties or on texture. Segmentation can also be carried out according to a dual approach by considering the differences between these regions: two adjacent regions must actually present significant variations of properties along their common border. These variations can be quantified using conventional differential operators (for example, the amplitude of the gray level gradient) or more sophisticated techniques such as the phase-based approach developed by [MUL 00]. Following these definitions, segmentation methods are conventionally qualified as "region-based approaches" or "contour-based approaches".

Figure 1.2. General outline of a segmentation approach

Due to imperfections presented by images (i.e. noise, occlusions, lack of contrast, etc.), to perform a segmentation using only the region or contour characteristics previously referred to reveals itself in most cases to be difficult, if not impossible. That is why a priori knowledge is usually introduced, relative to the intrinsic properties of the object to be detected, such as its shape, its grayscale distribution or its motion when it comes to image sequence. This knowledge may be purely abstract (for example "the form of the object must be smooth") or built from the statistical analysis of a training set representative of the images to process. Once established, these a priori must be formalized and incorporated as constraints in the segmentation process. It is worth noting that the majority of the constraints used refer to the shape of the objects to segment.

These two types of information - image properties and a priori constraints - must then be integrated into a common formalism, itself numerically implemented as an algorithm. The importance of image segmentation research, highlighted above, has led to the development of many approaches, such as active contours, active shape models (ASM), approaches by classification, Markov fields, etc. We will focus in this chapter on one of the most important approaches in cardiac and thoracic imaging, namely deformable models.

1.3. Summary of the different classes of deformable models

Deformable models constitute a dominant approach to segmentation. They were originally introduced by Kass et al. [KAS 88] with the "snakes model" and quickly found applications in medical imaging. This significance relates to the fact that their formulation is very flexible, allowing the integration of many types of image properties and a priori constraints. As such, the literature concerning deformable models is highly significant and in this introductory section, we consider very synthetically two broad classes of approaches:

1. - energy-based approaches: the energy reflecting the properties of the object to segment (gray levels, shape, etc.) and expressed in terms of the deformable model (position and shape) is built. The segmentation process then corresponds to the minimization of this energy;
2. - in contrast, "non-energy-based approaches" do not involve energy directly dependent on the model. It should be noted that if some of these methods make use of a criterion minimization stage, it is therefore not expressed directly as a function of the deformable model (thus, for example, "atlas approaches" perform a registration step by minimizing a similarity criterion).

Following this section, we will detail more particularly two deformable model approaches: deformable templates (DTs) in section 1.4 and variational active contours in section 1.5.

1.3.1. Non-energy approaches

1.3.1.1. Active shape models

ASMs were originally described by Cootes in 1995 [COO 95]. This approach can be seen as a method of deformable models incorporating intrinsically an a priori on the shape of the object to segment, this a priori being built using a statistical representation of the space of the eligible shapes.

In practice, this representation is constructed from a training set of images, where contours are manually plotted, aligned and sampled on N points. This step enables the construction of a model of distribution of contour points from which shape statistics are established by using principal component analysis (PCA), which provides the average shape and the K main variation modes of this shape. The object to segment is then detected by iteratively deforming an initial contour: each of the N points of this outline is shifted in order to move it closer to the edge of highest amplitude located in its neighborhood. This set of displacements provides a new set of points that is projected onto the K main variation modes: the new shape obtained is thus forced to belong to the space of eligible shapes defined by these modes. This process is iterated until convergence, namely when the displacements can be considered as negligible.

Active appearance models (AAMs) constitute an extension of the ASMs [COO 01]. In this approach, the constraint concerns not only the shape but also the appearance, defined as the average and the principal variation modes of the normalized gray levels of the region corresponding to the reference contours. An example of the application of this technique in echocardiography can be found in [BOS 02].

1.3.1.2. Atlas-based approaches

The basic principle of atlas-based segmentation is conceptually simple. An atlas corresponds to a pair made of an image of a given modality and its segmentation, represented by a set of labeled regions. This segmentation is most often obtained by performing a manual outline. The segmentation of a new image of the same modality is then performed in two stages. The atlas image and the new image are first mapped using a registration algorithm, which uses the local properties of these images (from gray levels). Thus, this registration phase provides, on output, the transformation that allows us to map the "atlas image" to the new image. This transformation is then applied to the labeled regions of the atlas, thus providing the new image segmentation.

Within this framework, the different approaches of atlas segmentation are distinguished by the type of registration used, namely by the type of transformation (affine, rigid, nonlinear, etc.) and the similarity measure (absolute differences, mutual information, etc.) implemented in the algorithm. Another important feature lies in the construction and use of the atlas: if the base method considers a single atlas, a number of authors have proposed to...