1
Overview of Biomedical Image Understanding Methods

Wei Xiong, Jierong Cheng, Ying Gu, Shimiao Li and Joo-Hwee Lim

Department of Visual Computing, Institute for Infocomm Research, A*STAR, Singapore

Computerized image understanding is the process of extracting meaningful features (e.g., color, intensity, and geometry of group of pixels) from the images, inferring and aggregating the symbolic information into unique concepts, matching them with physical world models and producing descriptions of the images and their relationship in the world that the images represent [1]. Biomedical images are those acquired from biology, medicine, pathology, dentistry, and other specialized healthcare domains. With the advancement of modern imaging devices, enormous amounts of digital still and dynamic image data are generated from nano to macro, from protein to cells, and to organs and from animals to human. Computerized image analysis plays an important role in understanding and interpreting these images accurately and efficiently to assist biologists and clinicians in decision making. Being a highly multidisciplinary research field, biomedical image understanding requires knowledge, theories, methods, and techniques from computer science, engineering, mathematics as well as from general and specialized healthcare domains. Developments in related disciplines have rapidly advanced over the past decade. Various imaging modalities and acquiring procedures result in large differences in biomedical images.

The computerized understanding of these biomedical images requires a few or all of the following essential computational processes:

• Segmentation and object detection
• Registration and matching
• Object tracking
• Classification
• Knowledge-based systems (KBSs).

The schematic diagram in Fig. 1.1 shows the coherent relationships and functions of these basic processes. As a fundamental process in biomedical image understanding, segmentation delineates the image into meaningful regions and unique concepts. These detected regions/objects can be compared with the world models by registration and matching. When analyzing images changing with time, that is, videos, the object motion is tracked and characterized. One way is to first segment the objects and then track them by associating the segmented objects. Some particular features such as shape and context could be extracted for associating. Another way is to perform simultaneous segmentation and tracking.

Figure 1.1 Basic computational processes for image understanding.

Classification is to categorize items into subcategories, such as different attributes, and so on. The output of classification is their labels of different properties. After segmentation, the features, regions, objects, and/or their motions (determined by tracking) may also be further categorized into subclasses. The object motions tracked can also be further classified into different types to enhance the understanding of the deformation and velocity fields in the image. In classifier- or cluster-based segmentation methods, image pixels are grouped into foreground or background and thereby form segments of regions in the image. In such cases, classification and segmentation are processed simultaneously.

Besides segmentation, another fundamental process for the understanding is registration (or matching), which means to align two components for comparisons. Comparing with the world models generates descriptions of similarities and dissimilarities. Registration may not need an explicit clearcut region delineation as input. It may also be used during segmentation, such as atlas construction and multimodal segmentation. Registration may be processed in constituent component levels in images and the detected components come from segmentation or classification.

Segmentation, tracking, and classification involve geometric, structural, and functional features, regions, or objects extracted from the image/video. These features may be from different spaces, represented differently, explicitly, or implicitly.

Whenever necessary and available, knowledge can always be helpful to assist these computation processes. It may be used to initialize a computation, to constrain solution boundaries, to provide feedback on solution feasibility, or as a standard to compare with, and so on. Knowledge could be either prior knowledge or learned during the computation. With prior knowledge, the matching of the above-mentioned symbolic information with world models can be faster, more accurate, more targeted, and/or more robust. Similarity/dissimilarity and labels of objects and their context against the world models in terms of geometry positions, structures, relations, and functions provide primary understanding of the image and its components. Semantic understanding of biomedical images requires the comparisons and matchings with specific domain concepts, models, and knowledge.

In the following sections, we review the above-mentioned essential computational methods and their latest and important applications for the understanding of biomedical images/videos.

1.1 Segmentation and Object Detection

Image segmentation is the process of partitioning an image into nonoverlapping, constituent regions that have homogeneous characteristics such as intensity or texture [2]. Let be the image domain, the segmentation problem is to determine a set of connected subsets that satisfy with when .

The purposes of segmentation in biomedical images are mainly [3]

• identifying region of interest (ROI);
• measuring organ/tumor volume;
• studying anatomical structure;
• treatment/surgical planning;
• cell counting for drug effect study.

We classify the medical image segmentation methods (Table 1.1) according to Reference [4].

Table 1.1 Taxonomy of Segmentation

source:From Reference [4]

1.1.1 Methods Based on Image Processing Techniques

Methods based on image processing techniques have three general categories: thresholding, edge-based methods, and region-based methods. When the ROI or object has homogeneous intensity against a background of different gray levels, one or multiple thresholds can be applied on an image histogram to segment the object from background. Edge-based segmentation relies on the assumption that boundaries between objects are represented by edges, that is, discontinuities in gray level [3]. The discontinuities are usually detected by operators that approximate gradient or Laplacian computation and then used as features in subsequent processes. The performance of various edge-based segmentation approaches was compared in Reference [8].

Region-based segmentation is based on the principal of homogeneity-pixels within each object have similar visual properties [3]. Region growing is a segmentation method that uses a bottom-up strategy. In region growing method [9], a set of seed points are required to initialize the process. Regions are grown iteratively by merging unallocated neighboring pixels depending on a merging criterion. Region growing is usually used in the segmentation of small or simple structures in medical images such as posterior fossa in fetal brain [10], aorta [11], and myocardial wall [12]. Split-and-merge is an algorithm related to region growing, but does not need seed points.

Watershed algorithm [41] is also a region-based segmentation method. It considers the gradient of a grayscale image as a topological relief, where the gray levels represent altitude of the relief. When this relief is flooded from regional minima, the set of barriers built, where adjacent catchment basins meet, is called watershed. To handle the problem of potential oversegmentation, region...