Color in Computer Vision
Description
More details
Other editions
Additional editions
Persons
Content
Preface xv
1 Introduction 1
1.1 From Fundamental to Applied 2
1.2 Part I: Color Fundamentals 3
1.3 Part II: Photometric Invariance 3
1.4 Part III: Color Constancy 4
1.5 Part IV: Color Feature Extraction 5
1.6 Part V: Applications 7
1.7 Summary 9
PART I Color Fundamentals 11
2 Color Vision 13
2.1 Introduction 13
2.2 Stages of Color Information Processing 14
2.3 Chromatic Properties of the Visual System 18
2.4 Summary 24
3 Color Image Formation 26
3.1 Lambertian Reflection Model 28
3.2 Dichromatic Reflection Model 29
3.3 Kubelka-Munk Model 32
3.4 The Diagonal Model 34
3.5 Color Spaces 36
3.6 Summary 44
PART II Photometric Invariance 47
4 Pixel-Based Photometric Invariance 49
4.1 Normalized Color Spaces 50
4.2 Opponent Color Spaces 52
4.3 The HSV Color Space 52
4.4 Composed Color Spaces 53
4.5 Noise Stability and Histogram Construction 58
4.6 Application: Color-Based Object Recognition 64
4.7 Summary 68
5 Photometric Invariance from Color Ratios 69
5.1 Illuminant Invariant Color Ratios 71
5.2 Illuminant Invariant Edge Detection 73
5.3 Blur-Robust and Color Constant Image Description 74
5.4 Application: Image Retrieval Based on Color Ratios 77
5.5 Summary 80
6 Derivative-Based Photometric Invariance 81
6.1 Full Photometric Invariants 84
6.2 Quasi-Invariants 101
6.3 Summary 111
7 Photometric Invariance by Machine Learning 113
7.1 Learning from Diversified Ensembles 114
7.2 Temporal Ensemble Learning 119
7.3 Learning Color Invariants for Region Detection 120
7.4 Experiments 124
7.5 Summary 134
PART III Color Constancy 135
8 Illuminant Estimation and Chromatic Adaptation 137
8.1 Illuminant Estimation 139
8.2 Chromatic Adaptation 141
9 Color Constancy Using Low-level Features 143
9.1 General Gray-World 143
9.2 Gray-Edge 146
9.3 Physics-Based Methods 150
9.4 Summary 151
10 Color Constancy Using Gamut-Based Methods 152
10.1 Gamut Mapping Using Derivative Structures 155
10.2 Combination of Gamut Mapping Algorithms 157
10.3 Summary 160
11 Color Constancy Using Machine Learning 161
11.1 Probabilistic Approaches 161
11.2 Combination Using Output Statistics 162
11.3 Combination Using Natural Image Statistics 163
11.4 Methods Using Semantic Information 167
11.5 Summary 171
12 Evaluation of Color Constancy Methods 172
12.1 Data Sets 172
12.2 Performance Measures 175
12.3 Experiments 180
12.4 Summary 185
PART IV Color Feature Extraction 187
13 Color Feature Detection 189
13.1 The Color Tensor 191
13.2 Color Saliency 205
13.3 Conclusions 218
14 Color Feature Description 221
14.1 Gaussian Derivative-Based Descriptors 225
14.2 Discriminative Power 229
14.3 Level of Invariance 235
14.4 Information Content 236
14.5 Summary 243
15 Color Image Segmentation 244
15.1 Color Gabor Filtering 245
15.2 Invariant Gabor Filters Under Lambertian Reflection 247
15.3 Color-Based Texture Segmentation 247
15.4 Material Recognition Using Invariant Anisotropic Filtering 249
15.5 Color Invariant Codebooks and Material-Specific Adaptation 256
15.6 Experiments 258
15.7 Image Segmentation by Delaunay Triangulation 263
15.8 Summary 268
PART V Applications 269
16 Object and Scene Recognition 271
16.1 Diagonal Model 272
16.2 Color SIFT Descriptors 273
16.3 Object and Scene Recognition 276
16.4 Results 280
16.5 Summary 285
17 Color Naming 287
17.1 Basic Color Terms 288
17.3 Color Names from Uncalibrated Data 304
17.4 Experimental Results 313
17.5 Conclusions 316
18 Segmentation of Multispectral Images 318
18.1 Reflection and Camera Models 319
18.2 Photometric Invariant Distance Measures 321
18.3 Error Propagation 325
18.4 Photometric Invariant Region Detection by Clustering 328
18.5 Experiments 330
18.6 Summary 338
Citation Guidelines 339
References 341
Index 363
Preface
Visual information is our most natural source of information and communication. Apart from human vision, visual information plays a vital and indispensable role in society and is the nucleus of current communication frameworks such as the World Wide Web and mobile phones. With the ever-growing production, use, and exploitation of digital visual information (e.g., documents, websites, images, videos, and movies), a visual overflow will occur, and hence demands are urgent for the (automatic) understanding of visual information. Moreover, as digital visual information is nowadays available in color format, there is the irreversible necessity for the understanding of visual color information. Computer vision deals with the understanding of visual information. Although color became a central topic in various disciplines (ranging from mathematics and physics to the humanities and art) quite early on, in the field of computer vision it has emerged only recently. We take on the challenge of providing a substantial set of tools for image understanding from a color perspective. The central topic of this book is to present color theories, representation models, and computational methods that are essential for image understanding in the field of computer vision.
The idea to make this book was born when the authors were sitting on a terrace overlooking the Amstel River. The rich artistic history of Amsterdam, the river, and that sunny day gave us the inspiration for discussing the role of color in art, in life, and eventually in computer vision. There, we decided to do something about the lack of textbooks on color in computer vision. We agreed that the most productive and pleasant way to reflect our findings on this topic was to write this book together. A book in which color is taken as a valuable collaborative source of synergy between two research fields: color science and computer vision. The book is the result of more than 10 years of research experience of all four authors who worked closely together (as PhDs, postdocs, professors, colleagues, and eventually friends) on the same topic of color computer vision at the University of Amsterdam. Because of this long-term collaboration among the authors, our research on color computer vision is a tight connection of color theories, color image processing methods, machine learning, and applications in the field of computer vision, such as image segmentation, understanding, and search. Even though many of the chapters in the book have their origin as a journal article, we ascertained that our work is rewritten and trimmed down. This process, the long-term collaboration, and many discussions resulted in a book in which a uniform style has emerged and in which the material represents the best of us.
The book is a valuable textbook for graduate students, researchers, and professionals in the field of computer vision, computer science, color, and engineering. The book covers upper-level undergraduate and graduate courses and can also be used in more advanced courses such as postgraduate tutorials. It is a good reference for anyone, including those in industry, interested in the topic of color and computer vision. A prerequisite is a basic knowledge of image processing and computer vision. Further, a general background in mathematics is required, such as linear algebra, calculus, and probability theory. Some of the material in this book has been presented as part of graduate and postgraduate courses at the University of Amsterdam. Also, part of the material has been presented at conference tutorials and short courses at image processing conferences (International Conference on Image Processing (ICIP) and International Conference on Pattern Recognition (ICPR)), computer vision conferences (Computer Vision and Pattern Recognition (CVPR) and the International Conference on Computer Vision (ICCV)), and color conferences (Colour in Graphics, Imaging, and Vision (CGIV) and conferences organized by the International Society for Optics and Photonics (SPIE)). Computer vision contains more topics than what we have presented in this book. The emphasis is on image understanding. However, the topic of image understanding has been taken as the path along which we were able to present our work. Although the material represents our view on color in computer vision, our sincere intention was to include all relevant research. Therefore, we believe this book is one of the first extensive works on color in computer vision to be published with over 360 citations.
This book consists of five parts. The topics range from (low-level) color image formation to (intermediate-level) color invariant feature extraction and color image processing to (high-level) semantic descriptors for object and scene recognition. The topics are treated from low-level to high-level processing and from fundamental to more applied research. Part I contains the (color) fundamentals of the book. This part presents the concept of trichromatic color processing and the similarity between human and computer vision systems. Furthermore, the basics are provided on the color image formation. Reflection models that describe the imaging process, the interplay between light and matter, and how photometric conditions influence the RGB values in an image are presented. In Part II, we consider the research area of extracting color invariant information. We build detailed models of the color image formation process and design mathematical methods to infer the quantities of interest. Pixel-based and derivative-based photometric invariance are discussed. An overview is given on the computation of both photometric invariance and differential information. Part III contains an overview on color constancy. Computational methods are presented to estimate the illumination. An evaluation of color constancy methods is given on large-scale datasets. The problem of how to select and combine different methods is addressed. A statistical approach is taken to quantify the priors of unknowns in noisy data to infer the best possible estimate of the illumination from the visual scene. Feature detection and color descriptors are discussed in Part IV. Color image processing tools are provided. An algebraic (vector-based) approach is taken to extend scalar-signal to vector-signal processing. Computational methods are introduced to extract a variety of local image features, such as circle detectors, curvature estimation, and optical flow. Finally, in Part V, different applications are presented, such as image segmentation, object recognition, color naming, and image retrieval.
This book comes with a large amount of supplementary material, which can be found at
Here you can find
- Software implementations of many of the methods presented in the book.
- Datasets and pointers to public image datasets.
- Slides corresponding to the material covered in the book.
- Slides of new material presented at tutorials at conferences.
- Pointers to workshops and conferences.
- Discussions on current developments, including latest publications.
Our policy is to make our software and datasets available as a contribution to the research community. Also, in case you want to share your software or dataset, please drop us a line so we can add a pointer to it on our website. If you have any suggestions for improving the book, please send us an e-mail. We want to keep the book accurate as much as possible.
Finally, we thank all the people who have worked with us over the years and shared their passion for research and color with us.
Arnold Smeulders at the University of Amsterdam is one of the best researchers we had the opportunity to work with. He was heading the group during the time we paved the way for this book. His insatiable passion for research and lively debates have been a source of inspiration to all of us. We enjoyed working with him.
We are very grateful to Marcel Lucassen who contributed Chapter 2 to this book. Furthermore, his thorough proofreading and enthusiasm were indispensable for the quality of the book. It is a fortune to have him as a human (color) vision scientist amidst us. It was certainly a pleasure to work with him. We are indebted to Jan van Gemert for his proofreading and Frank Aldershoff for LaTeX and Mathematica issues.
We are also grateful to NWO (Dutch Organisation for Scientific Research), who granted Theo Gevers with a VICI (#639.023.705) with the same title of this book “Color in Computer Vision” and Jan-Mark Geusebroek with a VENI. These grants were valuable for this book.
While working at the University of Amsterdam, we had the opportunity to collaborate with many wonderful colleagues. We want to thank Arnold Smeulders for his work on Chapters 6 and 13, Rein van de Boomgaard for Chapter 6, Gertjan Burghouts for Chapters 14 and 115, Koen van de Sande and Cees Snoek for their help on Chapter 16, and Harro Stokman for Chapter 18. Furthermore, we thank the following persons: Virginie Mes, Roberto Valenti, Marcel Worring, Dennis Koelma, and all other members of the ISIS group.
At the Computer Vision Center (Universitat Autònoma de Barcelona), we thank José Álvarez and Antonio López for their contribution to Chapter 7. Further, we are indebted to Robert Benavente, Maria Vanrell, and Ramon Baldrich for their contribution to Chapter 17. At the LEAR team in INRIA rh Ône Alpes, France, we thank Cordelia Schmid, Jakob Verbeek, and Diane Larlus for their help with Chapters 5 and 17. We also appreciate the contribution of Andrew Bagdanov at the Media Integration and Communication Center in Florence, Italy. Furthermore, Joost van de Weijer acknowledges the support...
