Chapter 1
Computer Vision in Chemical Reaction Monitoring and Analysis

Marc Reid*

UKRI Future Leaders Fellow, Department of Pure & Applied Chemistry, University of Strathclyde, Glasgow, UK

*Corresponding author: marc.reid.100@strath.ac.uk

1.1 Introduction

In February 2015, the Internet was consumed by an argument about fashion. A particular photograph of a particular dress was showing up wherever you might have scrolled on social media. Hanging on the rail of a shop, captured under a stream of daylight, this dress appeared, to many, to be white and gold. Yet, as if to represent the entire other half of the online community, others saw the same photo of the same dress as being blue and black. To this day, you can share that same photograph with unsuspecting live audiences, and by a show of hands, will recreate the eerie bipartisan opinion regarding the colors of the dress. But who was correct? Is the dress white and gold, blue and black, or some other combination in between?

The dress is, in fact, blue and black. While the consensus on the root cause of the debate is not clear [1], psychological variation in human perception of colors, and our predicted corrections of those colors, likely plays a role. An illustration of the phenomenon for the reader of this book lies on the right side of Figure 1.1. For the four regions of interest (ROI) highlighted, Table 1.1 shares the average colors expressed in terms of mathematical color models we will discuss in the next section. For now, note that, for regions A and D, the numerical expressions of their colors are not identical for this pair of regions. However, for regions B and C, each showing the same apron under a warm and cold shade of light, show identical numerical expressions of their color, despite what your brain may tell you is different when judging these colors by eye.

Figure 1.1 Left: the original dress image that sparked the Internet debate. Right: cartoon variant of the dress, shows changes in likely color perception under different light sources. The four individual square ROI (left to right on the cartoon) mark areas A-D described in Table 1.1.

Table 1.1 Average RGB, HSV, and CIE-L*a*b* pixel values for four 120 × 120 square ROI on the cartoon apron, under each light source shown in Figure 1.1.

a. All ROI represent 120 × 120 pixels.

While the story of the dress is not meant to set this chapter up as one about fashion history, it instead serves to demonstrate the subjectivity of our human perception of all things color. Furthermore, we see that the quantification of color, exemplified in Table 1.1, begins to reveal the more objective expression of color that lies "under the hood" of how it is captured by a camera. This subjective-to-objective shift is vital to enable a trustworthy and exploitable analysis of color-based phenomena.

Instead of formalwear, we might instead be concerned about the rate at which a palladium catalyst degrades into its notorious nightshade, palladium black. Equally, we could be making a note of the curious colors emerging from a settling bilayer, the dissolution of material in a new solvent, or capturing the characteristic colors emitted from a library of fluorescent toxins.

1.2 Fundamentals of Computer Vision in Chemistry

1.2.1 Color Theory

Color theory is a crucial concept for chemists, enabling them to actively monitor reactions and observe changes over time. With the widespread availability of imaging devices, particularly in the age of social media, capturing image data from experiments has become more accessible. This convenience, however, brings the challenge of encoding color data accurately, a problem rooted in the rich history of color theory.

The foundation of modern color theory lies in the Young-Helmholtz theory, or trichromatic theory, developed in the nineteenth century.*1 Initially proposed by Thomas Young, this theory suggested that the human eye has three types of photoreceptors. Hermann von Helmholtz later expanded on this idea, identifying these photoreceptors as sensitive to blue, green, and red light (more appropriately referred to as "short," "medium," and "long" wavelengths, given that the peak and distribution in response for each receptor span more wavelengths than one specific color label). Tristimulus color theory is fundamental to understanding how colors are perceived and represented [2]. Several key concepts expand on this foundation, including color-matching functions, spectral reflectance, illuminants, and tristimulus values.

Color matching functions are critical in converting the spectral power distribution of light into tristimulus values. These functions, denoted as , , and , describe the sensitivity of the human eye to different wavelengths of light. They are derived from experimental data on how humans perceive colors and are standardized by the International Commission on Illumination (CIE). Mathematically, the tristimulus values , , and can be calculated using the color-matching functions described in Eqs. (1.1)-(1.3):

where is the spectral power distribution of the light source. This, in turn, relates to spectral reflectance, which is a measure of how much light is reflected by a surface at each wavelength. It is defined as the ratio of reflected light to incident light and varies with the wavelength. The spectral reflectance function, , characterizes the color of a surface, in a relationship with the spectral power distribution of the light source (or illuminant), , according to Eq. (1.4). The visual intuition for arriving at XYZ color values is shown in Figure 1.2.

Figure 1.2 Visual representation of the calculation of the tristimulus XYZ values.

The color spaces, several of which are introduced below and first hinted at in Table 1.1, allow us to conceptualize colors in an exploitable numerical framework (think back to the dress conundrum). Understanding and utilizing these color spaces is essential for accurately capturing and interpreting the color data in chemical case studies in later sections of this chapter and beyond. Up front, there are two common factors of color spaces that cater toward intuitively quantifying what a human observer may perceive perceptual uniformity and visualization.

Perceptual uniformity means that equal distances between colors in the color space correspond to equal perceived differences in color. In other words, if two colors are separated by the same amount in this space, they will look equally different to the human eye. This makes the color space more consistent with how we actually perceive colors, even though it might not be a simple linear mathematical relationship. A linear space means that the points in the color space (i.e., the vectors) can be added and multiplied (or scaled) correctly and will result in another color in that space. When taking the Euclidean distance between two points in a color space, perceptual uniformity means that the distance is the same as the difference in how a human perceives color. Visualization (in this context) means the ability of a human observer to look at the individual elements of the color vector in a color space and understand what the element represents. These two factors account for a color space that is easily interpretable by a human but are not necessary for a color space to be functional (e.g., RGB spaces, while not having these factors are commonly used in capture devices).

1.2.1.1 RGB Color Spaces

The RGB color spaces (linear RGB, sRGB, Adobe RGB, and more) are among the most widely used color models, especially in digital imaging, computer graphics, and electronic displays. These models are based on the additive color theory, where colors are created by combining different intensities of red, green, and blue light. The additivity of red, green, and blue light, along with exemplar 8-bit color coding in RGB, is shown in Figure 1.3.

Figure 1.3 Top left: a demonstration of the additive properties of overlapping red (R), green (G), and blue (B) lights. Bottom: an overview of exemplar RGB values and their geometric positioning.

Certain RGB color spaces are device-dependent, meaning that the colors displayed can vary across different devices due to variations in hardware characteristics, such as screens or printers. Examples of such device-dependent spaces include those used in monitors, cameras, or printers. However, there are also device-independent RGB spaces, like sRGB, which aim to ensure consistent color reproduction across different devices by standardizing the color transformations. While this variation among devices may seem to initially be a major drawback of using RGB, the ability to transform between color spaces leads to color models that mitigate such...