1.

Introduction

The concentrations of chromophores in skin provide important information for making a diagnosis, and images can provide additional information, such as the size of a lesion, that aid visual understanding. Therefore, many studies have been conducted to evaluate the concentration of chromophores,^1–6 and some of these studies discuss imaging, in which visible light spectroscopy is usually used to achieve high spatial resolution.^7–13 Multiple regression analysis (MRA) based on the Beer–Lambert (B–L)³ law is a simple way to correlate the chromophore concentrations with the reflectance of skin. The B–L law has been applied to the three-layer skin model of epidermis, dermis, and fat, with the assumptions that no scattering occurs in the epidermis or dermis and no wavelength dependency exists in the reflectance from the fat layer.³ With these assumptions, the optical path length becomes constant and does not depend on wavelength. Therefore, in accordance with the B–L law, the absorbance (the logarithm of the inverse of the reflectance) can be expressed as a linear combination of the absorption coefficient spectrums of melanin, oxygenated hemoglobin, and deoxygenated hemoglobin, and a constant term. The constant term comes from the reflectance of the fat layer. The regression analysis uses the absorbance spectrum as a response variable and the extinction coefficient of each of the chromophores as the predictor variables. Although the implementation is easy, the model is quite different from actual conditions, and it leads to inconsistencies between the actual and the fitted spectrums.^4,14,15

To improve the estimation, the modified B–L method has been introduced.^4,5 Although the method is based on the B–L law, the optical path length is allowed to be wavelength dependent. This added degree of freedom allows a part of the scattering effect to be included, and it improves the coefficient of determination.^4,5 In this method, as the predictor variables in the MRA, the product of each absorption coefficient and the wavelength-dependent path length was used. However, the method assumes that the optical path length is independent of the concentrations of chromophores, and thus the results are still different from measurements of the actual phenomenon. Neither the nonlinearity between the concentration and the absorbance nor the cross-talk between the chromophores has been taken into account.^1,14,15

In order to fully account for the effect of scattering in the estimation of chromophore concentrations, several methods have been considered; these include methods based on random work theory,¹² Kubelka–Munk theory,⁸ a support vector machine,¹¹ and an empirical method aided by a Monte Carlo simulation (MCS).^1,9 Although scattering effects are fully considered in these methods, the methods are susceptible to shading-derived errors caused by the incomplete estimation of the irradiance.^8,11,12 This is a serious problem for visual understanding and quantitative evaluation of the chromophore imaging. Here, “shading” is the spatial distribution of irradiance. Shading results from the incident angle of the light, the spectral radiant intensity of the light source, and the distance from the light source. If the actual irradiance on the surface of a subject is lower than the estimate, then the apparent reflectance decreases and the chromophore concentrations will be overestimated; this tends to occur on the fringe areas of the subject. Therefore, for a precise calibration of reflectance, besides the radiant intensity distribution of the light, the shape of the subject also must be assessed (unless the areas of interest are limited to small, flat areas in which the irradiance can be fully assessed).¹ The problem with shading is common among methods that consider nonlinear scattering effects in imaging.¹⁰

Several studies have attempted to correct for the shading effect. In these studies, the shape of each subject was measured^13,16 or the curvature effect was otherwise extracted¹² and taken into account. However, these approaches require hardware modifications and additional measurements or else assumptions about the shapes of the samples.

Incidentally, in the simple and modified B–L methods, in which the absorbance is expressed as a linear combination of each chromophore component with a constant term, the shading-derived error is not observed in the images of chromophore concentrations. The effect of surface curvature is proportional to irradiance, and in the absorbance, it is a constant term, independent of wavelength. Therefore, the effect of surface curvature will appear only in the constant term and will not appear in the image of the chromophore concentrations.

In order to fully account for the scattering effect and to reduce the shading-derived artifact, we propose an approach based on the MCS-aided empirical method.^1,9 In this method, the simple B–L method is used to derive multiple regression coefficients related to the concentrations of melanin, oxygenated hemoglobin, and deoxygenated hemoglobin. Then, the multiple regression coefficients are converted to the concentrations of melanin and total hemoglobin (sum of oxygenated and deoxygenated hemoglobin) using compensation formulae (CF), which are derived from the diffuse reflectance spectra of human skin, numerically calculated in advance by a MCS.¹⁷ In the previous method,^1,9 the constant term of the multiple regression coefficients, as well as the regression coefficients of melanin and total hemoglobin, were used in the CFs. Although the use of the constant term is important for estimating the absolute values of chromophore concentrations, it can cause the shading-derived error in reflectance that results in an overestimation of the chromophore concentrations. Since the shading-derived error is concentrated into the constant term during the first step of the MRA, selecting the optimal combination of multiple regression coefficients has the potential to satisfy both of the requirements. In vivo experiments with a human forearm were performed to confirm the usefulness of the proposed approach for improving the robustness of the method against the estimation error in the irradiance.

2.

Materials and Methods

2.3.

Shading Effect

Shading has an effect on the results of MRA1, and thus consequently, on the estimated chromophore concentrations. Assuming Lambertian reflection and using ${\theta }_i$ to denote the angle between the surface and the incoming light, the irradiance is $\cos {\theta }_i$ times that of normal incidence, independent of wavelength. Therefore, the apparent reflectance can be expressed as $R^{\prime }=R\times \cos {\theta }_i$, where the actual reflectance is defined as $R$. Also, the apparent absorbance $A^{\prime }$ can be expressed as $A^{\prime }=\log (1/R^{\prime })=A+\log (1/\cos {\theta }_i)$, where the actual absorbance is $A=\log (1/R)$. Since the second term is independent of the wavelength, $a_0$ becomes $\log (1/\cos {\theta }_i)$ larger than it is in the case of normal incidence.¹⁰

We assumed that a light, a camera, and a cylindrical subject of radius $r$ are arranged as follows. The cylinder is placed so that the center is at the origin and the axis of symmetry lies along the $x$-axis, the camera lies on the $z$-axis, and the light source is in the $zx$-plane (Fig. 1). The light vector $\overrightarrow{S}$ can be expressed as $(\sin {\theta }_S,0,\cos {\theta }_S)$ at any point on the subject. Similarly, the normal vector $\overrightarrow{N}$ at the point $(x_0,y_0)$ is $(0,y_0/r,\sqrt{1-y_0^2/r^2})$. Therefore, the incident angle (the angle between $\overrightarrow{S}$ and $\overrightarrow{N}$) is ${\cos }^{-1}(\cos {\theta }_S·\sqrt{1-y_0^2/r^2})$. Under this arrangement, the incident angle depends on the $y$-coordinate and increases with increasing distance from the $x$-axis, so $a_0$ also increases. The intensity of the reflected light is expected to depend less on the viewing angle ${\theta }_o$ than on the incident angle, because, in the case of Lambertian reflection,²⁴ the observation point does not affect the intensity of the reflected light.

Fig. 1

Notation of positional and angular relationships for a point on a cylindrical subject in relation to a light source and a camera. Each symbol represents $\overrightarrow{S}$: light vector, $\overrightarrow{N}$: normal vector, $\overrightarrow{V}$: viewing vector, ${\theta }_i$: incident angle, ${\theta }_o$: viewing angle, ${\theta }_S$: angle between $\overrightarrow{S}$ and $\overrightarrow{V}$, $r$: radius of the cylindrical subject, ($x_0,y_0$): coordinate value of a point on the subject.

2.6.

Measuring Device

The following apparatus was prepared (Fig. 2). A camera ($\mathrm{CV}\text{-}M7+\mathrm{CL}$; jAi, Yokohama, Japan) was set in front of the subject. A liquid-crystal wavelength-tunable filter (VariSpec; CRi, Hopkinton, Massachusetts; full width at half maximum: 10 nm) and a polarizer were set in front of the camera. Two halogen lights were set at the left and right sides of the camera in such a way that the angle between the camera and the lights was 30 deg. An infrared cut filter and a polarizer, which was set to be orthogonal to the polarizer in front of the camera, were set between each light source and the subject; this was done to reduce the specular reflection from the skin surface.¹⁰

Fig. 2

Schematic of the measuring system.

Images were captured using a personal computer and by varying the transparent wavelength of the VariSpec. The resolution of the camera was $4\text{pixel}/\mathrm{mm}$ on a subject facing frontwards, and the pixel depth was 10 bits. The transparent wavelength was set from 500 to 600 nm, with an interval of 20 nm. The measurement time was about 10 s in total. Although we used a color camera, we used only the green channel. For calibration, we captured images of a gray card besides images of subjects, and the pixel values were converted to reflectance in a pixel-by-pixel manner. In the calibration, the pixel value of the white chart was also used, and it was set below the subject. The reflectance can be derived from this calibration if the surface of interest is perpendicular to the line of sight. However, if the surface of interest is not perpendicular to the line of sight, the shading-derived error in the reflectance, i.e., the calibration error in the reflectance, will be increased according to $\cos {\theta }_i$, as shown in Fig. 1.

3.

Results

3.1.

Correlation between Estimated and Expected Value of Chromophore Concentrations

For CF1, CF2, and CF3, the relationships between the original $C_i$ and the estimated $c_i$ ($i=m$, th) are shown in Fig. 3. The $\sigma$ were 0.025, 0.481, and 0.084 for the melanin concentration estimated with CF1, CF2, and CF3, respectively, and 0.008, 0.092, and 0.016 for the hemoglobin concentration measured with CF1, CF2, and CF3, respectively. The errors increased in the order of CF1, CF3, and CF2.

Fig. 3

Correlations between the original and the estimated values of (a) melanin and (b) hemoglobin, using CF1; (c) melanin and (d) hemoglobin, using CF2; and (e) melanin and (f) hemoglobin, using CF3. Horizontal axis: original values; vertical axis: estimated values. The unit of the horizontal and vertical axes is %.

3.2.

Measurement of Forearm Skin

Images of the inner forearm from sets of spectra images are shown in Fig. 4, and Fig. 5 shows the images of the regression coefficients of the framed area in Fig. 4(a). Although the systematic variation from the center to the fringe (upper and lower sides in the pictures) cannot be found in $a_m$ [Fig. 5(a)] or $a_{\mathrm{th}}$ [Fig. 5(b)], the value of $a_0$ became greater at the fringe [Fig. 5(e)]. In $a_{\mathrm{oh}}$ [Fig. 5(c)] and $a_{\mathrm{dh}}$ [Fig. 5(d)], the values became slightly larger and smaller at the edges, respectively, but the degree to which this happens is smaller than it was for $a_0$. The value of $a_{\mathrm{StO}2}$ was affected by the trend of $a_{\mathrm{oh}}$ and $a_{\mathrm{dh}}$ and decreased slightly at the fringe [Fig. 5(c)]. The average and standard deviation of each regression coefficient of the sites A, B, C, and D in Fig. 4(a) were calculated, and the values were as follows: $a_m=(2.1\pm 0.2)\times {10}^{-4}$; $a_{\mathrm{th}}=(1.6\pm 0.1)\times {10}^{-6}$; $a_{\mathrm{oh}}=(8.5\pm 0.9)\times {10}^{-7}$; $a_{\mathrm{dh}}=(7.6\pm 1.1)\times {10}^{-7}$; $a_0=(2.2\pm 0.2)\times {10}^{-1}$; and $a_{\mathrm{StO}2}=(55\pm 5)\%$.

Fig. 4

Representative example of measured images (appearance) from different angles. The inner side of the forearm faced (a) straight or (b) upward. The hand is to the left side. To correlate the positions, points were marked in a reticular pattern. The symbols drawn on the pictures are the measured points for Fig. 8. The framed area in (a) is the area shown in Figs. 5 and 6.

Fig. 5

Representative example of images of the multiple regression coefficients: (a) $a_m$; (b) $a_{\mathrm{th}}$ ($=a_{\mathrm{oh}}+a_{\mathrm{dh}}$); (c) $a_{\mathrm{oh}}$; (d) $a_{\mathrm{dh}}$; (e) $a_0$; and (f) $a_{\mathrm{StO}2}$ ($=a_{\mathrm{oh}}/a_{\mathrm{th}}$). The area of the images is the area surrounded by a border in Fig. 4(a). The background is masked in black. The value of each regression coefficient corresponds to the brightness. The lower limit is zero for all images, and the upper limit (“Max”) is $5\times {10}^{-4}$ for $a_m$, $4\times {10}^{-6}$ for $a_{\mathrm{th}}$, $1\times {10}^{-6}$ for $a_{\mathrm{oh}}$, $1\times {10}^{-6}$ for $a_{\mathrm{dh}}$, 1 for $a_0$, and 100% for $a_{\mathrm{StO}2}$. The reticular pattern is made up of points marked on the skin, and it can be clearly seen in (e).

Next, the chromophore concentrations were estimated with each CF. Figure 6 presents the images of a representative example of these. In CF1, the values of $c_m$ and $c_{\mathrm{th}}$ increased from the center to the fringe, but in CF3, no such tendency could be observed. This tendency is also seen in Fig. 7. From the cross-section, the values of CF3 near the center were close to those of CF1 [Figs. 7(a) and 7(b)]. On the other hand, the fluctuations of CF3 are larger than those of CF1. In CF2 also, a systematic variation could not be found, but the fluctuations were larger than those of CF3. The differences between these and the values with CF1 were especially observable in $c_{\mathrm{th}}$ [Fig. 7(b)]. The averages and standard deviations of the magnified area in Fig. 7 were $5.7\pm 0.6\%$, $4.7\pm 1.5\%$, and $5.1\pm 1.2\%$ for melanin with CF1, CF2, and CF3, respectively, and $0.19\pm 0.05\%$, $0.10\pm 0.08\%$, and $0.18\pm 0.05\%$ for hemoglobin with CF1, CF2, and CF3, respectively.

Fig. 6

Estimated $c_m$ and $c_{\mathrm{th}}$ with each CF. (a)–(b) with CF1: (a) $c_m$ and (b) $c_{\mathrm{th}}$; (c)–(d) with CF2: (c) $c_m$ and (d) $c_{\mathrm{th}}$; (e)–(f) with CF3: (e) $c_m$ and (f) $c_{\mathrm{th}}$. The value of each estimated concentration corresponds to the brightness; the lower limit is zero for all images, and the upper limit (“Max”) is 10% and 0.25% for the images of $c_m$ and $c_{th}$, respectively. The dotted line and the characters “$X$” and “$Y$” in (a) represent the cross-section and ends represented in Fig. 7. The background is masked in black. The reticular pattern is made up of points marked on the skin.

Fig. 7

Cross-sections of the estimated (a) $c_m$ and (b) $c_{\mathrm{th}}$, with CF1, CF2, and CF3. CF1: gray broken lines, CF2: dotted lines, CF3: solid black lines. The plots shown above are the magnified figures of the framed areas. The cross-sections were taken along the dotted lines in Fig. 6(a), and the sides $X$ and $Y$ correspond to the positions noted in Fig. 6(a). The unit of the vertical axes is %.

Finally, $c_m$ and $c_{\mathrm{th}}$ estimated from different angles at the same site were compared, and the results from a representative example are shown in Fig. 8. The differences between the values of different angles at each position with CF3 were smaller than those with CF1. The values of $\sigma$ for the representative example for melanin and hemoglobin (Fig. 8) were, respectively, 1.8% and 0.15% for CF1, 0.3% and 0.02% for CF2, and 0.3% and 0.01% for CF3. The values of $\sigma$ for melanin and hemoglobin for all sites and all subjects were, respectively, 1.3% and 0.15% for CF1, 0.4% and 0.02% for CF2, and 0.3% and 0.02% for CF3. Figure 9 shows a plot of $c_m$ and $c_{\mathrm{th}}$ for all subjects and all points for CF3.

Fig. 8

Correlation between the estimates from different angles (angles 1 and 2) of the melanin and the hemoglobin concentrations at each position of the representative example (same subject as in Fig. 4). (a)–(b): $c_m$ calculated with (a) CF1 and (b) CF3; (c)–(d): $c_{\mathrm{th}}$ calculated with (c) CF1 and (d) CF3. The symbols and legend (A to D) are the same as those used in Fig. 4. The unit of the horizontal and vertical axes is %.

Fig. 9

Correlation between the estimates from different angles (angles 1 and 2) of the melanin and the hemoglobin concentrations at each position for all subjects. (a): $c_m$; (b): $c_{\mathrm{th}}$ calculated with CF3. The diagonal solid line represents $y=x$, and the diagonal gray broken line represents $y=x\pm 1.96\sigma$, which is the 95% confidence interval assuming the $V$ follows a Gaussian distribution. The unit of the horizontal and vertical axes is %.

4.

Discussion

Under the condition that the forearm was set horizontally and the lights were set at the left and right sides of the camera, the incident angle at the upper and lower sides were greater than that at the center. Therefore, the irradiance at the surface became smaller. As we expected (see Sec. 2.3), the shading effect appeared in the $a_0$ image [Fig. 5(e)], and became greater at the fringe. As a consequence, with CF1 [Figs. 6(a) and 6(b)], since the formula contains $a_0$ [Fig. 5(e)], the effect of the shading can be seen in the estimation of chromophore concentrations. This tendency seems to be larger for hemoglobin than for melanin. With CF3 [Figs. 6(e) and 6(f)], which does not contain $a_0$, the artifact is hardly seen, as expected. The improvement is shown more clearly in Fig. 8. Each position on the subject should have the same chromophore concentrations even if the measurement angle is changed. Thus the points plotted in Fig. 8 should be on the line $x=y$. Obviously, the points plotted for CF3 were closer to $x=y$ than those for CF1, which were quantified with $\sigma$. From these results, we conclude that the estimation errors in the chromophore concentrations that are due to the shading-derived error in the reflectance, were greatly reduced by the proposed approach. With CF3, the $\sigma$ were 0.3% and 0.02% for $c_m$ and $c_{\mathrm{th}}$, and the ranges of $c_m$ and $c_{\mathrm{th}}$ were 3% to 6% and 0.15 to 0.25%, respectively (Fig. 9), which demonstrates that the ratio of the error to the chromophore concentrations is about 10%.

Near the center, where irradiance was evaluated correctly, CF3 estimated the same values, on average, as did CF1 (Fig. 7). In other words, CF3 is as accurate as CF1 when there is less of an effect of shading. Furthermore, the values are within the expected ranges of concentration: $c_m$, $c_{\mathrm{th}}$, and $a_{\mathrm{StO}2}$ for the representative example were $5.0\pm 0.7\%$, $0.17\pm 0.01\%$, and $55\pm 5\%$, respectively, which agrees with the average values for Japanese subjects as reported in the literature.^1,22

In the case of CF2 [Figs. 6(c) and 6(d)], although the shading effect was suppressed, the fluctuation was large. Moreover, the average value is very different from the value with CF1 near the center (Fig. 7). This tendency was notable for $c_{\mathrm{th}}$. This probably came from the error between the expected and estimated chromophore concentrations, which can be seen in Figs. 3(c) and 3(d).

Although in Sec. 2.2, we assumed Lambertian reflection for the angular dependency of reflection from skin as a precondition of the constancy of reflectance with varying outgoing angles, this precondition can be loosened. Lambertian reflection proved to be an inaccurate approximation for several real-world objects.²⁴ However, in the proposed method, to the extent that the angular dependency of the reflected light is proportional between wavelengths, the chromophore concentrations will not be affected by the viewing angle. This is because the multiplication of the reflectance by an independent factor of the wavelength will be included in $a_0$ in the same way that the shading effect is included in $a_0$. From the fact that the variation from the center to the fringe does not explicitly appear in the multiple regression coefficients other than $a_0$ (Fig. 5), we infer that the loosened precondition is almost true when the viewing angle increases from the center to the fringe, as does the incident angle. Nonetheless, the penetration depth seems dependent in some way besides the shading effect on the incident angle and viewing angles. At the very edge of the forearm, $a_{\mathrm{oh}}$ and $a_{\mathrm{dh}}$ became slightly smaller and larger, respectively [Figs. 5(c) and (d)], which can be explained by a shallower penetration depth with a larger incident angle. The oxygen saturation in a shallower region is expected to be lower than that in a deeper region,²⁵ which is the trend seen in the derived images.

There are some side effects from removing $a_0$ from the CFs, and these should be improved. First, although the fluctuations with CF3 were smaller than those with CF2, they were still larger than those with CF1 (Figs. 6 and 7). We tried increasing the maximum degree of CF3 from two to three, but the fluctuations did not improve (not shown). The fluctuations are due directly to $a_{\mathrm{oh}}$ and $a_{\mathrm{dh}}$, but did not appear in $a_{\mathrm{th}}$ [Fig. 5(b)]. This indicates that the noise in $a_{\mathrm{oh}}$ and $a_{\mathrm{dh}}$ should be removed in order to reduce the noise in $c_{\mathrm{th}}$. Next, the melanin images from CF3 show what seem to be veins [Fig. 6(e)]. This comes from the $a_m$ image [Fig. 5(a)], and the compensation did not seem to work well without $a_0$. Although the average values for CF3 were the same as those for CF1 (Fig. 7), this can create a serious problem in visual understanding. To reduce these problems, although the regression coefficients for the MRA1 will be converted to chromophore concentrations with CFs, maximizing the accuracy of MRA1 may be effective in terms of the propagation of errors. This means that the more linear the relationship between the regression coefficients of MRA1 and the respective chromophore concentrations, the smaller the estimation error will become. Therefore, for MRA1, employing the modified B–L method instead of the simple B–L method may be effective, because this will improve the correlation between the concentration and the respective regression coefficients of MRA1.

This technique for reducing the shading-derived error does not require any measurements or assumptions about the shape of the subjects and has no limitations as to skin type. In addition, the method is robust against the angular distribution of the reflected light, as discussed in the previous paragraph. However, the method as a whole has inherited some limitations from the base method. First, the estimated chromophore concentrations depend on the optical parameters and geometries used in the MCS. The thicknesses and scattering coefficients of the epidermis and dermis layer can vary with the individual, the site, and the condition of the skin (e.g., water content). Second, an adequate amount of light should be reflected from the dermis for a precise evaluation of the chromophore concentrations. In the case of darker skin, light is strongly absorbed in the epidermis, and the intensity of the light that reaches the dermis is weak. As a consequence, the error in the estimation of the chromophore concentration, especially of oxygenated and deoxygenated hemoglobin, will be larger than that for lighter skin.

In the case of oily skin, the specular reflection is stronger than that of nonoily skin, but this was corrected by using a polarizer in our measuring system. However, the amount of sebum will affect the scattering power of the epidermis, and thus it will also affect the estimated chromophore concentration. The existence of sebum is expected to decrease the scattering power of the surface of the epidermis and thus increase the penetration depth. This may result in an overestimation of the chromophores.

This method is very useful for the diagnoses of skin spots or lesions, especially when they are so large that the surface curvature becomes non-negligible; an example of this is a port-wine stain.²⁶ Also, in aesthetic evaluations in dermatology and cosmetology, evaluations of each body part (especially the face) as a whole are important.²⁷ The robustness of this method against fluctuations in the intensity of the light source provide a practical advantage. In practical applications, the intensity of the light source is sometime not stable; for example, when using a flashbulb. However, to the extent that the spectrum of the light source is proportional, the derived chromophore concentrations will not be affected, since the effect is gathered into $a_0$ in MRA1. This method is not limited to imaging: it can also be used for point measurements when the stability of the light source intensity cannot be guaranteed. On the other hand, the method cannot be applied when wavelength-dependent information cannot be utilized, such as for the derivation of the absorption and reduced scattering coefficient ${\mu }_a$ and ${\mu }_s^{\prime }$.^16,28 In this case, the effect of shading will also cause errors in the estimated concentrations of chromophores.

5.

Conclusion

In this article, we presented a method for quantitative evaluation of skin chromophores over wide regions. We achieved the reduction of shading-derived artifact by eliminating the constant term of the B–L method in the conversion of the regression coefficients to concentrations. This method is robust to errors in estimating the irradiance and also accounts for the scattering effect. Measurements of a forearm confirmed that, by optimizing the CF, the proposed approach can dramatically reduce the shading-derived artifacts and fully account for the scattering effect. The proposed approach will enable more accurate imaging of large areas, such as whole arms, hands, faces, or legs. Although the method has problems with noise and artifacts, these can be overcome by applying the modified B–L method to MRA1. This issue should be further investigated.