A Perceived Contrast Compensation Method Adaptive to Surround Luminance Variation for Mobile Phones

  • cc icon
  • ABSTRACT

    The loss in contrast-discrimination ability of the human visual system under high ambient illumination level can cause image quality degradation in mobile phones. In this paper, we propose a perceived contrast compensation method by processing the original displayed image. With consideration that the perceived contrast significantly varies across the image, this method extracts the local band contrast from the original image; it then compensates these contrast components to counteract the perceived contrast degradation. Experimental results demonstrate that this method can maintain most contrast details even in high ambient illumination levels.


  • KEYWORD

    Local band contrast , Perceived contrast , Contrast sensitivity function , Surround luminance

  • I. INTRODUCTION

    Mobile phones are viewed under various illumination levels. The displayed images on a mobile display can be perceived with a significant loss in contrast in daylight conditions. Image quality degradation in ambient lighting is mainly caused by screen reflections and light adaptation of the human visual system (HVS). Screen reflections decrease the contrast and color gamut of a mobile display by increasing the luminance of dark areas. Light adaptation of the HVS is another important factor. Images displayed in daylight are perceived as relatively dark due to light adaptation of the HVS because the luminance of a mobile display is considerably lower than that of the outdoor environment [1]. Moreover, the perceived image contrast changes according to different surround luminance levels [2]. CIECAM 97s [3] and CIECAM02 [4] color appearance models, developed by the Commission Internationale de L'Eclairage (CIE), determined the surround compensation ratios for average, dim, and dark levels. Various algorithms, such as illuminant adaptive color reproduction [1, 5] and veiling glare correction [6] for mobile displays, have been proposed to solve this image quality degradation problem. Display adaptive tone mapping operators can also be used to minimize visible contrast distortions for mobile displays [7]. All these works considered little about the spatial contrast discrimination ability of HVS in ambient lighting.

    In recent work, Kim [8] examined the effects of surround luminance on the shape of the spatial luminance contrast sensitivity function (CSF), which represents the amount of minimum contrast at each spatial frequency that is necessary for a visual system to distinguish a sinusoidal grating or Gabor patterns over a range of spatial frequencies from a uniform field. In bright surrounding conditions, a large amount of reduction in contrast sensitivity at middle and high spatial frequencies can be observed; however, only a small amount is observed at low frequencies. As a result, a greater number of image details cannot be detected because the contrast perception threshold rises in ambient lighting, especially at middle and high frequencies. Figure 1 shows the degradation of contrast perception for a middle frequency sine wave image.

    In later research [9] by Kim, a global image enhancement method is proposed to compensate the perception degradation. This method enhances whole spatial frequencies of the display image based on the degraded contrast sensitivity function with a frequency filter. Kim’s method implies that the contrast sensitivity function measured at various spatial frequencies can be implemented as a modulation transfer function of the system in the Fourier domain for image filtration.

    Employing an image processing method to obtain the same perceived image for viewers is an attractive idea. However, the perceived contrast may vary greatly across the image because human contrast sensitivity varies with local average luminance [10]. Thus, the perceived contrast in complex image cannot be represented by a single value or different values in the spatial frequency domain. A number of “local band contrast” measure methods have been proposed for other image processing approaches. These definitions of contrast [10, 11] in these methods assign a contrast value to every point in the image for each spatial frequency band. Numerous applications have been identified for these definitions, especially in HVS-related image processing problems [12-14].

    The objective of this paper is to enhance the local band contrast in images to compensate for perceived contrast degradation in ambient light. We proposed a perceived contrast compensation method based on local band contrast, which is extracted from the displayed image considering display factors. As our aim is not to mitigate the degradation in image quality caused by screen reflections in ambient lighting, our proposed method avoids changing other image properties, such as lightness and hue. Thus, it can be combined with other illuminant adaptive color reproduction and tone mapping methods.

    II. HVS CONTRAST SENSITIVITY REDUCTION

    The HVS is a nonlinear system with a very large dynamic range that behaves as a band-pass spatial filter [15]. The spatial filtering property is characterized by the CSF, which describes how contrast sensitivities vary as a function of spatial frequency. CSF exhibits a peak in contrast sensitivity at 4.0 ~ 5.0 cycles per degree (cpd) [16] and drops at both lower and higher frequencies. By measuring the minimum contrast at each spatial frequency in experiments, it has been deemed necessary for visual systems to distinguish a sinusoidal grating or a Gabor pattern over a range of spatial frequencies from a uniform field.

    The CSF model used was originally proposed by Barten [17] as a function of spatial frequency and as dependent on field size (or viewing angle in degrees) and mean luminance of the sinusoidal grating stimulus. As the mean luminance of the sinusoidal grating stimulus decreases, the contrast sensitivity at each spatial frequency and maximum resolvable spatial frequency decrease. In addition, the shape of luminance CSF likewise changes: the peaks in the functions shift toward lower spatial frequencies, broaden, and eventually disappear [18, 19].

    Kim identified characteristics of the HVS in the spatial frequency domain by considering surround luminance[8] as shown in Fig. 2. In general, the effect of surround luminance on the luminance CSF appears similar to that of mean luminance. A variable for compensating for surround luminance effect φ is multiplied with adaptive luminance L, as shown in Equation 1 [9].

    image

    where L is the mean luminance of white and black of the display under the given surround luminance Ls, and φ is the surround luminance effect function as given in Equation 2.

    image

    The surround luminance effect function φ forms a nonlinear function of luminance for approximating the perceived brightness reduction effect caused by increases in ambient illumination level. The surround luminance Ls can be measured by a luminance meter or be estimated by Equation 3.

    image

    where Eamb is ambient illumination given in lux and k is the reflectivity for surround surface. The surround adaptive CSF works as a weighting function to determine which parts of the image, regardless of their spatial frequency, will appear under a given ambient illumination level.

    According to the HVS model, an image detail can be detected if its contrast is greater than the threshold. Consequently, the amount of perceived contrast information changes with the contrast threshold level. Because ambient lighting affects the contrast threshold level of human vision, the same information for the image displayed at different ambient lighting levels cannot be perceived.

    III. PERCEIVED CONTRAST COMPENSATION METHOD

    An image stored as a matrix of pixel values is the most common representation for image processing; however, it does not reflect the way humans perceive images [20]. The response of the HVS depends much less on absolute luminance than on the relation of local variations to surrounding luminance. Meaningful visual information in images is mainly conveyed by contrast because the HVS has specialized cells that process signal contrast (rather than absolute signals) [20, 21]. The proposed method serves to compensate local contrast in images to improve image quality in ambient light.

    Perceived contrast may vary greatly across the image. To simulate nonlinear, threshold characteristics of spatial vision, several local band contrast definitions have been proposed [10, 11]. These definitions provide a method for transforming a pixel value image into several local band contrast images. Inspired by this research, we propose a perceptual contrast processing framework as depicted in Fig. 3.

    In this method, the image is not directly enhanced; instead, the image is transformed into local band contrast data. Because these contrasts degrade under ambient illumination as described by the Kim CSF, our method enhances them to compensate the details and reconstruct the output image with these enhanced contrasts.

       3.1. Local Band Contrast Computation

    The contrast of simple patterns, such as the sinusoidal pattern or a square on a uniform background, is well defined as a single value, such as the Weber contrast and Michelson contrast; however, this is not true for real-world images. As mentioned earlier, perceived contrast may greatly vary across the image because human contrast sensitivity varies with local average luminance. Peli [10] proposed a complex contrast model in which the contrast of an image can be locally defined. Because human contrast sensitivity is highly dependent on spatial frequency, local contrast for each spatial frequency band is separately calculated. Lubin [22] modified Peli’s contrast for image quality metric, and Winkler [11] defined an isotropic contrast measure computed from an oriented filter. Since these definitions do not consider display factors, such as display black level and reflection light, thus resulting in contrast deviations in low luminance areas of images, we modify the local band contrast.

    A mobile display is limited by maximum luminance and black luminance. The displayed luminance in ambient lighting can be modeled as Equation 4 [7]:

    image

    where l(x, y) is displayed luminance, f(x, y) is the original pixel value (0-1), γ is a display gamma (usually close to 2.2), Lmax is the peak display luminance, Lblack is the display black level, which is the luminance of the black pixel displayed in a perfectly dark room, Lrefl is ambient light reflected from a display surface.

    Using l(x, y)instead of the original pixel value, an image is decomposed into different frequency bands using a band-pass filter with a center frequency of 2i cpd as Peli’s method.

    image

    Here, l0 is the low-pass image and ai are different band-pass images, hn is the high-pass image. In our work, the different bands ai are limited in range from 3 to 32 cpd, for instance, four different bands of 3, 6, 12 and 24 cpd, as the main differences between surround adaptive CSF and the original one are in the frequencies larger than 3 cpd and the sensitivity of human vision drops to nearly zero in the frequencies larger than 32 cpd.

    For every band-pass filtered image ai(x, y), we define the corresponding local luminance mean image li(x, y), which is a low-pass filtered version of the original image that contains the frequency components below the band of ai(x, y) :

    image

    The local band contrast image is calculated as in Equation 7:

    image

       3.2. Perceived Contrast Compensation in Different Band Contrast Images

    The amount of perceived contrast information changes with the threshold contrast level. In Fig. 4, fewer contrast details can be perceived in the high frequency contrast images (Fig. 4(c) and (d)) compared to the low frequency contrast images (Fig. 4(b)) because a larger number of contrast values in the high-frequency contrast images are lower than the threshold.

    Further, because ambient lighting affects the contrast threshold level of human vision, the same information for the image displayed at different ambient lighting levels cannot be perceived. For example, assuming that the surround luminance is 3000 cd/m2, a greater number of details is below the threshold and fewer are perceived, as shown in Fig. 5. Most contrast information is lost in the middle frequency, while only a small amount of information is lost in the low and high frequencies.

    The ratios of different band contrast details are calculated as in Equation 8.

    image

    where Ni is the number of pixels for which the contrast value is larger than the contrast sensitivity threshold, and M×N is the total number of pixels of the band contrast image.

    To compensate for the loss in image contrast caused by an increase in surround luminance, the compensation ratio in different bands should be confirmed. These compensation ratios of different bands are determined by the contrast threshold difference between the reference (dark level) and given target surround luminance. In each contrast image, the contrast value above the threshold should be amplified by the matching ratio; those below the threshold are not processed to prevent noise amplification because human vision cannot detect them even in an environment without ambient lighting.

    image

    where LR is the reference luminance and Ls is the given surround luminance. The reference luminance can be chosen as an ideal environment luminance value or a low-level luminance value in a darkroom. In our work, we set the “reference luminance” as 0 cd/m2.

    Note that contrast masking is not involved when counting compensation rate for different frequency band here. Contrast masking [23] typically involves an increase in the HVS contrast threshold for a signal in the presence of another one. Some contrast information above the contrast threshold may be undetectable for the contrast masking phenomenon. In contrast perception evaluation, ignoring the contrast masking effect will result in an excessive contrast estimate. However, in the compensation method, the contrast masking phenomenon exists in both the original and the output image, because of which their effects are counteracted to some extent. Contrast masking is hence not involved in our proposed method.

    Finally, the output image is reconstructed in incremental steps by these enhanced contrast images. The reconstruction must be a low frequency to high frequency progression because li+1(x, y) are generated by li(x, y). By taking all li(x, y) into Equation 5, the final displayed luminance image can be obtained as shown in Equation 10.

    image

    IV. EXPERIMENTAL SETUP

    The aim of our proposed approach is to compensate for all contrast details to match those in an ideal environment without changing other image characteristics. We therefore designed an experiment to test contrast detail visibility in different environments.

    A darkroom with a large LED display was selected as the test environment (Fig. 6). This display was designed with surround lighting for which the brightness could be automatically adjusted, and the color temperature was approximate to D65. A mobile phone was placed in front of the LED display and the same orientation was maintained. This ensured that the background brightness could be accurately controlled when viewing the mobile phone. The LED display was secured as the only light source in the darkroom; therefore, light reflected on the mobile phone screen would be controlled at a low level.

    Five test images were selected from the LIVE Image Quality Assessment Database [24]. The images were of a lighthouse, a sailing boat, a statue, a standing woman and a woman in a hat as shown in Fig. 7. Because our method only enhanced the contrast extracted from the gray image, these RGB images should be converted into the HSV. Only brightness was applied through the enhancement procedure; chrominance properties were preserved.

    The test comprised two stages, one in the darkroom environment and the other in a high ambient light environment, controlled by LED display adjusted in 3000 cd/m2 as the surround luminance. In the darkroom, the five images were tested. In the high ambient light environment, the five images were enhanced by Kim’s method and our proposed method. As a result, a total of 15 test images, including five original ones and 10 enhanced ones, were randomly displayed for testing in the high ambient light environment.

    In both stages, observers sat in front of the display and gradually adapted to the ambient light. The 4.7-in mobile phone—placed at a fixed distance of 25 cm—randomly displayed each image. All images were randomly display for two rounds. In the first round, observers just browsed all images and were told to notice “detail visibility” of each image. In the second round, they were asked to score each image by assessing the level of “detail visibility”. The score was a 5-rating scale, from “hardly visible” (1) to “the most visible” (5).

    V. RESULTS AND DISCUSSION

       5.1. Experimental Results

    Nine observers (five male and four female) with normal color vision were selected as subjects for our experiment. The experimental results, average scores for each test image, are provided in Table 1. The detail visibility of images processed by our proposed method are higher than that of the original images in the ambient light and enhanced by Kim’s method, and nearly equal to that in a dark room.

    In Fig. 8, two test images enhanced by Kim’s method and the proposed method are presented. In both methods, the contrast of details has significantly improved, as shown by the cloud and grass in Fig. 8(b) and 8(c), the rock and leaves in Fig. 8(e) and 8(f). Kim’s method globally enhanced all frequencies in the Fourier domain, including the direct current component, with consideration of the contrast masking. As a result, we can observe higher brightness and the loss of contrast details in the highlighted region (as shown by the lighthouse in Fig. 8(b)).

    In the proposed approach, only local band contrast is extracted from original image and adjusted, so other image characteristics, such as lightness and hue, are maintained. On the other hand, as the compensation ratios for each of the local band contrasts are strictly deduced by the CSF degradation value as Equation 9, an over-enhancement problem can be avoided. So this approach can enhance the contrast of images without changing other image characteristics.

       5.2. Contrast Ratio Metric

    Limited research has been conducted on the influence of viewing conditions in objective visual quality metrics; moreover, the issues relating to ambient illumination have been largely unexplored [25]. Here, we use a ratio of perceived contrast amounts in ambient lighting to those in an ideal environment to evaluate the compensation effect. We sum all pixels in different contrast bands to determine the perceived contrast ratio (CR) as Equation 11:

    image

    where is the pixel number of which the contrast value is larger than threshold in band i enhanced contrast is that in . The sum of reflects the perceived contrast amount in ambient lighting and that of reflects the amount in a reference environment.

    In Table 2, perceived contrast ratios of the origin image in ambient light are quite small, for many contrast details are lost. In both Kim’s method and the proposed method, the contrast ratios become larger as contrasts increase; however, they nevertheless do not reach 100% of Origin1 because the contrast gains are limited by the dynamic range of the image.

       5.3. Dynamic Range

    The dynamic range is a limitation of this contrast compensation method. Because all band contrast images are enhanced by a factor greater than 1.0, the range of final compensation images—the sum of all band images—is typically greater than the original range of gray levels. Therefore, the final image does not provide sufficient enhancement at different frequencies, especially at the high frequency. Fortunately, as the output image is reconstructed from low frequency band contrast to a high frequency one, in most cases only the highest frequency band contrast, which contains less detail than other bands, is limited by dynamic range. In Kim’s method, all frequencies are enhanced with consideration of the contrast masking, which results in more gray values exceeding the dynamic range of the image and smaller contrast perceived ratios as in Table 2.

    VI. CONCLUSION

    In this paper, we proposed a perceived contrast compensation method that processes the extracted local band contrast from the original image. Experimental results demonstrated that this method can effectively enhance contrast details without changing other image characteristics; moreover, most contrast details can be compensated even in high ambient illumination. Although the image dynamic range may be a limitation of our method, only a small number of contrast details are limited by it.

  • 1. Kim J.-M., Park K.-H., Kwon O.-S., Cho Y.-H., Ha Y.-H. 2006 “Illuminant-adaptive color reproduction for mobile display,” [Electronic Imaging] Vol.2006 P.60580J google
  • 2. Baek Y. S., Kim H.-S., Park S.-O. 2014 “Determination of the perceived contrast compensation ratio for a wide range of surround luminance,” [J. Opt. Soc. Korea] Vol.18 P.89-94 google doi
  • 3. Fairchild M. D. 2001 “A revision of CIECAM97s for practical applications,” [Color Research & Application] Vol.26 P.418-427 google doi
  • 4. Moroney N., Fairchild M. D., Hunt R. W., Li C., Luo M. R., Newman T. 2002 “The CIECAM02 color appearance model,” [Proc. Color and Imaging Conference] P.23-27 google
  • 5. Kim J.-M., Son C.-H., Lee C.-H., Ha Y.-H. 2006 “Illuminant adaptive color reproduction based on lightness adaptation and flare for mobile phone,” [Proc. Image Processing, 2006 IEEE International Conference] P.1513-1516 google
  • 6. Park Y., Luo M. R., Li C. J., Kwak Y., Park D. S., Kim C. 2013 “Correcting veiling glare of refined CIECAM02 for mobile display,” [Color Research & Application] Vol.38 P.14-21 google doi
  • 7. Mantiuk R., Daly S., Kerofsky L. 2008 “Display adaptive tone mapping,” [ACM Transactions on Graphics (TOG)] Vol.27 P.68 google
  • 8. Kim Y.-J., Kim H.-S. 2010 “Spatial luminance contrast sensitivity: Effects of surround,” [J. Opt. Soc. Korea] Vol.14 P.152-162 google doi
  • 9. Kim Y. J. 2010 “An automatic image enhancement method adaptive to the surround luminance variation for small sized mobile transmissive LCD,” [Consumer Electronics, IEEE Transactions on] Vol.56 P.1161-1166 google doi
  • 10. Peli E. 1990 “Contrast in complex images,” [JOSA A] Vol.7 P.2032-2040 google doi
  • 11. Winkler S., Vandergheynst P. 1999 “Computing isotropic local contrast from oriented pyramid decompositions,” [Proc. ICIP] Vol.4 P.420-424 google
  • 12. Toet A. 1992 “Multiscale contrast enhancement with applications to image fusion,” [Opt. Eng.] Vol.31 P.1026-1031 google doi
  • 13. Lin T.-H., Kao T. 2000 “Adaptive local contrast enhancement method for medical images displayed on a video monitor,” [Medical Engineering & Physics] Vol.22 P.79-87 google doi
  • 14. Westland S., Owens H., Cheung V., Paterson­Stephens I. 2006 “Model of luminance contrast­sensitivity function for application to image assessment,” [Color Research & Application] Vol.31 P.315-319 google doi
  • 15. Campbell F., Green D. 1965 “Optical and retinal factors affecting visual resolution,” [The Journal of Physiology] Vol.181 P.576 google doi
  • 16. Wandell B. A. 1995 Foundations of Vision google
  • 17. Barten P. G. 1999 Contrast Sensitivity of the Human Eye and its Effects on Image Quality google
  • 18. Rohaly A. M., Buchsbaum G. 1989 “Global spatiochromatic mechanism accounting for luminance variations in contrast sensitivity functions,” [JOSA A] Vol.6 P.312-317 google doi
  • 19. Patel A. 1966 “Spatial resolution by the human visual system. The effect of mean retinal illuminance,” [JOSA] Vol.56 P.689-694 google doi
  • 20. Mantiuk R., Myszkowski K., Seidel H.-P. 2006 “A perceptual framework for contrast processing of high dynamic range images,” [ACM Transactions on Applied Perception (TAP)] Vol.3 P.286-308 google doi
  • 21. Hubel D. H. 1995 Eye, Brain, and Vision google
  • 22. Lubin J. 1995 “A visual discrimination model for imaging system design and evaluation,” [Vision Models for Target Detection and Recognition] Vol.2 P.245-357 google
  • 23. Legge G. E., Foley J. M. 1980 “Contrast masking in human vision,” [JOSA] Vol.70 P.1458-1471 google doi
  • 24. Sheikh H. R., Wang Z., Cormack L., Bovik A. C. 2005 “LIVE image quality assessment database release 2,” google
  • 25. Lin W., Jay Kuo C.-C. 2011 “Perceptual visual quality metrics: A survey,” [Journal of Visual Communication and Image Representation] Vol.22 P.297-312 google doi
  • [FIG. 1.] (a) Sine wave image. The sine amplitude gradually increases from 0 to 1, representing how the contrast becomes greater from top to bottom. (b) The same image perceived in ambient light. It is perceived as dark, and the contrast perception threshold becomes larger, representing how additional details could not be detected.
    (a) Sine wave image. The sine amplitude gradually increases from 0 to 1, representing how the contrast becomes greater from top to bottom. (b) The same image perceived in ambient light. It is perceived as dark, and the contrast perception threshold becomes larger, representing how additional details could not be detected.
  • [FIG. 2.] Comparison of CSFs under varying ambient luminance. As the surrounding luminance increases from 0 nit (cd/m2) to 10,000 nit, contrast sensitivity at each spatial frequency decreases. A large reduction in contrast sensitivity at medium and high spatial frequencies can be observed. However, only a small reduction is observed at low frequencies.
    Comparison of CSFs under varying ambient luminance. As the surrounding luminance increases from 0 nit (cd/m2) to 10,000 nit, contrast sensitivity at each spatial frequency decreases. A large reduction in contrast sensitivity at medium and high spatial frequencies can be observed. However, only a small reduction is observed at low frequencies.
  • [] 
  • [] 
  • [] 
  • [FIG. 3.] Data flow of perceptual contrast processing in the proposed framework.
    Data flow of perceptual contrast processing in the proposed framework.
  • [] 
  • [] 
  • [] 
  • [] 
  • [FIG. 4.] (a) Original Lena image and (b-d) Local band contrast images at 3, 12, 24 cycles/degree. Fewer contrast details can be perceived in the high frequency contrast images (c) and (d) compared with the low frequency contrast image (b).
    (a) Original Lena image and (b-d) Local band contrast images at 3, 12, 24 cycles/degree. Fewer contrast details can be perceived in the high frequency contrast images (c) and (d) compared with the low frequency contrast image (b).
  • [FIG. 5.] (a) Contrast threshold functions under different surround luminance. The contrast threshold, which is the reciprocal of contrast sensitivity value, represents the just noticeable contrast in each frequency band. A higher contrast threshold value implies that a greater contrast cannot be perceived in an image. (b) Ratios of local band contrast values above contrast threshold under different ambient illuminations. These data comes from local band contrast images of Lena at 3, 6, 12, 24 cycles/degree. The decreased ratios reflect that fewer contrast details can be perceived in a high ambient illumination.
    (a) Contrast threshold functions under different surround luminance. The contrast threshold, which is the reciprocal of contrast sensitivity value, represents the just noticeable contrast in each frequency band. A higher contrast threshold value implies that a greater contrast cannot be perceived in an image. (b) Ratios of local band contrast values above contrast threshold under different ambient illuminations. These data comes from local band contrast images of Lena at 3, 6, 12, 24 cycles/degree. The decreased ratios reflect that fewer contrast details can be perceived in a high ambient illumination.
  • [] 
  • [] 
  • [] 
  • [FIG. 6.] Experimental Setup. (a) Schematic top view of experimental setup. (b) Experiment environment photograph.
    Experimental Setup. (a) Schematic top view of experimental setup. (b) Experiment environment photograph.
  • [FIG. 7.] Five test images selected from the LIVE image quality assessment database.
    Five test images selected from the LIVE image quality assessment database.
  • [TABLE 1.] Five levels of detail visibility
    Five levels of detail visibility
  • [FIG. 8.] Two test images (a) and (d), corresponding to enhanced images (b) and (e) obtained by Kim’s method, and (c) and (f) corresponding to the proposed method. Under surrounding luminance of 3,000 nit, since ambient lighting affects the contrast threshold level of human vision, some contrast information of test images (a) and (d) cannot be perceived. In enhanced images, the contrast in the details is significantly improved, as shown by the cloud and grass in (b) and (c), and the rock and the leaves in (e) and (f).
    Two test images (a) and (d), corresponding to enhanced images (b) and (e) obtained by Kim’s method, and (c) and (f) corresponding to the proposed method. Under surrounding luminance of 3,000 nit, since ambient lighting affects the contrast threshold level of human vision, some contrast information of test images (a) and (d) cannot be perceived. In enhanced images, the contrast in the details is significantly improved, as shown by the cloud and grass in (b) and (c), and the rock and the leaves in (e) and (f).
  • [] 
  • [TABLE 2.] Perceived contrast ratios in five images
    Perceived contrast ratios in five images