Scene statistics

Scene statistics is a discipline within the field of perception. It is concerned with the statistical regularities related to scenes. It is based on the premise that a perceptual system is designed to interpret scenes.

Biological perceptual systems have evolved in response to physical properties of natural environments. Therefore natural scenes receive a great deal of attention.

Natural scene statistics are useful for defining the behavior of an ideal observer in a natural task, typically by incorporating signal detection theory, information theory, or estimation theory.

Within-domain versus across-domain

Image generated from a database of segmented leaves that simultaneously registers natural images (scene information) with the exact locations of leaf boundaries (information about the physical environment). Such a database can be used to study across-domain statistics.

Geisler (2008) distinguishes between four kinds of domains: (1) Physical environments, (2) Images/Scenes, (3) Neural responses, and (4) Behavior.

Within the domain of images/scenes, one can study the characteristics of information related to redundancy and efficient coding.

Across-domain statistics determine how an autonomous system should make inferences about its environment, process information, and control its behavior. To study these statistics, it is necessary to sample or register information in multiple domains simultaneously.

Applications

Prediction of picture and video quality

One of the most successful applications of Natural Scenes Statistics Models has been perceptual picture and video quality prediction. For example, the Visual Information Fidelity (VIF) algorithm, which is used to measure the degree of distortion of pictures and videos, is used extensively by the image and video processing communities to assess perceptual quality, often after processing, such as compression, which can degrade the appearance of a visual signal. The premise is that the scene statistics are changed by distortion, and that the visual system is sensitive to the changes in the scene statistics. VIF is heavily used in the streaming television industry. Other popular picture quality models that use natural scene statistics include BRISQUE, and NIQE both of which are no-reference, since they do not require any reference picture to measure quality against.

Bibliography

• Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A 4, 2379–2394.
• Ruderman, D. L., & Bialek, W. (1994). Statistics of Natural Images – Scaling in the Woods. Physical Review Letters, 73(6), 814–817.
• Brady, N., & Field, D. J. (2000). Local contrast in natural images: normalisation and coding efficiency. Perception, 29, 1041–1055.
• Frazor, R.A., Geisler, W.S. (2006) Local luminance and contrast in natural images. Vision Research, 46, 1585–1598.
• Mante et al. (2005) Independence of luminance and contrast in natural scenes and in the early visual system. Nature Neuroscience, 8 (12) 1690–1697.
• Bell, A. J., & Sejnowski, T. J. (1997). The "independent components" of natural scenes are edge filters. Vision Research, 37, 3327–3338.
• Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy by V1? Vision Research, 37(23), 3311–3325.
• Sigman, M., Cecchi, G. A., Gilbert, C. D., & Magnasco, M. O. (2001). On a common circle: Natural scenes and Gestalt rules. PNAS, 98(4), 1935–1940.
• Hoyer, PO and Hyvärinen, A. A multi-layer sparse coding network learns contour coding from natural images, Vis. Res., vol. 42, no. 12, pp. 1593–1605, 2002.
• Geisler, W. S., Perry, J. S., Super, B. J., & Gallogly, D. P. (2001). Edge co-occurrence in natural images predicts contour grouping performance. Vision Research, 41, 711–724.
• Elder JH, Goldberg RM. (2002) Ecological statistics for the Gestalt laws of perceptual organization of contours. J. Vis. 2:324–53.
• Krinov, E. (1947). Spectral reflectance properties of natural formations (Technical translation No. TT-439). Ottawa: Nation Research Council of Canada.
• Ruderman, D. L., Cronin, T. W., & Chiao, C. (1998). Statistics of cone responses to natural images: implications for visual coding. Journal of the Optical Society of America A, 15, 2036–2045.
• Stockman, A., MacLeod, D. I. A., & Johnson, N. E. (1993). Spectral sensitivities of the human cones. J Opt Soc Am A Opt Image Sci Vis, 10, 1396–1402.
• Lee TW, Wachtler, T, Sejnowski, TJ. (2002) Color opponency is an efficient representation of spectral properties in natural scenes. Vision Research 42:2095–2103.
• Fine, I., MacLeod, D. I. A., & Boynton, G. M. (2003). Surface segmentation based on the luminance and color statistics of natural scenes. Journal of the Optical Society of America a-Optics Image Science and Vision, 20(7), 1283–1291.
• Lewis A, Zhaoping L. (2006) Are cone sensitivities determined by natural color statistics? Journal of Vision. 6:285–302.
• Lovell PG et al. (2005) Stability of the color-opponent signals under changes of illuminant in natural scenes. J. Opt. Soc. Am. A 22:10.
• Endler, J.A. 1993. The color of light in forests and its implications. Ecological Monographs 63:1–27.
• Wachtler T, Lee TW, Sejnowski TJ (2001) Chromatic structure of natural scenes. J. Opt. Soc. Am. A 18(1):65–77.
• Long F, Yang Z, Purves D. Spectral statistics in natural scenes predict hue, saturation, and brightness. PNAS 103(15):6013–6018.
• Van Hateren, J. H., & Ruderman, D. L. (1998). Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proceedings of the Royal Society of London B, 265, 2315–2320.
• Potetz, B., & Lee, T. S. (2003). Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes. Journal of the Optical Society of America a-Optics Image Science and Vision, 20(7), 1292–1303.
• Howe, C. Q., & Purves, D. (2002). Range image statistics can explain the anomalous perception of length. Proceedings of the National Academy of Sciences of the United States of America, 99(20), 13184–13188.Howe, C. Q., & Purves, D. (2005a). Natural-scene geometry predicts the perception of angles and line orientation. Proceedings of the National Academy of Sciences of the United States of America, 102(4), 1228–1233.
• Howe, C. Q., & Purves, D. (2004). Size contrast and assimilation explained by the statistics of natural scene geometry. Journal of Cognitive Neuroscience, 16(1), 90–102.
• Howe, C. Q., & Purves, D. (2005b). The Muller–Lyer illusion explained by the statistics of image-source relationships. Proceedings of the National Academy of Sciences of the United States of America, 102(4), 1234–1239.
• Howe, C. Q., Yang, Z. Y., & Purves, D. (2005). The Poggendorff illusion explained by natural scene geometry. Proceedings of the National Academy of Sciences of the United States of America, 102(21), 7707–7712.
• Kalkan, S. Woergoetter, F. & Krueger, N., Statistical Analysis of Local 3D Structure in 2D Images, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2006.
• Kalkan, S. Woergoetter, F. & Krueger, N., First-order and Second-order Statistical Analysis of 3D and 2D Structure, Network: Computation in Neural Systems, 18(2), pp. 129–160, 2007.