Principles and consequences of the initial visual encoding (2021). Brian Wandell and David Brainard To appear in “New Handbook of Mathematical Psychology“, Volume 3 edited by F. Gregory Ashby, Hans Colonius and Ehtibar N. Dzhafarov.
See the wiki for how the images were computed.
Abstract
Vision science combines ideas from physics, biology, and psychology. The language and ideas of mathematics help scientists communicate and provide an initial framing for understanding the visual system. In recent years, mathematics is often combined with computational modeling to add important realism to the mathematical formulation. Together, the mathematics and computational tools provide a realistic estimate of the initial signals that the brain analyzes to render visual judgments of (e.g. motion, depth, and color). This chapter first traces the calculations from the representation of the light signal, to how that signal is transformed by the lens to the retinal
image, and then how the image is converted into cone photoreceptor excitations. The central steps in the initial encoding rely heavily on linear systems theory and the mathematics of signal-dependent noise. We then describe computational methods that add more realism to the description of how light is encoded by cone excitations. Finally, we describe the mathematical formulation of the ideal observer that uses all the encoded information to perform a visual discrimination task, and Bayesian methods that combine prior information and sensory data to estimate the light input. These tools help us reason about the information that is present in the neural representation, what information is lost, and the types of neural circuits that might extract information to make judgments about the scene.
Mathematics and color science, from von Kries in 1902
We agree:
‘Only infrequently is it possible to subject the manifold phenomena of life to simple and strict forms of mathematical treatment without forcing the data and encountering contradiction, probably never without a certain abandonment of the immense multiplicity of details to which those phenomena owe their aesthetic attractiveness. Nevertheless, however, it has often proved to be possible and useful to establish, for wide fields of biological processes and organic arrangements, comparatively simple mathematical formulas which, though they are probably not applicable with absolute accuracy, nevertheless simulate to a certain approximation a large number of phenomena. Such representations not only offer preliminary orientation in a field that at first seems completely incomprehensible, but they also often direct research into a correct course, inasmuch as first an insight into those fundamental formulations is sought, and then the deviations from their strict validity, which become apparent here and there, are made the subject of special investigations. Among the fields of physiology which have permitted the establishment of such guiding formulas the theory of visual sensations and of color mixture assumes a particularly distinguished position (von Kries, 1902).’
