Scale-space theory for visual operations

Scale-space theory was originally developed as a principled framework for handling image structures at different scales. Now, it has evolved into a general theory of visual operations. We perform theoretical, algorithmic and experimental work to generalize this theory, in particular concerning new methods for feature detection and image descriptors.

Measurements by cell recordings have shown that mammalian vision has developed receptive fields tuned to different sizes and orientations in the image domain as well as to different image velocities in space-time. We have shown how such receptive field profiles can be derived mathematically from a small set of basic assumptions that correspond to structural properties of the environment.

Thus, there are close similarities between receptive fields as obtained by necessity from scale-space theory and receptive field profiles registered in biological vision.

Concerning recent developments, we have generalized the axiomatic formulation of linear scale-spaces over spatial and spatio-temporal image domains, and shown how it is possible to formulate a unified theory that comprises both linear and affine scale-space over spatial domains and spatio-temporal scale-spaces over both non-causal and time-causal space-time, based on a requirement of non-creation of new image structure over scale formulated in terms of non-enhancement of local extrema:

• Lindeberg (2011) ``Generalized Gaussian scale-space axiomatics comprising linear scale-space, affine scale-space and spatio-temporal scale-space'', Journal of Mathematical Imaging and Vision, 40(1): 36-81. (Download PDF)

or non-creation of local extrema with increasing scale for a purely temporal signal:

• Lindeberg (2016) "Time-causal and time-recursive spatio-temporal receptive fields", Journal of Mathematical Imaging and Vision, 55(1): 50-88. (Download PDF)
• Lindeberg (2015) "Separable time-causal and time-recursive spatio-temporal receptive fields", Proc. SSVM 2015: Scale-Space and Variational Methods in Computer Vision, Springer LNCS 9087: 90-102. (Download PDF)

Moreover, we have shown that this theory leads to predictions about receptive field profiles in good agreement with those found in biological vision, and we have used extensions of these theoretical results as a basis for a computational and normative theory of receptive fields in biological vision.

A survey of Gaussian and related scale-space representations over spatial, spatio-chromatic and spatio-temporal image domains that result from this generalized axiomatic scale-space theory aimed at providing theoretically well-founded processing of image data under natural image transformations is given in:

• Lindeberg (2013) ``Generalized axiomatic scale-space theory'', Advances in Imaging and Electron Physics, 178: 1-96. (Download PDF)

A shorter and more specific overview of how the rotationally symmetric Gaussian scale-space concept over spatial image domains provides a foundation for basic operations in computer vision regarding feature detection, multi-view matching and object recognition is presented in:

• Lindeberg (2009) ``Scale-space'', Encyclopedia of Computer Science and Engineering, John Wiley and Sons, Volume IV: 2495-2504. (Download PDF)

Earlier overviews of scale-space theory over spatial domains are given in:

• Lindeberg (1994) ``Scale-space theory: A basic tool for analysing structures at different scales'', Journal of Applied Statistics, 21(2): 224-270. (Download PDF)
• Lindeberg (1993) Scale-Space Theory in Computer Vision, Springer. doi:10.1007/978-1-4757-6465-9

A tutorial and retrospective overview of our contributions to methodologies for scale selection and their applications in computer vision is presented in:

• Lindeberg (2014) ``Scale selection'', Computer Vision: A Reference Guide, (K. Ikeuchi, ed.), Springer, pages 701-713. (Download PDF)

Our other contributions to scale-space theory and receptive field based approaches to vision can be found in our list of publications.