Handbook of Texture Analysis

The following sections are included:

• Preface

• Contents

Textures are characteristic intensity (or colour) variations that typically originate from roughness of object surfaces. For a well-defined texture, intensity variations will normally exhibit both regularity and randomness, and for this reason texture analysis requires careful design of statistical measures. While there are certain quite commonly used approaches to texture analysis, much depends on the actual intensity variations, and methods are still being developed for ever more accurately modelling, classifying and segmenting textures. This introductory chapter explores and reviews some fundamental techniques.

Texture describes a myriad of spatial patterns. Some can be quite simple, for example a checker board pattern, while others can exhibit extremely complex behaviour, as seen in nature. The science of texture nalysis has been on the search for a model that can give a mathematical description to these patterns, and thus, a definition, for currently no precise definition of texture exists. However it is generally accepted that texture is a pattern that can be characterised by its local spatial behaviour, and is statistically stationary. The second pillar implies that all local spatial extents of a texture exhibit like behaviour. These sound like obvious pillars, but what this means is that the Markov Random Field model is very applicable to modelling texture. Today, it is the Markov Random Field model, or variations of it, that is most often used in modelling texture. However when applying the model to a texture, there are still some specific questions that do not have complete answers. Firstly, given a texture, what local spatial characteristics need to be modelled, and secondly, what is local? Complete answers to these questions are not currently available, so generally these questions are rephrased to ask, what model gives us the result we want? That is, if given say a classification problem, which model best discriminates between the desired texture classes? Or given say a synthesis problem, which model best replicates the desired texture (where "best" has its own qualities of measure)? Unfortunately, these two model driving forces, produce opposingly different model formulations. To really understand texture, the "Holy Grail" of texture models would be one that could uniquely describe a texture, giving both optimal discrimination and synthesis properties. To date, this has only really been achieved for a select few textures.

We investigate texture classification from single images obtained under unknown viewpoint and illumination. It is demonstrated that materials can be classified using the joint distribution of intensity values over extremely compact neighbourhoods (starting from as small as 3 × 3 pixels square), and that this outperforms classification using filter banks with large support. It is also shown that the performance of filter banks is inferior to that of image patches with equivalent neighbourhoods.

We develop novel texton based representations which are suited to modelling this joint neighbourhood distribution for MRFs. The representations are learnt from training images, and then used to classify novel images (with unknown viewpoint and lighting) into texture classes. Three such representations are proposed, and their performance is assessed and compared to that of filter banks.

The power of the method is demonstrated by classifying 2806 images of all 61 materials present in the Columbia-Utrecht database. The classification performance surpasses that of recent state of the art filter bank based classifiers such as Leung and Malik (IJCV 01), Cula and Dana (IJCV 04), and Varma and Zisserman (IJCV 05). We also benchmark performance by classifying all the textures present in the Microsoft Textile database as well as the San Francisco outdoor dataset.

We conclude with discussions on why features based on compact neighbourhoods can correctly discriminate between textures with large global structure and why the performance of filter banks is not superior to the source image patches from which they were derived.

Random textures are notoriously more difficult to deal with than regular textures particularly when detecting abnormalities on object surfaces. In this chapter, we present a statistical model to represent and analyse random textures. In a two-layer structure a texture image, as the first layer, is considered to be a superposition of a number of texture exemplars, possibly overlapped, from the second layer. Each texture exemplar, or simply texem, is characterised by mean values and corresponding variances. Each set of these texems may comprise various sizes from different image scales. We explore Gaussian mixture models in learning these texem representations, and show two different applications: novelty detection and image segmentation.

This chapter presents a novel and generic framework for image segmentation using a compound image descriptor that encompasses both colour and texture information in an adaptive fashion. The developed image segmentation method extracts the texture information using low-level image descriptors (such as the Local Binary Patterns (LBP)) and colour information by using colour space partitioning. The main advantage of this approach is the analysis of the textured images at a micro-level using the local distribution of the LBP values, and in the colour domain by analysing the local colour distribution obtained after colour segmentation. The use of the colour and texture information separately has proven to be inappropriate for natural images as they are generally heterogeneous with respect to colour and texture characteristics. Thus, the main problem is to use the colour and texture information in a joint descriptor that can adapt to the local properties of the image under analysis. We will review existing approaches to colour and texture analysis as well as illustrating how our approach can be successfully applied to a range of applications including the segmentation of natural images, medical imaging and product inspection.

This chapter deals with the analysis of 3D surface texture. A model of the surface-to-image function is developed. This theory shows that sidelighting acts as a directional filter of the surface height function. Thus the directionality and power of image texture is a function of the illuminant's slant and tilt angles.

The theory is extended to common texture features such as those of Gabor and Laws to show that their behaviours follow that of simple harmonic motion. Variation of illuminant's tilt causes class centres to describe a trajectory on a hyper-ellipse, while changes in slant cause the hyper-ellipse to grow or shrink. Thus it is not surprising that classifiers trained under one set of illumination conditions will fail under another.

Finally we develop a classifier that exploits a simplified version of this theory. Given a single image of a surface texture taken under unknown illumination conditions our classifier will return both the estimated illumination direction and the class assignment.

3D Image texture due to the illumination of rough surfaces provides cues about the light field and the surface geometry on the meso and on the macro scales. We discuss 3D texture models, their application in the computer vision domain, and psychophysical studies. Global, histogram-based cues such as the "texture contrast function" allow for simple robust inferences with regard to the light field and to the surface structure of 3D objects. If one additionally takes the spatial structure of the image texture into account, it is possible to calculate local estimates of surface geometry and the local illumination orientation. The local illumination orientation can be estimated within a few degrees for rendered Gaussian as well as real surfaces, algorithmically and by human observers. Using such local estimates of illumination orientation, we can determine the global structure of the "illuminance flow" for 3D objects. The illuminance flow is a robust indicator of the light field and thus reveals global structure in a scene. It is an important prerequisite for many subsequent inferences from the image such as shape from shading.

Appearance modeling is fundamental to the goals of computer vision and computer graphics. Traditionally, appearance was modeled with simple shading models (e.g. Lambertian or specular) applied to known or estimated surface geometry. However, real world surfaces such as hair, skin, fur, gravel, scratched or weathered surfaces, are difficult to model with this approach for a variety of reasons. In some cases it's not practical to obtain geometry because the variation is so complex and fine-scale. The geometric detail is not resolved with laser scanning devices or with stereo vision. Simple reflectance models assume that all light is reflected from the point where it hits the surface, i.e. no light is transmitted into the surface. But in many real surfaces, a portion of the light incident on one surface point is scattered beneath the surface and exits at other surface points. This subsurface scattering causes difficulties in accurately modeling a surface such as frosted glass or skin with a simple geometry plus shading model. So even when a precise geometric profile is attainable, applying a pointwise shading model is not sufficient. Because of these issues, image-based modeling has become a popular alternative to modeling with geometry and point-wise shading.

Real world surfaces are often textured with a variation in color (as in a paisley print or leopard spots) or a fine-scale surface height variation (e.g. crumpled paper, rough plaster, sand). Surface texture complicates appearance prediction because local shading, shadowing, foreshortening and occlusions change the observed appearance when lighting or viewing directions have changed. As an example, consider a globally planar surface of wrinkled leather where large local shadows appear when the surface is obliquely illuminated and disappear when the surface is frontally illuminated. Accounting for the variation of appearance due to changes in imaging parameters is a key issue in developing accurate models. The terms BRDF and BTF have been used to describe surface appearance. The BRDF (bidirectional reflectance distribution function) describes surface reflectance as a function of viewing and illumination angles. Since surface reflectance varies spatially for textured surfaces, the BTF was introduced to add a spatial variation. More specifically, the bidirectional texture function (BTF) is observed image texture as a function of viewing and illumination directions. In this chapter, topics in BRDF and BTF modeling for vision and graphics are presented. Two methods for recognition are described in detail: (1) bidirectional feature histograms and (2) symbolic primitives that are more useful for recognizing subtle differences in texture.

In this chapter we present a modeling framework for video sequences that exhibit certain temporal regularity properties, intended in a statistical sense. Examples of such sequences include sea-waves, smoke, foliage, talking faces, flags in wind, etcetera. We refer to them as dynamic textures. The models we describe are non-linear and designed to capture the joint temporal variability of the appearance and of the shape of the scene, or a portion of it. We discuss the problems of modeling, learning, and synthesis of dynamic textures in the context of time series analysis, system identification theory, and finite element methods. We show that this framework allows inferring models capable of synthesizing infinitely long video sequences of complex dynamic visual phenomena.

Many textures require complex models to describe their intricate structures, or are even still beyond the reach of current texture synthesis. Their modeling can be simplified if they are considered composites of simpler subtextures. After an initial, unsupervised segmentation of the composite texture into subtextures, it can be described at two levels. One is a label map texture, which captures the layout of the different subtextures. The other consists of the different subtextures. Models can then be built for the "texture" representing the label map and for each of the subtextures separately. Texture synthesis starts by creating a virtual label map, which is subsequently filled out by the corresponding, synthetic subtextures. In order to be effective, this scheme has to be refined to also include mutual influences between textures, mainly found near their boundaries. The proposed composite texture model also includes these. Obviously, one could consider such a strategy with sub-subtextures. Several experiments are shown, for example synthetic textures that represent entire landscapes.

The trace transform is a generalisation of the Radon transform. It consists of tracing an image with straight lines along which certain functionals of the image function are calculated. When a second functional is applied over all values computed along parallel lines, a function of the orientation of the parallel lines is produced. When a third functional over the values of this function is applied, a so called "triple feature" is produced. Different combinations of the three successive functionals used may be chosen so that the triple feature is invariant to rotation, translation, scaling or affine transform of the imaged object. The theory of triple feature construction, however, is accurate in the continuous domain. The application of the process in the digital domain may lead to severe loss of the desired properties of the computed features. This chapter first reviews the trace transform from the theoretical and applications point of view, and then shows how to implement it in practice, so the desirable properties of the triple features are retained. Finally, it presents an application of the theory to the problem of texture feature extraction.

Local Binary Pattern (LBP) is a simple yet very efficient texture operator which labels the pixels of an image by thresholding the neighborhood of each pixel with the value of the center pixel and considers the result as a binary number. Due to its discriminative power and computational simplicity, LBP texture operator has become a popular approach in various applications. This chapter presents the LBP methodology and its application to face image analysis problems, demonstrating that LBP features are also efficient in nontraditional texture analysis tasks. We explain how to easily derive efficient LBP based face descriptions which combine into a single feature vector the global shape, the texture and eventually the dynamics of facial features. The obtained representations are then applied to face and eye detection, face recognition and facial expression analysis problems, yielding in excellent performance.

The aim of this chapter is to give experienced and new practitioners in image analysis and computer vision an overview and a quick reference to the "galaxy" of features that exist in the field of texture analysis. Clearly, given the limited space, only a corner of this vast galaxy is covered here! Firstly, a brief taxonomy of texture analysis approaches is outlined. Then, a list of widely used texture features is presented in alphabetical order. Finally, a brief comparison of texture features and feature extraction methods based on several literature surveys is given.

The following sections are included:

• Index