Multifractal characterisation and classification of bread crumb digital images
- Rodrigo G Baravalle^{1}Email author,
- Claudio A Delrieux^{2} and
- Juan C Gómez^{1}
https://doi.org/10.1186/s13640-015-0063-8
© Baravalle et al.; licensee Springer. 2015
Received: 18 February 2014
Accepted: 18 March 2015
Published: 11 April 2015
Abstract
Adequate models of the bread crumb structure can be critical for understanding flow and transport processes in bread manufacturing, creating synthetic bread crumb images for photo-realistic rendering, evaluating similarities, and establishing quality features of different bread crumb types. In this article, multifractal analysis, employing the multifractal spectrum (MFS), has been applied to study the structure of the bread crumb in four varieties of bread (baguette, sliced, bran, and sandwich). The computed spectrum can be used to discriminate among bread crumbs from different types. Also, high correlations were found between some of these parameters and the porosity, coarseness, and heterogeneity of the samples. These results demonstrate that the MFS is an appropriate tool for characterising the internal structure of the bread crumb, and thus, it may be used to establish important quality properties it should have. The MFS has shown to provide local and global image features that are both robust and low-dimensional, leading to feature vectors that capture essential information for classification tasks. Results show that the MFS-based classification is able to distinguish different bread crumbs with very high accuracy. Multifractal modelling of the underlying structure can be an appropriate method for parameterising and simulating the appearance of different bread crumbs.
Keywords
Fractal Multifractal Image analysis Image classification Feature extraction1 Introduction
The goals of this research are (1) to evaluate if the MFS [1] can be applied to characterise and discriminate the bread crumb structure for different bread types from digital images and (2) to investigate the effectiveness of the method in the classification of these structures.
One of the most important factors to evaluate the quality of a bread loaf is related to its crumb structure. Close examination of different slices reveals considerable variation in the cell (air bubble) size even within a single sample of the same bread type.
Fractal and multifractal analysis of images has proved to be able to capture useful properties of the underlying material being represented. These features have been successfully applied in different areas, such as medicine [2,3] and texture classification [4]. In food research, fractal and multifractal analysis has been applied in the study of apple tissues [5], pork sirloins [6], and also in chocolate, potato, and pumpkin surfaces [7]. Through several procedures [8,9], it is possible to obtain different fractal dimensions (FD), each of them capturing a different property of the material (e.g. porosity, rugosity).
Data analysis of the results of the feature extraction process is useful for obtaining key properties of materials. This information could then be used in quality measurements of real samples and in the validation of synthetic representations of them. In other words, these processes are useful to determine if a given image presents the observed features in that material, allowing to associate quality measure parameters to it. In [10], a bread crumb quality test based on Gabor filters was performed, obtaining good quality assessment. Nevertheless, a small database was used (30 images). In [9], several fractal features were obtained for one type of bread, demonstrating that a vector of FDs would be capable of obtaining key features of the crumb texture more accurately than using a single FD.
In this work, we propose the application of the multifractal spectrum (MFS) to describe and classify different bread crumb types. One of the main features of the MFS is its bi-Lipschitz invariance, that is, invariance to perspective transforms (viewpoint changes) and smooth texture surface deformations. It is shown that the MFS is also locally invariant to affine changes in illumination. In other words, MFS analysis is in theory a robust feature extractor, what makes it specially adequate for the purposes of our study.
Food classification has already been applied using fractal and other techniques in [11,12], but these works do not address the intra-class problem, i.e. the classification is made among different foods and not by making different classes out of the same food.
In a previous work [13], we showed that the MFS in combination with other fractal features was able to classify different bread crumb types with high accuracy. The present work aims to simplify the model and strengthen these results by comparing only the MFS with other state-of-the-art features and using different classifiers and also to study the correlations of the features obtained with the procedure and different texture features obtained from the images.
The proposed method is compared to other state-of-the-art features for texture classification. The results of this feature extraction procedure show that the classifier is robust and presents good discrimination properties to distinguish between different bread types and also bread from non-bread images.
This paper is organised as follows. In Section 2, the theory underlying fractal sets is introduced, and the materials and methods employed in this work are presented. In Section 3, the results obtained in the characterisation and classification procedures are shown and discussed. In Section 4, the conclusions are summarised, as well as possible future works.
2 Materials and methods
2.1 Fractals and multifractals
The term fractal was first employed by the mathematician B. Mandelbrot in [14]. Fractal objects have the property of self-similarity (i.e. the geometrical or topological properties are invariant at different scales), and they are characterised by a non-integer dimension. Fractal objects can have one or more FDs. Most of the famous fractal sets (i.e. the Cantor set, the Von Koch curve, and the Sierpinski gasket) can be characterised by a single exponent that relates how their geometrical properties vary under scale changes. On the other hand, there are cases where the fractal object exhibits different exponents under different scales. Those are called multifractals [15] and are characterised by a sequence of FDs or even a function that establishes the local variance of the geometrical properties under scale changes. It is assumed that these structures are composed by different fractals coexisting simultaneously. The self-similarity, then, can be characterised by a multifractal spectrum that establishes the specific fractal behaviour of the set at a given scale. The multifractal approach characterises better the objects than the fractal one, since variations in local regions are captured in a more accurate manner. A particular definition of dimension, the so-called box dimension, is employed in fractal and multifractal analysis.
2.1.1 2.1.1 Box dimension
2.2 The theory behind multifractal analysis
In [9], several procedures were applied to analyse the bread crumb structure showing that a vector of FDs could better characterise those structures. Based on that assumption, in this work, a multifractal analysis of the bread crumb is carried out. The idea behind multifractal analysis is to examine, in the limit, the local behaviour of a measure μ at each point of the set under study.
2.2.1 2.2.1 Practical procedure for the MFS
There are several techniques described in the literature to obtain the MFS, which lead to different representations of the multifractal information present in the structure. Usually, the method of moments is used [5,6], but it produces a feature vector which is not always suitable for classification tasks. In this work, the procedure presented in [1] is employed, due to its better classification performance.
The technique first computes α(x) for each pixel x of the image. Denote with B(x,ε) the closed disk of radius ε>0 centred at x, then, α(x) is defined as a straight line fit of the values log(μ(B(x,ε))) and log(ε). Then, a discrete sample set {α _{ i },i=1,…,M} is taken from the interval [ 0,1], and the point set corresponding to that value of α _{ i } is formed by grouping the pixels with values that are close to that α _{ i } under some threshold. The FD for each point set is computed as the straight line fit of the values log((N _{ ε }(α _{ i })) and log(ε). The value M determines the vector length, i.e. the number of FDs of the MFS.
As previously stated, the f(α) spectrum (MFS) and the method of moments produce vectors which contains the same information, but in this work, the first is employed, since it also outperforms the method of moments in classification tasks. This process produces a finite vector which is used as the feature vector later in this paper. In the next sections, the vector length (the number of FDs) is chosen based on the classification performance of the computed feature vector.
2.2.2 2.2.2 Multifractal measures
where ∗ is the 2D convolution operator and G _{ ε } is a Gaussian smoothing kernel with variance ε, i.e. μ is the weighted average intensity value in the disk of radius ε centred at x (B(x,ε)). This is the density of the intensity function, and it describes how the intensity at a point changes over scale.
All these alternative measures modify the computed FD and MFS (except for trivial or monofractal images) and therefore are valuable choices in finding adequate features, as will be shown below.
2.3 Image acquisition
where x _{ c },y _{ c } are the coordinates of the actual pixel, and W is the window surrounding that pixel. Two parameters must be set in the algorithm: the size of the window (W _{size}) and the bias. It was found that different values for the bias are needed for better results when different capturing methods are used. The optimal values for the scanner samples were 80 for the window size and 1.15 for the bias. In the case of the digital camera samples, the optimal values found were 80 for the window size and 1 for the bias. These differences seem to be caused by the different illumination conditions present in the images resulting from these different capturing conditions. Further research is required in order to determine automatic values for these parameters.
3 Results and discussion
In this section, we will attempt to show how the MFS behave adequately as a feature descriptor able to distinguish bread from non-bread images. For this purpose, the MFS, using 20 FDs, was computed for each of the 200 images (i.e., 40 images of each bread type and 40 randomly selected non-bread images, getting 5 balanced classes). In the next subsections, we show how the computed data is analysed and used for classification purposes.
3.1 Data analysis
Self-organising maps (SOM) [20] of the feature vectors associated with each bread image were useful to represent them in a lower dimensional view, in order to better understand the meaning of their respective MFS. A SOM maps high-dimensional data into a (typically) two-dimensional representation, using neighbourhood information. Topological information of the original data is preserved.
In Figure 7, it becomes clear that the coefficients behave similarly for the first 5 dimensions (α∈[ 0,0.23]) in all the bread types but differently for the FD around 5 and above. It could be concluded that the first 5 dimensions are highly correlated with the void fraction (porosity) of the scanned samples. This means that the first FDs increase when the void fraction increases. Other FDs also have a high (positive or negative) correlation, but it depends on the bread type which dimension is correlated.
From the plots of the correlation coefficients of the MCA and stCA, in Figures 8 and 9, respectively, it could also be pointed out that the MCA has a higher correlation than the stCA with the FDs of the MFS. It means that the coarseness of the bread crumb structure could be better characterised by the features than its heterogeneity, using the MFS. In addition, the last 5 fractal dimensions of the spectra (α∈[ 0.79,1]) are highly (inversely) correlated with the MCA of the scanned samples. This implies that the last FDs increase when the MCA decrease. The same observation could be mentioned for the stCA of the samples, but the correlations are lower. In both cases, the correlation coefficients of the sandwich class are the lower among the bread types.
To summarise, the dimensions of the MFS which corresponds to α∈[ 0,0.23] are useful to measure the porosity of the scanned samples. Also, coarseness and heterogeneity could be measured by the dimensions with α∈[ 0.79,1]. As was suggested in a previous work [9], the bread crumb structure is better characterised by the use of a vector of fractal dimensions, since the three properties could be measured by the MFS, employing different sections of this feature vector.
3.2 Bread classification
In order to test for the discriminative capability of the method, a classification experiment is made. Five classes are defined, viz., baguette, sliced, bran, sandwich, and non-bread, assigning 40 images to each class. A comparison is made between the MFS and state-of-the-art features in the computer vision literature. This classification scheme corresponds to an intra-class problem, which is harder to solve than an ordinary inter-class one.
K-fold cross validation is applied to the entire set (with K=4), employing three different classifiers: support vector machines (SVM), random forests (RF) [21], and nearest neighbours (NN). Results show that the MFS presents good classification performance regardless of the classifier employed. The libsvm implementation [22] was used for the SVM classifier (with RBF kernel). In the case of the RF (100 trees) and the NN (1 neighbour) classifiers, the scikit-learn python library was employed.
Bread crumb classification results with different numbers of FDs for the MFS and different classifiers
#FDs | 10 | 20 | 30 |
---|---|---|---|
SVM | 96% | 94.5% | 95.5% |
RF | 91.5% | 93.5% | 93% |
NN | 88.5% | 90.5% | 90% |
Bread crumb classification results using different combinations of the MFS and different classifiers
Method | MFS | MFS+L | MFS+G | CIELab |
---|---|---|---|---|
SVM | 94.5% | 95.5% | 97.5% | 97.5% |
RF | 93.5% | 96% | 95% | 96% |
NN | 90.5% | 90% | 87% | 92% |
#FDs | 20 | 40 | 40 | 60 |
Bread crumb classification results for different state-of-the-art features and different classifiers
Method | Haralick | Lbp | SIFT |
---|---|---|---|
SVM | 94% | 78.5% | 96.5% |
RF | 91% | 71.5% | 92% |
NN | 79% | 70% | 86% |
#FDs | 13 | 36 | 128 |
Confusion matrix for the best results (CIELab method, using the SVM classifier)
Class | Baguette | Sliced | Bran | Sandwich | Non-bread |
---|---|---|---|---|---|
Baguette | 39 | 1 | 1 | 0 | 0 |
Sliced | 0 | 38 | 0 | 0 | 0 |
Bran | 0 | 0 | 39 | 1 | 0 |
Sandwich | 1 | 1 | 0 | 39 | 0 |
Non-bread | 0 | 0 | 0 | 0 | 40 |
The classification performance of the MFS for the bread crumb database is the highest among the algorithms studied. The MFS captures robust and useful information for classification in low-dimensional features. These results also agree with results obtained in [12] for the classification of other food products.
4 Conclusions
The visual appearance of different types of bread crumbs can be successfully characterised by the multifractal dimensions of their digital images. The FDs obtained from the MFS method whose α∈[ 0,0.23] provided a good measure of the bread crumb porosity, meaning that the higher these FDs, the higher the measure. In addition, the FDs whose α∈[ 0.79,1] are useful to measure coarseness and heterogeneity of bread crumb. The MFS contains useful data to characterise the three measures, combining the information in one feature vector.
The use of multifractal features in bread crumb texture classification showed excellent performance. The MFS demonstrated to be accurate enough to perform a classification of different bread types and also to discriminate non-bread from bread images. The classification performance of the MFS for the bread crumb database outperforms other state-of-the-art techniques employed in the computer vision literature. The information present in the MFS of the L, a, and b channels of the CIElab colour space obtained the best classification performance in all the developed tests. This result appears to be a consequence of the different capturing devices used in this work. Also, it was shown that the MFS is sensitive to changes in the illumination conditions during image acquisition.
The results found could also be applied to validate synthetic samples, in the sense that they should have similar features to the bread type they are trying to simulate. The features found with the MFS could be employed to tune bread crumb quality parameters.
Declarations
Authors’ Affiliations
References
- Y Xu, H Ji, C Fermüller, Viewpoint invariant texture description using fractal analysis. Int. J. Comput. Vis. 83(1), 85–100 (2009).View ArticleGoogle Scholar
- J Andjelkovic, N Zivic, B Reljin, V Celebic, I Salom, in Proceedings of the 8th Conference on Signal, Speech and Image Processing. SSIP’08. Classifications of digital medical images with multifractal analysis (World Scientific and, Engineering Academy and Society (WSEAS),Stevens Point, Wisconsin, USA, 2008), pp. 88–92.Google Scholar
- L Yu, D Qi, in Control and Decision Conference (CCDC), 2011 Chinese. Holder exponent and multifractal spectrum analysis in the pathological changes recognition of medical CT image (IEEE,Piscataway, NJ, USA, 2011), pp. 2040–2045.View ArticleGoogle Scholar
- H Wendt, P Abry, S Jaffard, Ji H, Z Shen, in 16th IEEE International Conference Image Processing (ICIP). Wavelet leader multifractal analysis for texture classification (IEEE Signal Processing SocietyPiscataway, NJ, USA, 2009), pp. 3829–3832.Google Scholar
- F Mendoza, P Verboven, QT Ho, G Kerckhofs, M Wevers, Nicolaï B, Multifractal properties of pore-size distribution in apple tissue using x-ray imaging. J. Food Eng. 99(2), 206–215 (2010).View ArticleGoogle Scholar
- S Serrano, F Perán, FJ Jiménez-Hornero, E Gutiérrez de Ravé, Multifractal analysis application to the characterization of fatty infiltration in Iberian and White pork sirloins. Meat Sci. 93(3), 723–732 (2012).View ArticleGoogle Scholar
- R Quevedo, L-G Carlos, JM Aguilera, L Cadoche, Description of food surfaces and microstructural changes using fractal image texture analysis. J. Food Eng. 53(4), 361–371 (2002).View ArticleGoogle Scholar
- H-O Peitgen, H Jürgens, D Saupe, Chaos and Fractals: New Frontiers of Science (Springer, New York, Berlin, Paris, 2004).View ArticleGoogle Scholar
- U Gonzales-Barron, F Butler, Fractal texture analysis of bread crumb digital images. Eur. Food Res. Technol. 226, 721–729 (2008).View ArticleGoogle Scholar
- Y Fan, H Zhang, in Mechatronics and Automation, Proceedings of the 2006 IEEE International Conference On. Application of Gabor filter and multi-class SVM in baking bread quality classification (IEEE,Piscataway, NJ, USA, 2006), pp. 1498–1502.View ArticleGoogle Scholar
- Z Zong, DT Nguyen, P Ogunbona, W Li, in Proceedings of the 2010 IEEE International Symposium on, Multimedia. ISM ’10. On the combination of local texture and global structure for food classification (IEEE Computer Society,Washington, DC, USA, 2010), pp. 204–211.View ArticleGoogle Scholar
- M Bosch, F Zhu, N Khanna, J Boushey Carol, EJ Delp, in Proceedings of the XIX European Signal Processing Conference. EUSIPCO 2011. Food texture descriptors based on fractal and local gradient information (IEEE,Kessariani, Greece, 2011), pp. 764–768.Google Scholar
- R Baravalle, C Delrieux, JC Gómez, in Proceedings of the X Congreso Argentino de Mecánica Computacional (MECOM 2012). Bread crumb classification using fractal and multifractal features (AMCA,Santa Fe, Argentina, 2012), pp. 3013–3025.Google Scholar
- BB Mandelbrot, The Fractal Geometry of Nature, 1st edn (W.H. Freeman, New York, 1982).MATHGoogle Scholar
- BB Mandelbrot, Multifractal measures, especially for the geophysicist. Pure Appl. Geophys. 131, 5–42 (1989).View ArticleGoogle Scholar
- A Silvetti, C Delrieux, Multifractal analysis of medical images. Actas de la trigésimo novena Jornada Argentina de Informática e Investigación Operativa (JAIIO) (SADIO, Buenos Aires, Argentina, 2010).Google Scholar
- L Fei-Fei, R Fergus, P Perona, in Proceedings of the Workshop on Generative-Model Based Vision. Learning generative visual models from few training examples an incremental Bayesian approach tested on 101 object categories (Washington, DC, 2004).Google Scholar
- JM White, GD Rohrer, Image thresholding for optical character recognition and other applications requiring character image extraction. IBM J. Res. Dev. 27(4), 400–411 (1983).View ArticleGoogle Scholar
- L-K Huang, M-JJ Wang, Image thresholding by minimizing the measures of fuzziness. Pattern Recognit. 28(1), 41–51 (1995).View ArticleGoogle Scholar
- T Kohonen, MR Schroeder, TS Huang, Self-Organizing Maps, 3rd edn (Springer, Secaucus, NJ, USA, 2001).View ArticleMATHGoogle Scholar
- L Breiman, Random forests. Mach. Learn. 45(1), 5–32 (2001).View ArticleMATHGoogle Scholar
- C-C Chang, C-J Lin, LIBSVM: A library for support vector machines. Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm (2001).
- RS Hunter, Photoelectric color difference meter. J. Opt. Soc. Am. 48(12), 985–993 (1958).View ArticleGoogle Scholar
- RM Haralick, K Shanmugam, I Dinstein, Textural features for image classification. Syst. Man Cybernet. IEEE Trans. SMC-3(6), 610–621 (1973).View ArticleGoogle Scholar
- T Ojala, M Pietikainen, D Harwood, in Pattern Recognition, 1994. Vol. 1 - Conference A: Computer, Vision & Image Processing., Proceedings of the 12th IAPR International Conference On, 1. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions (IEEE Computer Society Press,Los Alamitos, Calif., USA, 1994), pp. 582–585.Google Scholar
- DG Lowe, Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004).View ArticleGoogle Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.