Please download to get full document.

View again

of 10
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report



Views: 5 | Pages: 10

Extension: PDF | Download: 0

Related documents
924 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 8, AUGUST 2002 Active Shape Model Segmentation With Optimal Features Bram van Ginneken*, Alejandro F. Frangi, Joes J. Staal, Bart M. ter Haar Romeny, and Max A. Viergever Abstract—An active shape model segmentation scheme is presented that is steered by optimal local features, contrary to normalized first order derivative profiles, as in the original formulation [Cootes and Taylor, 1995, 1999, and 2001]. A nonlinear NN-classifier is used,
  924 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 8, AUGUST 2002 Active Shape Model Segmentation WithOptimal Features Bram van Ginneken*, Alejandro F. Frangi, Joes J. Staal, Bart M. ter Haar Romeny, and Max A. Viergever  Abstract— An active shape model segmentation scheme is pre-sentedthatissteeredbyoptimallocalfeatures,contrarytonormal-ized first order derivative profiles, as in the srcinal formulation[Cootes and Taylor, 1995, 1999, and 2001]. A nonlinear NN-clas-sifier is used, instead of the linear Mahalanobis distance, to findoptimal displacements for landmarks. For each of the landmarksthat describe the shape, at each resolution level taken into accountduring the segmentation optimization procedure, a distinct set of optimal features is determined. The selection of features is auto-matic, using the training images and sequential feature forwardand backward selection. The new approach is tested on syntheticdata and in four medical segmentation tasks: segmenting the rightand left lung fields in a database of 230 chest radiographs, and seg-menting the cerebellum and corpus callosum in a database of 90slices from MRI brain images. In all cases, the new method pro-duces significantly better results in terms of an overlap error mea-sure (    using a paired T-test) than the srcinal activeshape model scheme.  Index Terms— Active shape models, medical image segmenta-tion, model-based segmentation. I. I NTRODUCTION S EGMENTATION is one of the key areas in computer vi-sion. During the 1970s and 1980s, many researchers ap-proachedthesegmentationproblemina  bottom-up fashion:em-phasis was on the analysis and design of filters for the detectionof local structures such as edges, ridges, corners and T-junc-tions. The structure of an image can be described as a collectionof such syntactical elements and their (spatial) relations, andsuch descriptions can be used as input for generic segmentationschemes. Unfortunately, these segmentations are often not verymeaningful. On the other hand,  top-down  strategies (also re-ferredtoas model-based  or active approaches)forsegmentationwereusedsuccessfullyinhighlyconstrainedenvironments,e.g.,in industrial inspection tasks. Often these methods are based Manuscript received August 10, 2001; revised May 21, 2002. This work wassupported by the Ministry of Economic Affairs, The Netherlands through theIOP Image Processing program. The Associate Editor responsible for coordi-nating the review of this paper was M. Sonka.  Asterisk indicates correspondingauthor. *B. van Ginneken is with the Image Sciences Institute, University MedicalCenterUtrecht,Heidelberglaan100,3584CXUtrecht,TheNetherlands( F. Frangi is with the Grupo de Tecnología de las Comunicaciones, Depar-tamentodeIngenieríaElectrónicayComunicaciones,UniversidaddeZaragoza,50015 Zaragoza, Spain.J. J. Staal and M. A. Viergever are with the Image Sciences Institute, Univer-sity Medical Center Utrecht, 3584 CX Utrecht, The Netherlands.B. M. ter Haar Romeny is with the Faculty of Biomedical Engineering, Med-ical and Biomedical Imaging, Eindhoven University of Technology, 5600 MBEindhoven, The Netherlands.Digital Object Identifier 10.1109/TMI.2002.803121 on template matching. Templates incorporate knowledge aboutboth the shape of the object to be segmented and its gray-levelappearance in the image, and are matched for instance by cor-relation or with generalized Hough transform techniques. Buttemplate matching, or related techniques, are likely to fail if theobject and/or background exhibit a large variability in shape orgray-level appearance, as is often the case in real-life imagesand medical data.Active contours or snakes [4], [5] and wave propagation methods such as level sets [6], have been heralded as a newparadigmsforsegmentation.Itwastheirabilitytodeformfreelyinstead of rigidly that spurred this enthusiasm. Nevertheless,such methods have two inherent limitations which make themunsuited for many medical segmentation tasks. First, little a priori  knowledge about the shape to be segmented can beincorporated, except for adjusting certain parameters. Second,the image structure at object boundaries is prescribed by lettingthe snakes attract to edges or ridges in the image, or by termi-nation conditions for propagating waves. In practice, objectboundaries do not necessarily coincide with edges or ridges.Toovercometheselimitations,researchersexperimentedwithhand-crafted parametric models. An illustrative example is thework of Yuille  et al.  [7] where a deformable model of an eye isconstructed from circles and parabolic patches and a heuristiccost function is proposed for the gray-level appearance of theimage inside and on the border of these patches. There are twoproblemswithparametricmodels.Firstofalltheyare dedicated  ,that is, limited to a single application. Second, there is no proof that theshape model and cost function proposed by the designerof the model are the optimal choice for the given application.Consequently, there is a need for generic segmentationschemesthatcanbetrainedwithexamplesastoacquire amodelof the shape of the object to be segmented (with its variability)and the gray-level appearance of the object in the image (withits variability). Such methods are prototype-based which makesit easy to adapt them to new applications by replacing theprototypes; they use statistical techniques to extract the majorvariations from the prototypes in a principled manner.Severalofsuchschemeshavebeenproposed.Foranoverviewsee the book of Blake and Isard [8] and the review by Jain  et al. [9]. In this paper, we focus on active shape models (ASMs) putforward by Cootes and Taylor [1], [3]. We have implemented the method based on the description of the ASM segmentationmethod detailed in [10]. The shape model in ASMs is given bythe principal components of vectors of landmark points. Thegray-level appearance model is limited to the border of the ob- ject and consists of the normalized first derivative of profiles 0278-0062/02$17.00 © 2002 IEEE  VAN GINNEKEN  et al. : ACTIVE SHAPE MODEL SEGMENTATION WITH OPTIMAL FEATURES 925 centered at each landmark that run perpendicular to the objectcontour. The cost (or energy) function to be minimized is theMahalanobis distance of these first derivative profiles. The fit-ting procedure is an alternation of landmark displacements andmodel fitting in a multiresolution framework.Several comparable approaches are found in the literature.Shapes and objects have been modeled by landmarks, finite-el-ement methods, Fourier descriptors and by expansion in spher-ical harmonics (especially for surfaces in three dimensions[11], [12]). Jain  et al.  [13] have presented a Bayesian frame-work in which templates are deformed and more probable de-formations are more likely to occur. They use a coarse-to-finesearch algorithm. Ronfard [14] has used statistics of objectand background appearance in the energy function of a snake.Brejl and Sonka [15] have described a scheme similar to ASMsbut with a nonlinear shape and appearance model that is op-timized with an energy function after an exhaustive search tofind a suitable initialization. Pizer  et al.  [16] describe an objectmodel that consists of linked primitives which can be fittedto images using methods similar to ASMs. Cootes and Taylorhave explored active appearance models (AAMs) [2], [17], [18] as an alternative to ASMs. In AAMs, a combined principalcomponent analysis of the landmarks and pixel values insidethe object is made which allows one to generate plausible in-stances of both geometry and texture. The iterative steps in theoptimization of the segmentation are steered by the differencebetween the true pixel values and the modeled pixel valueswithin the object. Sclaroff and co-workers [19], [20] have pro- posed a comparable method in which the object is modeled asa finite-element model.While there are differences, the general layout of theseschemes is similar in that there are: 1) a shape model thatensures that the segmentation can only produce plausibleshapes; 2) a gray-level appearance model that ensures that thesegmentation places the object at a location where the imagestructure around the border or within the object is similar towhat is expected from the training images; and 3) an algorithmfor fitting the model by minimizing some cost function. Usu-ally, the algorithm is implemented in a multiresolution fashionto provide long-range capture capabilities.ASMs have been used for several segmentation tasks in med-ical images [21]–[27]. Our contribution in this paper consists of a new type of appearance model for the gray-level varia-tions around the border of the object. Instead of using the nor-malized first derivative profile, we consider a general set of local image structure descriptors,  viz.  the moments of local his-tograms extracted from filtered versions of the images using afilter bank of Gaussian derivatives. Subsequently a statisticalanalysis is performed to learn which descriptors are themost in-formativeateachresolution,andateachlandmark.Thisanalysisamounts to feature selection with a -nearest neighbors ( NN)classifier and sequential feature forward and backward selec-tion. The NN classifier with the selected set of features is usedto compute the displacements of landmarks during optimiza-tion,insteadoftheMahalanobisdistanceforthenormalizedfirstderivative profile. In this paper, we refer to this new segmenta-tion method as “ASMs with optimal features,” where the termoptimal must be understood as described above. TABLE IP ARAMETERS FOR  A CTIVE  S HAPE  M ODELS  (O RIGINAL  S CHEME AND  N EW M ETHOD  W ITH  O PTIMAL  F EATURES ). V ALUES  U SED IN THE  E XPERIMENTS A RE  G IVEN  B ETWEEN  P ARENTHESES This paper is organized as follows. In Section II, there is astep-by-step description of the srcinal ASM scheme. In Sec-tion III, some observations regarding ASMs are made and pos-sible modifications to the method are discussed. In Section IV,the new method is explained. In Section V, the experiments onsynthetic images, chest radiographs, and brain MR data withboth the srcinal ASM scheme and the new method with op-timal features are described and the results are presented inSection VI. Discussion and conclusions are given in the Sec-tion VII.II. A CTIVE  S HAPE  M ODELS This section briefly reviews the ASM segmentation scheme.Wefollowthedescriptionandnotationof[2].Theparametersof the scheme are listed in Table I. In principle, the scheme can beused in D, but in this paper we give a two-dimensional (2-D)formulation.  926 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 8, AUGUST 2002  A. Shape Model An object is described by points, referred to as landmark points. The landmark points are (manually) determined in aset of training images. From these collections of landmark points, a point distribution model [28] is constructed as follows.Thelandmarkpoints arestackedinshapevectors(1)Principal component analysis (PCA) is applied to the shapevectors by computing the mean shape(2)the covariance(3)and the eigensystem of the covariance matrix. The eigenvectorscorresponding to the largest eigenvalues are retained in amatrix . A shape can now be approximatedby(4)where is a vector of elements containing the model param-eters, computed by(5)When fitting the model to a set of points, the values of areconstrained to lie within the range , where usuallyhas a value between two and three.The number of eigenvalues to retain is chosen so as toexplain a certain proportion of the variance in the trainingshapes, usually ranging from 90% to 99.5%. The desirednumber of modes is given by the smallest for which(6)BeforePCAisappliedtotheshapes,theshapescanbealignedby translating, rotating and scaling them so as to minimize thesum of squared distances between the landmark points. An iter-ative scheme known as Procrustes analysis [29] is used to alignthe shapes. This transformation and its inverse are also appliedbefore and after the projection of the shape model in (5). Thisalignment procedure makes the shape model independent of thesize,position,andorientationoftheobjects.Alignmentcanalsohelp to better fulfill the requirement that the family of point dis-tributions isGaussian,whichisanunderlying assumption of thePCA model. To this end, a projection into tangent space of eachshape may be useful; see [2] for details.However, the alignment can also be omitted. In that case, theresult is a shape model that can generate only shapes with asize,position,andorientationthatisconsistentwiththesuppliedexamples. If, for a certain application, the objects occur onlywithin a specific range of sizes, positions and orientations, sucha model might lead to higher segmentation performance. In anunalignedshape model, the first fewmodes of variation areusu-ally associated with variations in size and position and the vari-ation seen in the first few modes had the shape model been con-structed from aligned shapes, is usually shifted toward modeswith lower eigenvalues. Therefore, the parameter should belarger than in the case of aligned shapes in which no variationis present with respect to size and position. In our experience,building an unaligned shape model can improve segmentationperformance provided that enough training data is available.  B. Gray-Level Appearance Model The gray-level appearance model that describes the typicalimage structure around each landmark is obtained from pixelprofiles, sampled (using linear interpolation) around each land-mark, perpendicular to the contour.Note that this requires a notion of connectivity between thelandmark points from which the perpendicular direction can becomputed. The direction perpendicular to a landmark is computed by rotating the vector that runs fromto over 90 . In the applications presented in thispaper, all objects are closed contours, so for the first landmark,the last landmark and the second landmark are the points fromwhich a perpendicular direction is computed; for the last land-mark, the second to last landmark and the first landmark areused.On either side pixels are sampled using a fixed step size,which gives profiles of length . Cootes and Taylor [2]propose to use the normalized first derivatives of these profilesto build the gray-level appearance model. The derivatives arecomputed using finite differences between the th andthe th point. The normalization is such that the sum of absolute values of the elements in the derivative profile is 1.Denoting these normalized derivative profiles as ,the mean profile and the covariance matrix are computedfor each landmark. This allows for the computation of the Ma-halanobis distance [30] between a new profile and the profile model(7)Minimizing isequivalenttomaximizingtheprobabilitythat srcinates from a multidimensional Gaussian distribu-tion. C. Multiresolution Framework  These profile models, given by and , are constructed formultiple resolutions. The number of resolutions is denoted by. The finest resolution uses the srcinal image and a stepsizeofonepixelwhensamplingtheprofiles.Thenextresolutionis the image observed at scale and a step size of twopixels.Subsequent levels areconstructed by doubling theimagescale and the step size. 1 The doubling of the step size means that landmarks are dis-placed over larger distances at coarser resolutions. The blurringcauses small structures to disappear. The result is that the fitting 1 Notethatwedonotsubsampletheimages,asproposedbyCootesandTaylor.  VAN GINNEKEN  et al. : ACTIVE SHAPE MODEL SEGMENTATION WITH OPTIMAL FEATURES 927 at coarse resolution allows the model to find a good approxi-mate location based on global images structures, while the laterstages at fine resolutions allow for refinement of the segmenta-tion result.  D. Optimization Algorithm Shapes are fitted in an iterative manner, starting from themean shape. Each landmark is moved along the direction per-pendicular to the contour to positions on either side, evalu-ating a total of positions. The step size is, again,pixels for the th resolution level. The landmark is put at theposition with the lowest Mahalanobis distance. After movingall landmarks, the shape model is fitted to the displaced points,yielding an updated segmentation. This is repeated timesat each resolution, in a coarse-to-fine fashion.There is no guarantee that the procedure will converge. It isour experience, however, that in practice the scheme almost al-ways converges. The gray-level model fit improves steadily andreaches a constant level within a few iterations at each resolu-tion level. Therefore, we, conservatively, take a large value (10)for . 2 III. I MPROVING  ASM S There are several ways to modify, refine and improve ASMs.In this section, we mention some possibilities.  Bounds for the Shape Model . The shape model is fitted byprojectingashapeinthe -dimensionalspace( thenumberof landmarks, the factor two is because we consider 2-D images)upon the subspace spanned by the largest eigenvectors and bytruncating the model parameters so that the point is inside thebox bounded by . Thus there is no smooth transition;all shapes in the box are allowed, outside the box no shape is al-lowed.Clearlythiscanberefinedinmanyways,usingapenaltyterm or an ellipsoid instead of a box, and so on [2], [16].  Nonlinear Shape Models . The shape model uses PCAand, therefore, assumes that the distribution of shapes in the-dimensional space is normal. If this is not true, nonlinearmodels, such as mixture models, could be more suitable (seefor example [31]). Projecting the Shape Model . By projecting a shapeaccording to (5), i.e., fitting the shape model, the resultingmodel parameters minimize the sum of squared distancesbetween true positions and modeled positions. In practice, itcan be desirable to minimize only the distance between trueand model positions in the direction perpendicular to the objectcontour because deviation along the contour does not changewhether pixels are inside or outside the object. In [32], it isdemonstrated how to perform this projection on the contour.  Landmark Displacements . After the Mahalanobis distanceat each new possible position has been computed, Behiels  et al. [25]proposetousedynamicprogrammingtofindnewpositionsfor the landmarks, instead of moving each point to the positionwith the lowest distance. This avoids the possibility that neigh-boring landmarks jump to new positions in different directions 2 We always perform    iterations, contrary to Cootes and Taylor whomove to a finer resolution if a convergence criterion is reached before the   th iteration. and thus leads to a “smoother” set of displacements. This canlead to quicker convergence [25]. Confidence in Landmark Displacements . If information isavailable about the confidence of the proposed landmark dis-placement, weighted fitting of the shape model can be used, asexplained in [21].  Initialization . Because of the multiresolution implementa-tion, the initial position of the object (the mean shape, i.e., themean location of each landmark) does not have to be very pre-cise, as long as the distance between true and initial landmark positions is well within pixels. But if the objectcanbelocatedanywherewithintheinputimage,an(exhaustive)search to find a suitable initialization, e.g., as described in [15],can be necessary. Optimization Algorithm . Standard nonlinear optimizationalgorithms, such as hill climbing, Levenberg–Marquardt, or ge-netic algorithms can be used to find the optimal model param-eters instead of using the algorithm of alternating displace-ment of landmarks and model fitting. A minimization criterioncould be the sum of the Mahalanobis distances, possibly com-plemented by a regularization term constructed from the shapemodelparameters.Notethatamultiresolutionapproachcanstillbe used with standard nonlinear optimization methods. Alterna-tively, a snake algorithm can be used in which the shape modelprovides an internal energy term and the gray-level appearancemodel fit is used as external energy term.This list is not complete, but it is beyond the scope of this ar-ticletopresentacompletediscussionof thestrengths andweak-nesses of the ASM segmentation method. The issues describedabove are not considered in this work. Instead, we focus on thefollowing points:  Normalized First Derivative Profiles . The srcinal versionof the gray-level appearance model is always based on normal-izedfirstderivativeprofiles.Thereisno apriori reasonwhythisshould be an optimal choice. In this paper, we propose an alter-native.  MahalanobisDistance .TheMahalanobisdistancein(7)as-sumes a normal distribution of profiles. In practice, the distribu-tions of profiles will often be nonnormal, for example in caseswhere the background of the object may be one of several pos-sible choices. The ASM scheme proposed here uses a nonlinearclassifier in the gray-level appearance model and can, therefore,deal with nonnormal distributions.IV. ASM S  W ITH  O PTIMAL  F EATURES In this section, a new gray-level appearance model is de-scribed that is an alternative to the construction of normalizedfirst derivative profiles and the Mahalanobis distance cost func-tion of the srcinal ASMs.Theaimistobeabletomovethelandmarkpointstobetterlo-cations during optimization, along a profile perpendicular to theobjectcontour.Thebestlocationistheoneforwhicheverythingon one side of the profile is outside the object, and everythingon the other side is inside of it. 3 Therefore, the probability that 3 This assumes that the thickness of the object, in the direction perpendicularto a landmark, is larger than half the length of the profile. We will return to thispoint later.
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks