Facial Action Unit Tracking And Facial Activity .
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference ProceedingsFacial Action Unit Tracking and Facial ActivityRecognition Based On Dynamic Bayesian NetworkM. Shanmuga Priya11PG Student, Park College of Engineering and TechnologyEmail : priyamanimozhi1991@gmail.comrecognition, facial feature tracking,simultaneous tracking and recognition.I INTRODUCTIONThe recovery of facial activities in imagesequence is an important and challengingproblem. In recent years, plenty ofcomputer vision techniques have beend e v e l o p e d to track or recognize facialactivities in three levels. First, in thebottom level, facial feature tracking,which usually detects and tracks prominentfacial feature points (i.e., the faciallandmarks) surrounding facial components(i.e., mouth, eyebrow, etc.), captures thedetailed face shape i n f o rm at i o n . Second,facial actions recognition, i.e., recognize facialAction Units (AUs)defined in theFacial Action Coding System (FACS) [1],try to recognize some meaningful facialactivities (i.e., lid tightened, eyebrow raiser,etc.). In the top level, facial expressionanalysis attempts to recognize facialexpressions that represent the humanemotional states. The facial feature tracking,AU recognition and represent the facialactivities in three levels from local tog l o b a l , and they are interdependentproblems. For example, facial featuretracking can be usedIJERTAbstract-The tracking and recognition offacial activities from images or videos haveattracted great attention in computer visionfield. Facial activities are characterized bythree levels. First, in the bottom level, facialfeature points around each facial component,i.e., eyebrow, mouth, etc., which describe thedetailed of face shape information arecaptured. Second, in the middle level, facialaction units, defined in the facial actioncoding system that represents the contractionof a specific set of facial muscles, i.e., lidtightened, eyebrow raiser, etc are identified.Finally, in the top level, six prototypicalfacial expressions representing the globalfacial muscle movement which arecommonly used to describe the humanemotion states are measured. In existingapproaches focus on the three levels of facialactivities, and track them separately. Butproposed work introduces a unifiedprobabilistic framework based on theDynamicBayesianNetworktosimultaneously and efficiently represent thefacial movements in different levels whichimproves the efficiency of the system whencompared with existing systems.IndexTerms—Bayesiannetwork,expression recognition, facial action unitwww.ijert.org17
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference ing robustness during imageinterpretation. They argue that a modelshould only be able to deform in wayscharacteristic of the class of objects itrepresents. It describes a method for buildingmodels by learning patterns of variabilityfrom a training set of correctly annotatedimages. These models can be used for imagesearch in an iterative refinement algorithmanalogous to that employed by ActiveContour Models The key difference is thatour Active Shape Model (ASM) can onlydeform to fit the data in ways consistent withthe training -statisticalmodels of shape which can be constructedfrom training sets of correctly labeledimages. A PDM represents an object as a setof labeled points, giving their mean positionsand a small set of modes of variation whichdescribe how the object’s shape can change.Applying limits to the parameters of themodel enforces global shape constraintsensuring that any new examples generatedare similar to those in the training set. Givena set of shape parameters, an instance of themodel can be calculated rapidly. The modelsare compact and are well suited to generateand-test image search strategies. ActiveShape Models (ASMs) exploit the linearformulation of PDMs in an iterative searchprocedure capable of rapidly locating themodeled structures in noisy, cluttered imageseven if they are partially occluded. Objectidentification and location are robust becausethe models are specific in the sense thatinstances are constrained to be similar tothose in the training set.IJERTIn the feature extraction stage inexpression/AUsrecognitionandexpression/AUs recognition results canprovide a prior distribution for facial featurepoints. However, most current methods onlytrack or recognize the facial activities inone or two levels, and track themseparately, either ignoring their interactionsor limiting the interaction to one way. Inaddition, the estimates obtained by imagebased methods in each level are alwaysuncertain andambiguous b e c a u s e ofnoise, occlusion a n d the imperfect natureof the vision algorithm. The proposed facialactivity recognition system consists of twomain stages: offline facial activity modelconstruction and online facial motionmeasurement and inference. Specifically,using training data and subjectivedomain knowledge, the facial activitymodel is constructed offline. During theonline recognition, as shown in Fig. 1,various computer vision techniques areused to track the facial feature points,and to g e t the m eas u rem en t s of facialmotions, i.e., AUs. These measurements arethen used as evidence to infer the true statesof the three level facial activitiessimultaneously.II LITERATURE REVIEW1) Active Shape Models-Their Trainingand ApplicationModel-based vision is firmlyestablished as a robust approach torecognizing and locating known rigid objectsin the presence of noise, clutter, andocclusion. It is more problematic to applymodel based methods to images of objectswhose appearance can vary, though anumber of approaches based on the use offlexible templates have been proposed. Theproblem with existing methods is that theysacrifice model specificity in order towww.ijert.org2) Facial Action Coding System(FACSThe Facial Expression CodingSystem (FACES) was developed as a less18
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference Proceedingspresent all human face expressions and rigidhead movements.3) Robust Facial Feature Tracking UnderVarying Face Pose and Facial ExpressionHierarchicalmulti-stateposedependent approach for facial featuredetection and tracking under varying facialexpression and face pose. For effective andefficient representation of feature points, ahybrid representation that integrates Gaborwavelets and gray-level profiles is proposed.To model the spatial relations among featurepoints, a hierarchical statistical face shapemodel is proposed to characterize both theglobal shape of human face and the localstructural details of each facial component.Furthermore, multi-state local shapemodels are introduced to deal with shapevariations of some facial components underdifferent facial expressions. During detectionand tracking, both facial component statesand feature point positions, constrained bythe hierarchical face shape model, aredynamically estimated using a SwitchingHypothesized Measurements (SHM) model.Experimental results demonstrate that theproposed method accurately and robustlytracks facial features in real time underdifferent facial expressions and face poses.IJERTtime consuming alternative to measuringfacial expression that is aligned withdimensional models of emotion. The systemprovides information about the frequency,intensity, valence, and duration of facialexpressions. The selection of the variablesincluded in the system was based on theoryand previous empirical studies. Adopting thedescriptive style of Ekman and similar to thework of Notarious and Levenson (1979), anexpression is defined as any change in theface from a neutral display (i.e., noexpression) to a non-neutral display and backto a neutral display.When this activity occurs, afrequency count of expressions is initiated.Next, coders rate the valence (positive ornegative) and the intensity of eachexpression detected. Notice that this is quitedifferent from assigning an emotion term toeach expression. While FACES requirescoders to decide whether an expression ispositive or negative, it does not require theapplication of specific labels. That is,judgments about emotion, in this casewhether an expression is positive ornegative, are made by persons who areconsidered to be familiar with emotion in aparticular culture. In addition to valence andintensity, coders also record the duration ofthe expression.They have developed Facial ActionCoding System for facial expressions basedon discrete emotions specifically designed tomeasure human muscle movements. Theyfirst measure only two facial expressionssuch as anger or happiness. Most of thediscrete emotions are designed to measureonly basic emotions. They achieved how toidentify human facial expression with theFACS model. But, this approach fails toexpose all human facial movements. Thisapproach is inconsistent to measure the facialdimensions. In future they would like towww.ijert.orgThey presented a posed expressionfor human face. Because of facial expressionrecognition is very challenging due to posedexpressions. They developed multi-statepose-dependent hierarchical shape model fortracking varying face pose and faceexpression. They proposed posed expressionmethod for facial recognition and achievedsmall pose variation in human face. Thisapproach will lose to identify large posevariation. It improves the accuracy androbustness of facial feature tracking. Need19
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference Proceedingslot of training dataset. In future they plan tocombine the relationship between differentfacial expressions.III EXISTING SYSTEMFacial feature points encode criticalinformation about face shape and face shapedeformation. Accurate location and trackingof facial feature points are important in theapplications such as animation, computergraphics, etc. Model free approaches aregeneral purpose point trackers without theprior knowledge of the object. Each featurepoint is usually detected and trackedindividually by performing a local search forthe best matching position.IV PROPOSED SYSTEMIn proposed a hierarchical framework basedon Dynamic Bayesian Network (DBN) forsimultaneous facial feature tracking andfacial expression recognition.By systematically representing and modelinginter relationships among different levels offacial activities, as well as the temporalevolution information, the proposed modelachieved significant improvement for bothfacial feature tracking and AU recognition,compared to state of the art methods.IJERTFacial expression recognitionsystems usually try to recognize either sixexpressions or the AUs. Image-basedapproaches, which focus on recognizingfacial actions by observing the representativefacial appearance changes, usually try toclassify expression or AUs independentlyand statically.Figure 5.1 Block diagram of facialThe idea of combining trackingwith recognition has been attempted before,such as simultaneous facial feature trackingand expression recognition and integratingface tracking with video coding. However,the model free methods are susceptible to theinevitable tracking errors due to the apertureproblem; noise, and occlusion, Simpleparallel mechanism may not be adequate todescribe the interactions among facial featurepoints.Discrete states still cannot describe thedetails of each facial component movement,i.e., only three discrete states are notsufficient to describe all mouth movementwww.ijert.orgexpression identificationV MODULES DESCRIPTION1) Preprocessing & Segmentation:The first step is to adjust the gray level of aflame image according to its statisticaldistribution. This segmentation process to behandled according to the image size thesegmentation blocks will vary like 8X16,4X16.Segmentation is a process of extracting andrepresenting information from an image is to20
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference Proceedingsgroup pixels together into regions ofsimilarity.2) Dataset preparing:The overall possible datasets are collectedfrom all the sources to identify the particularhuman face expressions. Feature extractingbased on the datasets. So while collectingdatasets, must add all the possibleexpressions from the social people.Trackedthefacialfeaturepointmeasurements through an Active ShapeModel (ASM) based approach [1], whichfirst searches each, point locally and thenconstrains the feature points based on theASM model, so that the feature points canonly deform in specific ways found in thetraining data. The ASM model is trainedusing 500 key frames selected from thetraining data, which are 8-bit gray imageswith 640 480 image resolution. All the 26facial feature point positions are manuallylabeled in each.IJERTThe proposed model is evaluated on twodatabases, i.e., the CK database and MMIfacial expression database. The advantage ofusing this database is that it contains a largenumber of videos that display facialexpressions.the facial feature tracking performance. Thisfacial feature tracking could be classifiedinto two categories: Model free and Modelbased method. Model free method usuallydetected matching position. Model basedmethod such as ASM (Active Shape Model).ASM to improve the robustness and accuracyof feature point tracking. For example, formouth, they used three ASMs to representthe three states of mouth, i.e., widely open,open and closed.b) AU classification:List of AUs and their relationships3) Measurement Extractiona) Facial Feature TrackingFacial feature, which usually detects andtracks important facial feature points (i.e. thefacial landmarks) and facial components (i.e.mouth, eyes) capture the details of face shapeinformation. Facial feature tracking can beused to extract the face expression, and thisresult can provide a prior distribution forfacial feature points. DBN helps to improvewww.ijert.orgModel-based methods overcome thisweakness by making use of the relationshipsamong AUs, and recognize the AUssimultaneously [6]. Facial expressionrecognition systems usually try to recognizeeither six expressions or the AUs. Over thepast decades, there has been extensiveresearch on facial expression analysis.Current methods in this area can be groupedinto two categories: image-based methodsand model-based methods.Image-based approaches, which focus onrecognizing facial actions by observing therepresentative facial appearance changes,usually try to classify expression or AUsindependently and statically. This kind of21
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference Proceedingsmethod usually consists of two key stages.First, various facial features, such as opticalflow, explicit feature measurement (e.g.,length of wrinkles and degree of eyeopening), Local Binary Patterns (LBP),Independent Component Analysis (ICA),etc., are extracted to represent the facialexpressions or facial movements. Given theextracted facial features, the expression/AUsare identifies by recognition engines, such asSupport Vector Machines (SVM).Model based methods overcome thisweakness by making use of the relationshipsamong AUs, and recognize the AUssimultaneously.c) Facial expression identification:VI CONCLUSIONA hierarchical framework based on DynamicBayesian Network for simultaneous facialfeature tracking and facial activityrecognition systematically representing andmodeling inter relationships among differentlevels of facial activities, as well as thetemporal evolution information, the proposedmodel achieved significant improvement forboth facial feature tracking and AUrecognition, compared to state of the artmethods.Proposed method did not use anymeasurement specifically for expression, andthe global expression is directly inferredfrom AU and measurements, with improvedimage-based computer vision technology.IJERTGiven the facial action model [9] and imageobservations, all three levels of facialactivities are estimated simultaneouslythrough a probabilistic inference bysystematicallyintegratingvisualmeasurements with the proposed model.integrating visual measurements with theproposed model.Compared to the previous related works,proposed work has the following features.1) First, build a DBN model toexplicitly model the two-way interactionsbetween different levels of facial activities.In this way, not only the expression and AUsrecognition can benefit from the facialfeature tracking results, but also theexpression recognition can help improve thefacial feature tracking performance.2) Second, recognize all three levels offacial activities simultaneously. Given thefacial action model and image observations,all three levels of facial activities areestimatedsimultaneouslythroughaprobabilistic inference by systematicallywww.ijert.orgProposed system may achieve better resultswith little changes to the model.Facialfeature point measurements and from theirrelationships.The improvements for facial feature pointsand AUs come mainly from combining thefacial action model with the imagemeasurements.Specifically, the erroneous facial featuremeasurements and the AU measurements canbe compensated by the model’s build-inrelationships among different levels of facialactivities, and the build-in temporalrelationships.This proposed model systematically capturesand combines the prior knowledge with theimage.22
International Journal of Engineering Research & Technology (IJERT)ISSN: 2278-0181NCICCT' 14 Conference ProceedingsVII FUTURE ENHANCEMENTIn the future work, plan to introduce the rigidhead movements, i.e., head pose, into themodel to handle multi view faces. Inaddition, modeling the temporal phases ofeach AU, which is important forunderstanding the spontaneous expression, isanother interesting direction to pursue. Andalso plan to introduce the image extractionfrom blur image, i.e., would like to convertoriginal image by using Dynamic BayesianNetwork with Navye Bayes Classifier. Andalso would like to understand morerelationship between Facial Expression.VIII REFERENCESIJERT1) T. F. Cootes, C. J. Taylor, D. H. Cooper,and J. Graham, “Active Shape ModelsTheirTrainingandApplication,”Computer Vision and Image Understanding.,vol.61, no.1, pp. 38-59, 1995.2) B. A. Draper, K. Baek, M. S. Bartlett, andJ. R. Beveridge, “Recognizing Faces WithPCA and ICA,” Computer Vision andImage Understanding., vol.91, no.1-2,pp.115-137, 2003.3) P. Ekman and W. V. Frisen, “FacialAction Coding System(FACS),”Manual. Palo Alto, CA, USA: ConsultingPsychologists Press., vol.45, no.15, pp. 89111, 1978.4) X. W. Hou, S. Z. Li, H. J. Zhang, and Q.S. Cheng, “Direct appearance models,” inProcessing IEEE Conference ComputerVision Pattern Recognition., vol. 1, pp. 828–833,2001.5) Jacob Richard-Whitehill, “AutomaticReal-Time Facial Expression RecognitionforSigned Language Translation”,Department of Computer Science, Universityof the Western Cape., vol.32, no. 56, pp.115-167, 2006.6) S. J. McKenna, S. Gong, R. P. Würtz, J.Tanner, and D. Banin, “Tracking facialfeature points with Gabor wavelets andshape models,” in Processing InternationalConference Audio- Video-Based BiometricPerson Authentication., vol. 1206, pp. 35–42,1997.7) G. R. S. Murthy, R.S.Jadon“Effectiveness of Eigen spaces for FacialExpressions Recognition” , Vol. 1, no. 5, pp. 1793-8201,2009.8) Peng Yang, Qingshan Liu, Dimitris N.Metaxas, “Boosting Coded DynamicFeatures for Facial Action Units andFacial Expression Recognition”, IEEE .,vol.51, no. 42 pp. 110-142, 2007.9) C. Shan, S. Gong, and P. W. McOwan,“Facial Expression Recognition Based OnLocal Binary Patterns: A ComprehensiveStudy,” Image and Vision Computing.,vol.27, no.6, pp. 803-816, 2009.10) M. Valstar and M. Pan
recognition, facial feature tracking, simultaneous tracking and recognition. I INTRODUCTION The recovery of facial activities in image sequence is an important and challenging problem. In recent years, plenty of computer vision techniques have been developed to track or recognize facial activities in three levels. First, in the
Simultaneous Facial Feature Tracking and Facial Expression Recognition Yongqiang Li, Yongping Zhao, Shangfei Wang, and Qiang Ji Abstract The tracking and recognition of facial activities from images or videos attracted great attention in computer vision field. Facial activities are characterized by three levels: First, in the bottom level,
simultaneous facial feature tracking and expression recognition and integrating face tracking with video coding. However, in most of these works, the interaction between facial feature tracking and facial expression recognition is one-way, i.e., facial feature tracking results are fed to facial expression recognition. There is
Another approach of simultaneous facial feature tracking and facial expression recognition by Li et.al [21] describes about the facial activity levels and explores the probabilistic framework i.e. Bayesian networks to all the three levels of facial involvement. In general, the facial activity analysis is done either in one level or two level.
simultaneous tracking and recognition of facial expressions. In contrast to the mainstream approach "tracking then recognition", this framework simultaneously retrieves the facial actions and expression using a particle filter adopting multi-class dynamics that are conditioned on the expression. 2. Face and facial action tracking
KEYWORDS: Simultaneous Tracking and Recognition, Facial Feature Tracking, Facial Action Unit Recognition, Expression Recognition and Bayesian Network. I. INTRODUCTION The improvement of facial activities in image sequences is an important and challenging problem. Nowadays, many
facial landmark localization, it remains an unsolved prob-lem when applied to facial shape tracking in the real world video due to the challenging factors such as expression, illu-mination, occlusion, pose, image quality and so on. A suc-cessful facial shape tracking includes at least two character-istics.
facial feature tracking can be used in the feature extraction stage in expression/AUs recognition, and expression/ AUs recognition results can provide a prior distribution for facial feature points [1]. However, most of the current methods only recognize the facial activities in one or two levels, and track
Coronavirus germs live in people’s throats and mouths. When someone who has the coronavirus coughs or sneezes or breathes out, the germs come out of their mouth in tiny drops of water. It’s easy to get the coronavirus germs from inside your body on your hands when you touch your nose or your mouth. If the person with the coronavirus germs on their hands uses a door, the invisible germs can .
