您的当前位置：首页 Specifying and animating facial signals for discourse in embodied conversational agents. Co

Specifying and animating facial signals for discourse in embodied conversational agents. Co

来源：意榕旅游网

COMPUTERANIMATIONANDVIRTUALWORLDS

Comp.Anim.VirtualWorlds2004;15:27–38(DOI:10.1002/cav.5)

******************************************************************************************************

Specifyingandanimatingfacialsignalsfordiscourseinembodiedconversationalagents

ByDougDeCarlo*,MatthewStone,CoreyRevillaandJenniferJ.Venditti

******************************************************************************************************

Peoplehighlighttheintendedinterpretationoftheirutteranceswithinalargerdiscoursebyadiversesetofnon-verbalsignals.Thesesignalsrepresentakeychallengeforanimatedconversationalagentsbecausetheyarepervasive,variable,andneedtobecoordinatedjudiciouslyinaneffectivecontributiontoconversation.Inthispaper,wedescribeafreelyavailablecross-platformreal-timefacialanimationsystem,RUTH,thatanimatessuchhigh-levelsignalsinsynchronywithspeechandlipmovements.RUTHadoptsanopen,layeredarchitectureinwhichﬁne-grainedfeaturesoftheanimationcanbederivedbyrulefrominferredlinguisticstructure,allowingustouseRUTH,inconjunctionwithannotationofobserveddiscourse,toinvestigatethemeaningfulhigh-levelelementsofconversationalfacialmovementforAmericanEnglishspeakers.Copyright#2004JohnWiley&Sons,Ltd.Received:8April2003;Revised:8April2003

KEYWORDS:

facialanimation;embodiedconversationalagents

Introduction

Whenpeoplecommunicate,theysystematicallyemployadiversesetofnon-verbalcues,andhighlighttheintendedinterpretationoftheirutterances.ConsidertheexampleinFigure1(a),theﬁnalsegmentofabriefnewsstoryasreadbyJudyFortinonCNNheadlinenewsinOctober2000:

NASAscientistshavespottedsomethingﬂoatinginspacethat’sheadedourway.Butthey’renotsureifit’sanasteroidorpartofanoldspacecraft.TheoddsareoneinﬁvehundredtheunidentiﬁedobjectwillcollidewithEarth—fargreaterthananysimilarob-jecteverdiscovered.

JudyFortin’sexpressivemovementsinFigure1(a)includeatiltingnodtoherleftinsynchronywithwordsfargreaterwhichsheuttersasasinglespeechunit;raisedeyebrowsonthephraseanysimilarobject,alongwithabriefdownwardnodonsimilar;andanupward(andalsoslightlyrightward)headmotiononever.Weusethetermfacialconversationalsignalstorefertomovementssuchasthese.Incontext,thesemovementslinktheutterancewiththerestofthestory.Theyjuxtaposethe

*Correspondenceto:DougDeCarlo,DepartmentofComputerScienceandCenterforCognitiveScience,RutgersUniversity,Piscataway,NJ08854-8019,USA.E-mail:decarlo@cs.rutgers.edu

unidentiﬁedobjectwithalternativespaceobjects,em-phasizethewiderangeofobjectsbeingconsidered,andhighlighttheunidentiﬁedobject’suniqueness.Theytherebycallattentiontothepointofthestory—whythispossiblecollisionwithEarth,animprobableeventbyordinarystandards,remainsnewsworthy.

Thesemovementsarequitedifferentincharacterfromtheinterpersonalandaffectivedimensionsthathavebeeninvestigatedinmostpriorresearchoncon-versationalfacialanimation.Forexample,Cassellandcolleagues1,2havecreatedagentsthatuseanimatedheadandgazedirectiontomanagespeakingturnsinface-to-faceconversation.NagaoandTakeuchi3andPoggiandPelachaud4,5havecreatedagentsthatpro-ducespeciﬁcemblematicdisplays(thatis,completeexpressionsinvolvingbrows,mouth,eyesandhead,withasinglemeaning)toclarifyinteractionwithauser.Animatedemotionaldisplays(andcorrespondingdif-ferencesinpersonality)havereceivedevenwideratten-tion.6–10ThemovementsofFigure1(a)donotengagetheseinterpersonaloraffectivedimensions;theysignalinternalsemanticrelationshipswithinJudyFortin’spresentation.

Althoughthesesignalsandtheirinterpretationshavenotbeenmuchstudied,webelievethattheyrepresentakeychallengeforanimatedconversationalagents,be-causetheyaresopervasiveandsovariable.Inexplora-torydataanalysiswehavefoundthat,asinFigure1(a),smallheadmovementsrelatedtodiscoursestructure

******************************************************************************************************

D.DECARLOETAL.

******************************************************************************************************

Figure1.Naturalconversationalfacialdisplays(a,top),ahigh-levelsymbolicannotation(b,middle),anda

synthesizedautomaticallyfromtheannotation(c,bottom).andinterpretationareamongthemostcommonnon-verbalcuespeopleprovide.AndFigure1(a)alreadyshowsthreequalitativelydifferentheadmovementswhicheachsuitthesynchronousspeech.

Inthispaper,wedescribeafreelyavailablecross-platformreal-timefacialanimationsystem,RUTH(forRutgersUniversityTalkingHead),whichanimatessuchsignalsinsynchronywithspeechandlipmovements.RUTHadoptsanopen,layeredarchitectureinwhichﬁne-grainedfeaturesoftheanimationcanbederivedbyrulefrominferredlinguisticstructure.RUTHthereforeac-ceptsinputsimplyandabstractly,asacompactsym-bolicdescriptionofconversationalbehavior.Humananalystscanproducesuchspeciﬁcationsforobserveddata,throughtheprocesswerefertoascodingorannotation.

Forexample,Figure1(b)givesasenseofRUTH’sinputbypresentingtheannotationthatagroupoffouranalystsarrivedatincodingtheoriginalCNNfootagefromFigure1(a).TheintonationisspeciﬁedaccordingtheTonesandBreakIndices(ToBI)stan-dard;11,12LþH*,!H*andLþ!H*markaccentsonsyllables,whileH-,L-andL-L%recordtonesattheboundariesofprosodicunits.Theconversationalbrowmovementsarecategorizedintermsofthefacialactionunit(AU)involved,followingEkman;131þ2istheactionunitfortheneutralbrowraise.Finally,theheadmovementsarelabeledbynewcategoriesthatweobservedfrequentlyinourdata:TLforatiltingnodonaphrase;D*foradownwardnodaccompanyingasinglesyllable;andU*foranupwardnodaccompany-ingasinglesyllable.

RUTH

animation

TheannotationofFigure1(b)exhibitsatypicalpar-allelbetweenverbalandnon-verbalchannels:unitsofmotioncoincidewithunitsofspeechphrasingandpeaksofmovementcoincidewithprominentsylla-bles.13–16RUTH’sanimationretainsthisunity,becauseRUTHorchestratestherealizationofnon-verbalsignalsandspeechsoundsandmovementsaspartofasingleprocesswithaccesstorichinformationaboutlanguageandaction.Figure1(c)displaysstillshotsfromRUTH’srenditionoftheannotation.ThecomparisonisnotthatthemotionsofFortinandRUTHareidentical—thesym-bolicinputthatdrivesRUTHismuchtooabstractforthat—butthatthemotionsaresufﬁcientlyaliketomeanthesame.

RUTHimplementsapipelinearchitecturewithwell-deﬁnedinterfaceswhichcanlinkupeitherwithinternalmodulesorexternalapplications.Atthelowestlevel,RUTHanimatesascheduleofanimationinstructionsforourlifelikecharacter(thoughnotananatomicallyrea-listicone),byapplyingdeformationstoapolygonalmesh,inpartusingadominance-basedco-articulationmodel.17–19Ahigherlevelderivesascheduleofanima-tioninstructionsfromannotatedtext,byinstrumentingtheinternalrepresentationsofthepublic-domainspeechsynthesizerFestival20tokeeptrackofsynchro-nousnon-verbaleventsandﬂeshthemoutintoanima-tioninstructionsusingcustomizablerules;furtherutilitieshelpsupportRUTH’susefordialogueresearchandinconversationalsystems.RUTHisavailableforuseinresearchandeducationfromourwebsite:

http:==www:cs:rutgers:edu=evillage=ruth

******************************************************************************************************

SPECIFYINGANDANIMATINGFACIALSIGNALS

******************************************************************************************************

easilyachievesreal-timeframerates(i.e.,30persecondorbetter)onanymoderndesktopcomputerwith3Dgraphicshardware.

RUTHrequiresannotatedinputratherthanplaintextbecauseintonation,facialexpressionsandheadmove-mentscanoftenaddsomethingnewtotheinterpreta-tionofanutterance;theyarenotalwaysredundant.BavelasandChovil21offerarecentsurveyofthepsy-chologicalevidenceforsuchanintegratedmessagemodelofface-to-facecommunication.Onthisview,theindependentcontributionoffacialsignalscannotbederivedfromtext(automaticallyorotherwise);ithastobespeciﬁedseparately.ThusourperspectivecontrastswithapproachestofaceanimationsuchasPerlin’s22,23orBrand’s,24andanimatedagentssuchasSmidandPandzic’s,25whereanimationisdrivenfromgenerativestatisticalmodelsbasedsolelyonthetext.RUTH’sannotatedtextinputenablesresearcherstoexpe-rimentwithmeaningfulwaysofselectingintonation,facialexpressionsandheadmovementstocomplementsimultaneousspeech.RUTHisalsocompatiblewithtextinput,ofcourse.Forexample,RUTHcanbeusedwithsystemsthatautomaticallyannotatetextforembodieddelivery,suchasCassellandcolleagues’BEATsys-tem.26Alternatively,simpleheuristicstoannotatetextcanbequiteeffectiveinconstraineddomains.Never-theless,humanjudgmentsarestillnecessarytovarythesignalsofembodiedconversationmeaningfully.

RUTH

Implementation

Architecture

ThearchitectureofRUTHisdiagramedinFigure2.Theprogramconsistsofatierofindependentthreadsthat

usequeuestocoordinateandcommunicate.Thequeueimplementationenforcesmutualexclusionforqueueoperations,andallowsthreadswaitingonthequeuetosuspenduntilthestateofthequeuechanges.Thissema-nticsmakesthemultithreadedimplementationofstagesinthepipelinesimpleandelegant.

Thehighest-levelthreadisthecommandthread,whichinterfaceswithinteractiveapplications.Thecommandthreadacceptsandpostsabstractrequestsforanima-tion,suchastofollowapre-computedscript,tosynthe-sizespeechandcontrolinformationforanewutterance,ortointerruptanongoinganimation.

Nextistheloaderthread,whichsupportsﬂexibleprocessinginlinkinganimationwithspeechdata.Theloaderthreadisresponsibleforpopulatingarealizationqueuewithspeciﬁcactionstoanimateatprecisetimesrelativetothestartofspeech.Itimplementsanumberofalternativestrategiesformarshalingtherequiredinfor-mation,includingcommunicationwiththeFestivalspeech-synthesisserver20andaccesstopre-computeddata.

Finally,thedisplaythreadandthesoundthreadcoordi-natetorealizetheanimation,throughcarefuldeploy-mentofoperating-systemsprimitivesforconcurrency.Thedisplaythreadupdatesmodelgeometryandren-dersframesonareal-timescheduledrivenbyaglobalanimationclock.Thesoundthreadsendsdatatotheaudiodeviceinsmallunits(enablinggracefulinterrup-tion),andmonitorstheresultstokeeptheplayingsoundandtheanimationclockinagreement.

Model

supportsdeformablepolygonalmodels.Wecom-bineacommonunderlyinggeometryofthemodelwithasetofdeformations,parameterizedfrom0(represent-ingnodeformation)to1,whichrepresentindependentqualitativechangestothemodel.Currentdeformationsdescribethemouthmovementsandtonguemovementsinvolvedinspeech,asinFigure3;seealsoCohenandMassaro.17Therearealsodeformationsforbrowactionunits1(innerraise),2(outerraise),and4(frowning),smilingandblinking.Weapplyadeformationbyadd-ingoffsetstotheunderlyinggeometry;theoffsetisinterpolatedfromkeyoffsetvaluesasapiecewiselinearfunctionofthedeformationparameter.RUTHalsoper-mitsrotationsandtranslationsoverpartsofthemodel:theeyesrotate;theheadrotatesandtranslates,main-tainingasmoothjoinwiththeneck.Attheboundariesofparts,theeffectofthetransformationfadesoutgradu-allyacrossapre-speciﬁedregion.

RUTH

Figure2.ThearchitectureofRUTH.

******************************************************************************************************

D.DECARLOETAL.

******************************************************************************************************

Parameterrotatejawstretchmouthlowercornersroundupperlipraiseupperlippoutlowerliplowerlowerliptucklowerlipraisetonguesticktongueout

Effect

opensthemouthusedforlowvowelstightensthelips

commoninmanyvisemes

givesthelowerlipanarchedlookseenparticularlyinp,bandm

givestheupperliparoundedshape

seenforexamplewithroundedconsonantwraiseslipwithlessroundingseenforexampleinshbringslowerlipforwardseenforexampleinsh

givesthelowerliparoundedlookseenforexampleinw

drawsthelowerlipbackundertheteethseenparticularlywithfandvdrawsthetongueuptothepalateseenparticularlywithtandd

drawsthetongueoutoverandpasttheteethseenparticularlywithth

Figure3.DeformationsforvisiblespeechinRUTH.

Ourmodelandsomeofitsdeformationsareillu-stratedinFigure4.Indesigningthemodel,wehaveadoptedtheaestheticofillustrationratherthanthatofphotorealism,inordertoobtainanattractiveandbeliev-ableresultwithinreasonablecomputationaldemands.Inall,themodelhassome4000polygons;appearanceisdeterminedbyvaryingmaterialpropertiesratherthantexture.Wehave,moreover,attemptedtokeepthemodelrelativelyambiguousastosex,race,andage(e.g.,elementaryschooltoyoungadult);thisway,aswidearangeofusersaspossiblecanregardthemselvesandRUTHasmatched,animportantaspectofusability.27RUTHimplementsmouthmovementsforspeechusingadominance-basedco-articulationmodel;17–19seeKing18forexplanationandfurtherreferences.Theani-mationschedulespeciﬁesvisemes,categoriesoffacial

appearancethatcorrespondtoparticularcategoriesofspeechsounds.Visemeshavegoals,particularpara-metersforoffsetdeformationsatpeak;anddominancefunctions,whichcharacterizehowvisiblethesedeforma-tionsareinarticulationasafunctionoftime.Deforma-tionsthataffectthelips(suchassmiling)alsosupplydominancefunctionswhichfactorintothecomputationofspeechlipshapes.Mouthoffsetsineachframearecomputedbyapplyinggoalsforactivevisemesinrelativeproportiontotheircurrentdominance.

Animationforotherfacialactionscombinesagoalwithaparameterizedanimationtemplate,whichdirectlydescribesthedegreetowhichthegoalisachievedovertime.Individualactionsarethenspeciﬁedintermsofstarttime,endtime,peakintensity,attackanddecay.Figure5showshowwesynchronizetheseparameters

Figure4.RUTH’sunderlyinggeometry;deformationsfor1þ2,jawopening,puckeringmouthcornersandraisingupperlip.

******************************************************************************************************

withprosodicfeaturesinspeech.Actionsthatspanprosodicunitspeakfromthestartoftheﬁrstaccentinaphrasetotheendofthelastaccentinthephrase;theyrampupgraduallyatthestartofthephraseandfalloffgraduallyattheend.Actionsthathighlightindividualwordspeakjustonanaccentedsyllable.Thesetem-plateslinkcoarsespeciﬁcationsforconversationalac-tionstoconcreteanimationparameters,andthusunderlieRUTH’sabilitytoacceptqualitative,symbolicspeciﬁcations.Weofferahigher-levelperspectiveonthissynchronyinanimationwhenwedescribetheuseofRUTHlater.ThegeometrythatRUTHrendersforeachframeofanimationaddsthecomputedmouthoffsetsandthecomputedactionoffsetsforthattimetotheunderlyinggeometryofthemodel.

InterfacingwithSpeech

Keepingtrackofanimationduringtheprocessofspeechsynthesisisaperennialproblem.Wehaveinstrumentedtheopen-sourceFestivalspeechsynthesissystem20sothatitsynthesizestimingdataforspeechandanimationasanintegratedwhole.RUTH’sloaderthreadincludesaclientfortheresultingtext-to-timed-animated-speechserver,andRUTH’scommandthreadacceptsa‘synthe-size’commandwhichinstructstheloadertosendspeciﬁcmarked-uptexttoFestivalandtoanimatetheresults.

Festivalrepresentslinguisticstructuresusinggeneralgraphrepresentations.Nodesinthesegraphscorre-spondtoutteranceelements,includingsuchconstructsaswords,phrases,phonemesandtones.Aseparategraphdescribestherelationshipsamongelementsateachlinguisticlevel;elementscanalsohavearbitraryfeatures,includingfeaturesthatestablishlinksbetween

levelsoflinguisticanalysis.Inpututterancesarelistsofmarked-upwords;eachlistelementspeciﬁesawordand(optionally)alistofattribute–valuepairswhichspecifyhowthewordistoberealized.Forexample,suchattribute–valuepairscanspecifytheprosodywithwhichtorealizetheutterance.Theprocessoftext-to-speechinvolvesrepeatedlyenrichingthelingui-sticrepresentationofthisinput,byaddingnewrelation-ships,elementsandfeatures.Thisprocessismanagedbyafullycustomizableﬂow-of-controlininterpretedScheme.Eventually,thisprocessdeterminesacompletephoneticdescriptionofanutterance,includingpho-nemes,pitch,junctures,andpausesandtheirtiming;synthesisiscompletedbyacousticoperations.

Festival’sﬂexible,openarchitecturemeshesnaturallywiththerequirementsofanimation.WespecifyFestivalinputwithfeaturesonwordsforheadandbrowactionsaswehavecodedthem.Figure6givesanexampleofsuchinput.WeaddrulesfortimingtheseactionstoFestival’stext-to-speechprocess.BecauseofFestival’sdesign,theserulescandrawonstructuralandphoneticconsiderationsintheutterance(asinFigure5)byexploringitsﬁnalphoneticdescription.Wecanalsocustomizeremainingquantitativeparametersforspeci-ﬁcanimationactions.Weaddaﬁnaltraversalofutter-ance’sphoneticrepresentationsothattheservercanoutputaseriesofvisemesandanimationcommandscorrespondingtoasynthesizedwaveform.ForRUTH,wehavealsoreinstrumentedFestival(debuggingandex-tendingthestandardrelease)tocontrolpitchbyannota-tion;28,29weuseOGICSLUsynthesisandvoices.30AnimationschedulesandspeechwaveformsoutputbyFestivalcanbesaved,reusedandmodiﬁeddirectly.Thismakesiteasytovisualizelow-levelvariationsintimingandmotion.(Inthecommandthread,a‘save’instructionconstructsﬁlesforinputthatwillreproduce

D.DECARLOETAL.

******************************************************************************************************

Figure6.TaggedspeechinputtoFestivalcorrespondingtoFigure1(b);ﬁlesuse‘jog’forheadmotionsandsingletags(e.g.

‘(jog)’)tosignalendsofmovements.themost-recentlyrealizedanimation;the‘canned’in-structionreplaystheanimationfromaspeciﬁedﬁle.)Wealsosupportsimilarvisualizationsinvolvingre-cordedspeech,drawingonoff-the-shelftoolstoputwaveformsintemporalcorrespondencewiththeirtran-scriptsandtoannotatetheresults.

downstepped(annotatedby!asin!H*)toalowerpitchvalue(andlowerprominence).Pitch‘accents’arespe-ciﬁedtoRUTHasvaluesofaword’s‘accent’attribute.WordsaregroupedintotwohierarchicallevelsofprosodicphrasinginEnglish:thesmallerintermediatephraseandthelargerintonationphrase.Anintermedi-atephraseismarkedbyahigh(H-)orlow(L-)toneimmediatelyafterthelastaccentedsyllableinthephrase,andanintonationphraseisadditionallymarkedbyahigh(H%)orlow(L%)toneattherightphraseedge.CommonpatternsforintonationphrasesthusincludethefalloftenfoundindeclarativestatementsL-L%,theriseoftenfoundinyes–noquestionsH-H%,andacombinedfall–riseL-H%associatedgenerallywithcontributionstodiscoursethataresomehowin-complete.PhraseandboundarytonesarespeciﬁedtoRUTHasvaluesofthe‘tone’attribute,whichaccompa-niestheﬁnalwordinaphrase.

ToBIofferssophisticatedresourcesforcharacterizingthepitchcontourofEnglishutterances,intermsthatcorrelatecloselywiththemeaningsthatprosodicvaria-tioncanconveyinparticulardiscoursecontexts;seePierrehumbertandHirschberg.31Researcherscancallupontheseresourcesindecidingtorealizeembodiedutteranceswithsuitableintonation.However,richvar-iationisnotalwaysnecessary;forexample,itworksquitewelltojustputanH*oncontentwordsthathavenotbeenusedbeforeinthediscourse,32andtoputanL-orL-L%atnaturalboundaries,aftereveryfewcontentwords.Thesestrategiesofferasimplealterna-tiveforpreparingspeciﬁcationsforRUTHbyhand,orforwritingalgorithmsthatconstructthemautomatically.AnotherimportantaspectofEnglishprosodyispitchrange,theextremesofhighandlowthatareattainedoverawholephrase.Thisisalsoknownastheregisterofspeech.ToBIlabelsdescribethequalitativechangesinpitchwithrespecttowhateverpitchrangehappenstobeineffect.Butchangesinoverallpitchrangehelptosignaltheorganizationofdiscourse:atthebeginningsofdiscoursesegments,pitchrangeisexpandedandatthe

DrivingRUTHwithAnnotatedText

ThemostabstractwaytospecifyananimationforRUTHistosupplyRUTHwithtextthathasbeenmarked-uptospecifytheheadmotionsandotherfacialactionsthatshouldoccurasthetextisuttered.ThissectiondescribestherangeofdeliverythatRUTHsupportsandgivessomehintsabouthowtouseRUTH’sanimationcapabilitiesinthemostmeaningfulway.

RUTHInputanditsMotivation

Tospecifyprosody,RUTHusestheTonesandBreakIndices(ToBI)modelofEnglishintonation.11,12InToBI,prosodicstructureisdescribedintermsofphrasing,clusteringofwordsintogroupsdelimitedbyperceiveddisjuncture,andaccentuation,theperceivedprominenceofparticularsyllableswithinagroupofwords.Intona-tionaltuneisspeciﬁedbysymbolicannotationsthatdescribethequalitativebehaviorofpitchataccentsandphrasalboundaries.IntheToBIlabeling,eachutteranceisrequiredtoconsistofoneormorephrases.Eachphrasemustendwithappropriatephraseorboundarymarkers,andeachphrasemustcontainatleastoneaccentedword.

TheEnglishtonalinventoryincludespitchaccentssuchashigh(H*),low(L*),orrisingaccentsthatdifferinwhethertheriseprecedes(LþH*)orfollows(L*þH)thestressedsyllable.Accentswithahightonalcompo-nentaregenerallyrealizedhighinthespeaker’spitchrangeforthephrase,butcansometimesbe

******************************************************************************************************

SPECIFYINGANDANIMATINGFACIALSIGNALS

******************************************************************************************************

endsofdiscoursesegmentspitchrangeiscontractedandgenerallylowered.33Inaddition,thegenerallevelofpitchisasignalofaspeaker’sinvolvementinwhattheysay:moreimportantcontributionsaredeliveredwithhigherpitch.34Varyingpitchrangeisthusessentialtogivethevariabilityandorganizationofnaturalspeech.RUTH’sconventionisthata‘register’attributeontheﬁrstwordofaphrasesetsthepitchrangeforthewholephrasetooneofafewqualitativevalues.(Theconven-tionappliesforallintermediatephrases,notjustintona-tionphrases.)RUTH’squalitativevalues,asgiveninFigure7,arederivedfromtheworkofMoehlerandMayer.29,35RUTH’smodelsoffacialconversationalsignalsbuildonthisspeciﬁcationofprosody.Ournewmovementsmayfunctionasunderlinersthataccompanyseveralsuccessivewords,orasbatonsthathighlightasingleword.13Incalculatingthetemporaldynamicsofunder-linersandbatons,RUTHbuildsfromtheclosesynchronythatresearchers13–16havefoundbetweenembodiedactionandsimultaneousspeechinconversation.WeanticipatedthisalreadyindiscussingFigure5.RUTHassumesthatunderlinersspancompleteintermediateorintonationphrases.ThisallowsRUTHtoensureauto-maticallythatthemovementappearstopeakinsyn-chronywiththeﬁrstprosodicemphasisinaphraseandtobereleasedafterthelastprosodicemphasisinaphrase.Similarly,RUTHassumesthatbatonsonlyoccuronwordsthatarespeciﬁedforaccent,andtimesthepeakofthebatontosynchronizewiththestressedvowel.

Aligningconversationalfacialsignalswithspeechthiswaycanhelptosettledifﬁcultannotationdecisionsinaprincipledway.Itisquitedifﬁculttoannotatebeginningsandendsofbrowmovements,forexamplebylookingatavideorecordofaconversation.Thetypicaldifﬁcultyisjudgingwhereamovementstartsorendswithinaseriesofshortunaccentedwords.

HH^HH^LLL^LHLHL^Hprimaryhighregister(default)expandedhighregistercompressedhighregisterprimarylowregistercompressedlowregisterexpandedregisterincludinglowsandhighsfullpitchrange

Figure7.PossiblespeciﬁcationsofpitchrangeforRUTH.29,35Figure1isrepresentative:thephrasethananysimilarobjectbeginsandendswithunstressedsyllables.Coderswhohavetochooseseparatelywhethertoincludethanoranyasmarkedwithabrowraisefaceadifﬁcultandprobablymeaninglessjudgment.

InRUTH’sinput,aseparateattributeofwordscontrolseachindependentdimensionoffacialmovement.Foreachattribute,RUTHpermitsatmostoneunderlinerandatmostonebatonatatime;alabeledwordeithermarksthebeginningortheendofanunderlinerorcarriesabaton.RUTHadoptstheconventionthatbatonlabelsendin*,whilecorrespondingunderlinerlabelsomitthe*.RUTHfollowsEkmaninclassifyingbrowmovementsintermsofthefacialactionunit(AU)involved;AUsarepatternsofchangeinthefacethattrainedexpertscancodeandsometimesevenperformreliably.13BrowmovementsaremadeupofAU1,whichraisestheinsideofthebrow;AU2,whichraisestheoutsideofthebrow;andAU4,whichnarrowsanddepressesthebrow.RUTHcurrentlyimplementsaneutralraise,speci-ﬁedasvalues‘1þ2’or‘1þ2*’fortheattribute‘brow’,andaneutralfrown,speciﬁedasvalues‘4’or‘4*’.RUTH’ssmileisspeciﬁedwithanattribute‘smile’,andmaybeusedasanunderliner‘S’orbaton‘S*’.

RUTHallowsgeneralheadmovementsasfacialcon-versationalsignals.Theheadcannodupanddown,rotatehorizontallyleftandrightandtiltattheneckfromsidetoside;itcanalsobetranslatedfront-to-backandside-to-sidethroughmotionattheneck.Likebrowmovements,theseactionsmaygettheirmeaningsin-dividuallyorincombination;theymaysynchronizewithindividualwords,givingEkman’sbatonsorHadaretal.’srapidmovements,36ortheymaysynchronizewithlargerphrases,givingEkman’sunderlinersorHadaretal.’sordinarymovements.Headmovementsarespeci-ﬁedusingvaluesoftheattribute‘jog’.

Nostandardsymboliccodingofheadmovementsexists.Wehavedevelopedourown,drawingonourpreliminaryanalysisofvideotapedembodiedutter-ancesandinformalobservationsofeverydayconversa-tion.ThelabelsforheadmovementsthatwecurrentlysupportaregiveninFigure8,togetherwithsomeroughspeculationsaboutthefunctionsthatthesedifferentmovementsmightcarry.Weemphasizethatthisinven-toryisprovisional;categorizingthemovementsthataccompanyconversationalspeechandaccountingfortheirfunctionremainsanimportantproblemforfutureresearch.AtleasttwofurtherstepsarerequiredtovalidateasystemlikethatsuggestedinFigure8.Em-piricalresearchmustshowthatthecategoriesﬁtobservedconversationacrossarangeofindividuals

******************************************************************************************************

D.DECARLOETAL.

******************************************************************************************************

ValueDUFBRLJCDRURDLULTLTR

Effectandpossibleuse

nodsdownward

generalindicatorofemphasisnodsupward

perhapsindicatesa‘widerperspective’bringsthewholeheadforward

perhapsindicatesneedfor‘acloserlook’bringsthewholeheadbackward

perhapsemblemofbeing‘takenaback’turnstomodel’sright

perhapsindicatesavailabilityofmoreinformationturnstomodel’sleft

perhapsindicatesavailabilityofmoreinformationtiltswholeheadclockwise(aroundnose)

perhapsindicatesexpectationofengagementfrompartnertiltswholeheadcounterclockwise

perhapsindicatesexpectationofengagementfrompartnernodsdownwardwithsomerightwardmovementmeaningseemstocombinethatofDandRnodsupwardwithsomerightwardmovementmeaningseemstocombinethatofUandRnodsdownwardwithsomeleftwardmovementmeaningseemstocombinethatofDandLnodsupwardwithsomeleftwardmovementmeaningseemstocombinethatofUandLtiltsclockwisewithdownwardnoddingperhapsindicatescontrastofrelatedtopicstiltscounterclockwisewithdownwardnoddingperhapsindicatescontrastofrelatedtopics

Figure8.Possibleheadmovement(jog)codesin

RUTH.

acrossarangeofcontexts.Andempiricalresearchmustconﬁrmthatinterlocutorsalsoaresensitivetothedif-ferencesamongcategories.Sucheffortisproceeding;seeKrahmerandcolleagues,37,38forexample.

Finally,RUTHwillsynchronizeablinkjustattheendofanaccentedvowelwhenthewordcarriesthesimpleattribute‘(blink)’.

UsingRUTHinApplications

AssimpleillustrationsoftheuseofRUTH,wehaveimplementedtwoapplications:aversionofWeizen-baum’sfamousElizaprogram39whichoutputsspeciﬁ-cationsforanimatedspeech;andademonstrationofconversationalfeedbackthatanimatesRUTHperforminganindeﬁnitesequenceofrandomizedacknowledgmentbehaviors:nods,browraises,andnoiseslike‘mm-hmm’and‘uh-huh’.Bothprogramsareavailableaspartofthe

standardRUTHrelease;seealsoStoneandDeCarlo.40Theprogramsshareaconvenientoverallarchitecturethatasystem-buildercanusetoaddanimatedoutputtoanexistingapplication—pipingtheoutputofanordin-aryinteractivesystemasinputtoaRUTHprocessrun-ninginparallel.(TheElizaprogramalsoprintsouteachcommandbeforesendingittoRUTHsoyoucanseeexactlywhattheinputistotheanimation.)

OurElizaillustratessomeconvenientheuristicsforannotatingplaintexttosendittoRUTH.LikeallElizasystems,themeatoftheprogramisaseriesofcondition–responserulesthatdescribepossiblere-sponsesthatthesystemcouldgive.(OuranimatedversionofElizaextendsatextimplementationrealizedasaPerlscriptbyJonFernquistbutmodeledonaLispversionofElizadescribedbyNorvig.41)Theconditionlooksforaspeciﬁedsequenceofwordsintheuser’sutterance,andrecordsallthewordsfollowingthe

******************************************************************************************************

SPECIFYINGANDANIMATINGFACIALSIGNALS

******************************************************************************************************

matchedsequence.Theresponseisatexttemplateforananimatedutteranceandcanincludeapositionwheretherecordedwordsfromtheuser’sutterancecanbecopiedandpresentedbacktotheuser,perhapswithnewintonationorfacialdisplays.Tomark-uptheuser’sutterancesforprosody,templatescaninvokeproce-duresthatrealizeitwithoutaccents,realizeitjustwithasingleaccentontheﬁnalcontentword,orrealizeitwithaccentsonallcontentwords.

Ourfeedbackdemonstrationillustrateslow-levelin-teractionwithRUTH;itcreatesinstructionsforanima-tionson-the-ﬂy.Tocreatethefeedbackapplication,werecordedanddigitizedanumberofsamplesofac-knowledgmentsounds,andloggedwhenthesoundstarted,whenthesoundreacheditspeakintensity,andwhenthesoundﬁnished.Wealsotooknotewhetherthesoundshouldbeanimatedwiththemouthclosed(like‘mm-hmm’)orwiththemouthopen(like‘uhhuh’),andwhetherthesoundofferspositivefeed-back,expressingunderstanding,ornegativefeedback,expressingconfusion.Everyfewseconds,thefeedbackprogramwakesupandinstructsRUTHtoplayoneofthesoundﬁlesandanewanimationtimingﬁlethatgoeswithit,includingarandomizedselectionofactions—blinking,therightmouthshapestogowithwhateversoundﬁleisbeingplayed,perhapsaheadjog,andperhapsabrowaction.

Discussion

Conversationbringsmotionsandrequirementsbeyondthethelip-synchandemotionalexpressionemphasizedinsuchpriormodelsasCohenandMassaro’s17andKing’s.18Butmoregeneralmodels,deﬁnedintermsofmusculature42,43orsimulation,44introducecomplica-tionsthatcanstandinthewayofreal-timeperformanceandeasycustomization.Wehaveconstructedanewalternative,RUTH,byorganizingthedesignandimple-mentationofafaceanimationsystemaroundthein-vestigationofconversationalsignals.

Inparticular,RUTHisdesignedwithcodinginmind;RUTHacceptstextwithopen-endedannotationsspecify-ingheadmotionsandotherfacialactions,andpermitstheﬂexiblerealizationoftheseschedules.Manyappli-cationsdemandcoding.Inautonomousconversationalagents,forexample,arichintermediatelanguagebe-tweentheutterancegenerationsystemandtheanima-tionsystemhelpsorganizedecisionsaboutwhatmeaningtoconveyandhowtorealizemeaninginanimation.(SeetheworkofCassellandcolleagues45ongeneratingmeaningfulhandgesturesandcoordinat-ingthemwithothercommunicativeactions46.)RUTHstilllacksmanymeaningfulexpressions,includingemblemsofemotionsuchasdisgustandemblemsofthoughtsuchaspursingthelips.However,thefacialsignalsofprioragents26,47,48arejusteyebrowmovementsandareplannedindependentlyofothercommunicativedeci-sions;soRUTHalreadymakesiteasiertotakethenextsteps.

Likewise,indevelopingandtestingpsycholinguistictheoriesofconversation,predictable,rulegovernedrea-lizationofabstractdescriptionsmakescomputeranima-tionanimportantmethodologicaltool.45,47,49Coding-basedanimationsystemsallowanalyststovisualizedescriptionsofobservedevents,sothatanalystscanobtainamorespeciﬁcfeelforalternativemodels.Coding-basedsystemscanalsogeneralizeawayfromobservationsarbitrarily,sothatanalystscan,forexam-ple,exploreanomalousbehaviorswhichmightbeverydifﬁcultorimpossibletogetfrompeople(orstatisticalmodelsﬁttopeople).Thesameﬂexibilityandcontrolmakecoding-basedanimationanaturalingredientofempiricalstudiesofperception;Massaroandcollea-gues’explorationsofhumanspeechperceptionthatusemismatchedsoundandanimationaretheclassicexample.49Krahmerandcolleaguesareconductingpsycholinguisticstudiesofconversationalbrowmove-mentsusingcoding-basedanimation.37InformulatingRUTH’sinputasthisabstract,mean-ingfullayer,wedonotdiscounttheimportanceofquantitativevariablesinconversationalagents.Wesim-plyassumethatrangeofmovementandotherquanti-tativeaspectsofmotiondonotcontributetothesymbolicinterpretationofdiscourse.Rather,theypro-videquantitativeevidenceforspeakervariablessuchasinvolvementandaffect.Thisisalreadythenormforintonation,whereLaddetal.34presentsevidence(andCahn50providesanimplementation)linkingperceivedemotiontopitchrangeandvoicequalityofspeech;andformanualgesture,whereChiandcolleagues51modeltheemotionalvariablesthatquantitativelymodulatesymbolicaction.Badlerandcolleagues52,53areexploringasimilarapproachtomodulatefacialanimation.Integratingsuchmodality-independentspeciﬁcationsofaffectandpersonalitywithconversationalsignalsfordiscourseremainsimportantfutureworkforfacialanimation.Tothisend,weareextendingRUTHsothatplannedmotionscanundergoprobabilistictransformations,asinPerlin’swork,22,23soastoachievegreatervariabilitywithinRUTH’scoding-basedframework.

******************************************************************************************************

D.DECARLOETAL.

******************************************************************************************************

Withthesurgeofinterestininterfacesthatengageinnaturalembodiedconversation,asseeninrecentsur-veysofembodiedconversationalagents,54weexpectthatRUTHwillprovideahelpfulresourceforthescien-tiﬁccommunity.Inparticular,mostembodiedconver-sationalagentscreateabstractschedulesforanimationthatneedtoberealized;RUTHnaturallyﬁtsintosuchanarchitectureandenhancesitsfunctionality.Noristhereanyobstacle,atleastinprinciple,tointegratingtheinsightsofRUTH’sdesignandarchitectureintootherframeworksandanimationsystems.

ACKNOWLEDGEMENTS

ThisresearchwassupportedinpartbyNSFresearchinstru-mentationgrant9818322andbyRutgersISATC.DanDeCarlodrewtheoriginalRUTHconcept.RaduGruianandNikiShahhelpedwithprogramming;NathanFolsom-KovarikandChrisDymek,withdata.ThankstoScottKingfordiscussion.

References

1.CassellJ,BickmoreT,BillinghurstM,CampbellL,Chang

´lmssonH,YanH.EmbodimentinconversationalK,Vilhja

characters:Rea.InProceedingsofCHI,1999;pp520–527.

´rissonK.Thepowerofanodandaglance:2.CassellJ,Tho

envelopevs.emotionalfeedbackinanimatedconversa-tionalagents.AppliedArtiﬁcialIntelligence1999;13(3).

3.NagaoK,TakeuchiA.Speechdialoguewithfacialdis-plays:multimodalhuman–computerconversation.InPro-ceedingsofACL1994;pp102–109.

4.PoggiI,PelachaudC.Performativefacialexpressionsinanimatedfaces.InEmbodiedConversationalAgents,CassellJ,SullivanJ,PrevostS,ChurchillE(eds).MIT:Cambridge,MA,2000;155–188.

5.PelachaudC,PoggiI.Subtletiesoffacialexpressionsinembodiedagents.JournalofVisualizationandComputerAni-mation2002;13(5):301–312.

´E,RistT,vanMulkenS,KlesenM,BaldesS.The6.Andre

automateddesignofbelievabledialoguesforanimatedpresentationteams.InEmbodiedConversationalAgents,CassellJ,SullivanJ,PrevostS,ChurchillE(eds).MIT:Cambridge,MA,2000;220–255.

7.BallG,BreeseJ.Emotionandpersonalityinaconversa-tionalagent.InEmbodiedConversationalAgents,CassellJ,SullivanJ,PrevostS,ChurchillE(eds).MIT:Cambridge,MA,2000;189–219.

8.DeCarolisB,PelachaudC,PoggiI,deRosisF.Behaviorplanningforareﬂexiveagent.InProceedingsofIJCAI,2001.9.LesterJC,TownsSG,CallawayCB,VoermanJL,FitzGeraldPJ.Deicticandemotivecommunicationinanimatedpedagogicalagents.InEmbodiedConversationalAgents,CassellJ,SullivanJ,PrevostS,ChurchillE(eds).MIT:Cambridge,MA,2000;123–154.

10.MarsellaSC.Sympathyfortheagent:controllingan

agent’snonverbalrepertoire.InProceedingsofAgents,2000.11.BeckmanM,ElamGA.GuidelinesforToBIlabelling,

version3.0.Technicalreport,OhioStateUniversity,1997.http://ling.ohio-state.edu/Phonetics/etobi_homepage.html.

12.SilvermanKEA,BeckmanM,PitrelliJF,OstendorfM,

WightmanC,PriceP,PierrehumbertJ.ToBI:astandardforlabelingEnglishprosody.InProceedingsoftheInterna-tionalConferenceonSpokenLanguageProcessing,1992;pp867–870.

13.EkmanP.Aboutbrows:emotionalandconversationalsig-nals.InHumanEthology:ClaimsandLimitsofaNewDisci-pline:ContributionstotheColloquium,vonCranachM,FoppaK,LepeniesW,PloogD(eds).CambridgeUniver-sityPress:Cambridge,UK,1979;169–202.

14.McNeillD.HandandMind:WhatGesturesRevealabout

Thought.UniversityofChicagoPress:Chicago,1992.

15.BullP,ConnellyG.Bodymovementandemphasisin

speech.JournalofNonverbalBehavior1985;9(3):169–187.16.EngleRA.Towardatheoryofmultimodalcommunication:

combiningspeech,gestures,diagramsanddemonstrationsininstructionalexplanations.PhDthesis,StanfordUniver-sity,2000.

17.CohenMM,MassaroDW.Modelingcoarticulationinsyn-theticvisualspeech.InModelsandTechniquesinComputerAnimation,ThalmannNM,ThalmannD(eds).Springer:Berlin,1993;139–156.

18.KingSA.Afacialmodelandanimationtechniquesforani-matedspeech.PhDthesis,OhioStateUniversity,2001.19.Lo¨fqvistA.Speechasaudiblegestures.InSpeechProduction

andSpeechModeling,HardcastleWJ,MarchalA(eds).Kluwer:Dordrecht,1990;289–322.

20.BlackA,TaylorP.Festivalspeechsynthesissystem.Tech-nicalReportHCRC/TR-83,HumanCommunicationResearchCenter,1997.

21.BavelasJB,ChovilN.Visibleactsofmeaning:an

integratedmessagemodeloflanguageinface-to-facedia-logue.JournalofLanguageandSocialPsychology2000;19(2):163–194.

22.PerlinK.Layeredcompositioningoffacialexpression.In

SIGGRAPH,1997.TechnicalSketch.

23.PerlinK.Noise,hypertexture,antialiasingandgestures.In

TexturingandModeling:AProceduralApproach(2ndedn),EbertD(ed.).AcademicPress:London,1998;209–274.24.BrandM.Voicepuppetry.InSIGGRAPH,1999;pp21–28.25.SmidK,PandzicIS.Conversationalvirtualcharacterfor

theweb.InComputerAnimation,2002;240–247.

´lmssonH,BickmoreT.BEAT:thebeha-26.CassellJ,Vilhja

vioralexpressionanimationtoolkit.InSIGGRAPH,2001;pp477–486.

27.NassC,IsbisterK,LeeE-J.Truthisbeauty:resarching

embodiedconversationalagents.InEmbodiedConversa-tionalAgents,CassellJ,SullivanJ,PrevostS,ChurchillE(eds).MIT:Cambridge,MA,2000;374–402.28.JilkaM,Mo¨hlerG,DogilG.Rulesforthegenerationof

ToBI-basedAmericanEnglishintonation.SpeechCommuni-cation1999;28:83–108.29.Mo¨hlerG,MayerJ.Adiscoursemodelforpitch-range

control.InProceedingsofthe4thISCATutorialandResearchWorkshoponSpeechSynthesis,2001.

******************************************************************************************************

SPECIFYINGANDANIMATINGFACIALSIGNALS

******************************************************************************************************

30.MaconM,CronkA,KainA,WoutersJ.OGIresLPC:

diphonesynthesiserusingresiduallinearpredictioncoding.TechnicalReportCSE-97-007,OregonGraduateInstitute,1997.

31.PierrehumbertJ,HirschbergJ.Themeaningofintonational

contoursintheinterpretationofdiscourse.InIntentionsinCommunication,CohenP,MorganJ,PollackM(eds).MITPress:Cambridge,MA,1990;271–311.

32.HirschbergJ.Pitchaccentincontext:predictingintona-tionalprominencefromtext.ArtiﬁcialIntelligence1993;63(1–2):305–340.

33.HirschbergJ,NakataniC.Aprosodicanalysisofdiscourse

segmentsindirection-givingmonologues.InProceedingsofACL,1996.

34.LaddDR,SilvermanK,TolkmittF,BergmannG,Scherer

K.Evidencefortheindependentfunctionofintonationcontourtype,voicequalityandF0rangeinsignalingspeakeraffect.JournaloftheAcousticalSocietyofAmerica1985;78:435–444.35.Mo¨hlerG,MayerJ.Amethodfortheanalysisofprosodic

registers.InProceedingsofthe6thEuropeanConferenceonSpeechCommunicationandTechnology(EUROSPEECH),1999.

36.HadarU,SteinerTJ,GrantEC,RoseFC.Headmovement

correlatesofjunctureandstressatsentencelevel.LanguageandSpeech1983;26(2):117–129.

37.KrahmerE,RuttkayZ,SwertsM,WesselinkW.Pitch,eye-browsandtheperceptionoffocus.InSymposiumonSpeechProsody,2002.

38.KrahmerE,RuttkayZ,SwertsM,WesselinkW.Audiovi-sualcuestoprominence.InInternationalConferenceonSpo-kenLanugageProcessing,2002.

39.WeizenbaumJ.ELIZA:acomputerprogramforthestudy

ofnaturallanguagecommunicationbetweenmanandmachine.CommuncationsoftheACM1966;9(1):36–45.40.StoneM,DeCarloD.Craftingtheillusionofmeaning:

template-basedspeciﬁcationofembodiedconver-sationalbehavior.InComputerAnimationandSocialAgents,2003.

41.NorvigP.ParadigmsofArtiﬁcialIntelligence:CaseStudiesin

CommonLisp.MorganKaufmann:SanFrancisco,1992.42.PlattSM.Astructuralmodelofthehumanface.PhD

thesis,UniversityofPennsylvania,1985.

43.WatersK.Amusclemodelforanimatingthree-dimensionalfacialexpressions.ComputerGraphics1987;21(4):17–24.

44.TerzopoulosD,WatersK.Physically-basedfacialmodel-ing,analysisandanimation.JournalofVisualizationandComputerAnimation1990;1(2):73–80.

45.CassellJ,StoneM,DouvilleB,PrevostS,AchornB,

SteedmanM,BadlerN,PelachaudC.Modelingtheinter-actionbetweenspeechandgesture.InProceedingsoftheCognitiveScienceSociety,1994.

46.CassellJ,StoneM,YanH.Coordinationandcontext-dependenceinthegenerationofembodiedconversation.InFirstInternationalConferenceonNaturalLanguageGenera-tion,2000;pp171–178.

47.PelachaudC,BadlerN,SteedmanM.Generatingfacial

expressionsforspeech.CognitiveScience1996;20(1):1–46.

48.PoggiI,PelachaudC.Eyecommunicationinaconversa-tional3Dsyntheticagent.AICommunications2000;13(3):169–181.

49.MassaroDW.PerceivingTalkingFaces:FromSpeech

PerceptiontoaBehavioralPrinciple.MIT:Cambridge,MA,1998.

50.CahnJE.Thegenerationofaffectinsynthesizedspeech.

JournaloftheAmericanVoiceI/OSociety1990;8:1–19.

51.ChiD,CostaM,ZhaoL,BadlerN.TheEMOTEmodelfor

effortandshape.InSIGGRAPH,2000;pp173–182.

52.BadlerN,AllbeckJ,ZhaoL,ByunM.Representingand

parameterizingagentbehaviors.InComputerAnimation,2002;pp133–143.

53.ByunM,BadlerN.FacEMOTE:qualitativeparametric

modiﬁersforfacialanimations.InACMSIGGRAPHSym-posiumonComputerAnimation,2002;pp65–71.

54.CassellJ,SullivanJ,PrevostS,ChurchillE(eds).Embodied

ConversationalAgents.MIT:Cambridge,MA,2000.

Authors’biographies:

DougDeCarloisanAssistantProfessorofComputerScienceatRutgers,andreceivedhisPh.D.fromtheUniversityofPennsylvaniain1998.Heco-directsTheVILLAGEandholdsajointappointmentintheRutgersCenterforCognitiveScience.Hisresearchfocusesonthecognitivescienceofvisualinteraction,whichbridgestheﬁeldsofgraphics,visionandHCI.Headvocatesinteractivesystemsthatleverageusers’existingabilitiesofvisualperceptionandcommunication.

CoreyRevillaispursuingaMastersdegreeattheEntertainmentTechnologyCenteratCarnegieMellonUniversity.HehasaB.S.inComputerScienceandMathematicsfromRutgers,andworkedasaresearchassistantonthedevelopmentofRUTH.

******************************************************************************************************

D.DECARLOETAL.

******************************************************************************************************

MatthewStoneisanAssistantProfessorofComputerScienceatRutgers,withajointappointmentintheCenterforCognitiveScience.HereceivedhisPh.D.fromtheUniversityofPennsylvaniain1998,workingonknowledgerepresentationandreasoningforconver-sationalagents.Hisresearchinterestsincludecomputa-tionalapproachestoface-to-facedialogue,naturallanguagegenerationincomputationallinguistics,andtheoriesofthemeaningandcontext-dependenceoflanguage.Heco-directstheVILLAGElabatRutgers.

JenniferJ.VendittiisaPostdoctoralResearchScientistinComputerScienceatColumbiaUniversity.Shere-ceivedherPh.D.fromOhioStateUniversityLinguisticsin2000,specializinginphoneticsandintonation.Herresearchinterestsincludeintonationaltheoryandmod-eling,spokenlanguageprocessingindiscourse,andspeechsynthesis.

******************************************************************************************************

因篇幅问题不能全部显示，请点此查看更多更全内容

查看全文