Show simple item record

dc.contributor.authorSadoughi, Najmeh
dc.contributor.authorBusso, Carlos A.
dc.date.accessioned2019-08-05T19:50:27Z
dc.date.available2019-08-05T19:50:27Z
dc.date.created2018-05-15
dc.identifier.issn9781538623350 (ISBN)
dc.identifier.urihttps://hdl.handle.net/10735.1/6771
dc.descriptionFull text access from Treasures at UT Dallas is restricted to current UTD affiliates (use the provided Link to Article).
dc.description.abstractThe orofacial area conveys a range of information, including speech articulation and emotions. These two factors add constraints to the facial movements, creating non-trivial integrations and interplays. To generate more expressive and naturalistic movements for conversational agents (CAs) the relationship between these factors should be carefully modeled. Data-driven models are more appropriate for this task than rule-based systems. This paper provides two deep learning speech-driven structures to integrate speech articulation and emotional cues. The proposed approaches rely on multitask learning (MTL) strategies, where related secondary tasks are jointly solved when synthesizing orofacial movements. In particular, we evaluate emotion recognition and viseme recognition as secondary tasks. The approach creates shared representations that generate behaviors that not only are closer to the original orofacial movements, but also are perceived more natural than the results from single task learning.
dc.description.sponsorshipUS National Science Foundation grant IIS-1718944.
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.relation.isPartOfProceedings - International Conference on Automatic Face and Gesture Recognition, 13th
dc.relation.urihttp://dx.doi.org/10.1109/FG.2018.00066
dc.rights©2018 IEEE
dc.subjectLearning
dc.subjectGesture
dc.subjectSpeech
dc.subjectConversation
dc.subjectEmotion recognition
dc.titleExpressive Speech-Driven Lip Movements with Multitask Learning
dc.type.genrearticle
dc.description.departmentErik Jonsson School of Engineering and Computer Science
dc.identifier.bibliographicCitationSadoughi, N., and C. Busso. 2018. "Expressive speech-driven lip movements with multitask learning." Proceedings - International Conference on Automatic Face and Gesture Recognition, 13th: 409-415, doi:10.1109/FG.2018.00066
dc.contributor.utdAuthorSadoughi, Najmeh
dc.contributor.utdAuthorBusso, Carlos A.


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record