Towards Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues
Third COST 2102 International Training School, Caserta, Italy, March 15-19, 2010, Revised Selected Papers
Published on 19. January 2011
XXII, 474 pages
978-3-642-18184-9 (ISBN)
Description
This volume brings together the advanced research results obtained by the European COST Action 2102: "Cross Modal Analysis of Verbal and Nonverbal Communication". The research published in this book was discussed at the 3rd jointly EUCOGII-COST 2102 International Training School entitled "Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues ", held in Caserta, Italy, on March 15-19, 2010. The book is arranged into two scientific sections. The 18 revised papers of the first section, "Human-Computer Interaction: Cognitive and Computational Issues", deal with conjectural and processing issues of defining models, algorithms, and strategies for implementing cognitive behavioural systems. The second section, "Synchrony through Verbal and Nonverbal Signals", presents 21 revised lectures that provide theoretical and practical solutions to the modelling of timing synchronization between linguistic and paralinguistic expressions, actions, body movements, activities in human interaction and on their assistance for an effective communication.
More details
Other editions
Additional editions
Anna Esposito | Antonietta M. Esposito | Raffaele Martone
Towards Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues
Third COST 2102 International Training School, Caserta, Italy, March 15-19, 2010, Revised Selected Papers
Book
01/2011
Springer
€53.49
Shipment within 5-7 days
Content
- Title
- Preface
- Organization
- Table of Contents
- I - Human-Computer Interaction: Cognitive and Computational Issues
- Interaction and Resistance: The Recognition of Intentions in New Human-Computer Interaction
- GOFAI and GOHCI
- GOHCI "Good Old HCI"
- GOFAI "Good Old Fashioned AI"
- Computing
- Three Levels of Description
- Intelligence and Goals
- Thick Description
- HCI Is Part of Cognitive Systems Research
- Interaction Requires Resistance
- Resistance for New HCI
- References
- Speaking without Thinking: Embodiment, Speech Technology and Social Signal Processing
- Embodiment
- Multimodal Schemas - Developmental Evidence
- Multimodal 'Image' Schemas
- Image Schemas in the Brain
- Image Schemas and Language Comprehension
- Embodied Image Schemas in Speech Technology and Social Signal Processing
- References
- More Than Words: Inference of Socially Relevant Information from Nonverbal Vocal Cues in Speech
- Introduction
- Prosody: Spotting Journalists in Broadcast Data
- Turn-Taking: Automatic Role Recognition
- Social Network Extraction
- Role Recognition
- Experiments and Results
- Conclusions
- References
- A Timely Endeavor: Theoretical, Behavioral, Bioimaging, and Clinical Perspectives on Time Perception
- Introduction
- Main Issues on Time and TP
- Timely Endeavors
- References
- Instruction and Belief Effects on Sentential Reasoning
- Introduction
- The Experiment
- Participants
- Design
- Materials and Procedure
- Results
- Discussion
- References
- Audio-Visual Prosody: Perception, Detection, and Synthesis of Prominence
- Introduction
- Audio Visual Prominence
- Acoustic Prominence
- Visual Prominence
- Acoustic Detection of Prominence
- Prominence Model
- Feature Set
- Experiments and Results
- Visual Prominence and Intelligibility
- Method and Setup
- Marking Prominence
- Conditions
- Analysis and Results
- Naturalness and Gaze Behavior
- Stimuli and Setup
- Gesture Generation
- Gaze Analysis
- Questionnaires Analysis
- Discussions and Remarks
- Conclusion
- References
- Adaptation in Turn-Initiations
- Introduction
- Corpus Description
- Descriptive and Quantitative Observations
- Local Adaptation: Affirmative Cue Words in Adjacent Turns
- Global Adaptation: Timing of Turn-Initial Filled Pauses
- Discussion and Conclusion
- References
- Sentic Avatar: Multimodal Affective Conversational Agent with Common Sense
- Introduction
- Multimodal Affective HCI
- Overview of the System
- Affective Analysis Module
- Sentic Computing
- Facial Expression Analyzer
- Affective Integrator
- Conclusion and Future Work
- References
- Patterns of Synchronization of Non-verbal Cues and Speech in ECAs: Towards a More "Natural" Conversational Agent
- Introduction
- Embodied Conversational Agents: State of the Art
- Common Problems in ECAs
- Analysis of GRETA's Online Performance
- Proposal for a More Natural ECA
- Conclusions
- References
- Top-Down Influences on Eye-Movements during Painting Perception: The Effect of Task and Titles
- Introduction
- Influence of the Titles on Aesthetic Experience
- Eye-Movements and Aesthetic Experience
- Goals and Hypothesis of the Present Study
- Method
- Design of the Study
- Stimuli and Procedure
- Eye-Movements Recordings
- Results for the Aesthetic Ratings
- Results for the Eye-Movements Patterns
- Dali - 'Swans Reflecting Elephants'
- Dali - 'Metamorphosis of Narcissus'
- Caravaggio - 'Doubting Thomas'
- Caravaggio - 'Sacrifice of Isaac'
- Summary
- General Discussion and Conclusions
- References
- Visual Sociology in the Classroom: Fostering Interaction Awareness Using Video
- Introduction
- Methodological Issues
- Action Research
- Stimulated Recall
- Video as a Resource
- Conversation Analysis and Institutional Talk
- Method
- Results
- Students' and Teacher's Analysis
- Researchers' Conversation Analysis
- Students' and Teacher's Evaluation of the Action Research
- Conclusion
- References
- Analysis of Interrogatives in Different Domains
- Introduction
- Prosodic Analysis of Interrogatives in EP
- Corpora
- Statistical Analysis of Interrogatives
- Overall Frequency of Interrogative Types in the Training Corpora
- Building a Golden Set
- Punctuation Experiments
- Baseline Experiments
- Experiments with Lexical and Acoustic Features
- Experiments with Prosodic Features
- Conclusions and Future Work
- References
- Interviewers' Use of Coercive Questioning during a Midterm Period Favorable to the Opposition Party
- Introduction
- The Political Context
- Italian Media in 2004-2005
- Hypotheses
- Method
- Sample
- Si.Co.D.: A New Tool to Code Coercive Questions and Its Observer Reliability
- Procedure and Data Analyses
- Results
- Discussion and Conclusions
- References
- Emotions and Speech Disorders: Do Developmental Stutters Recognize Emotional Vocal Expressions?
- Introduction
- Scientific Background
- Materials and Methods
- Participants
- Investigation Tools
- Procedures
- Data Analysis
- Discussion
- Conclusions
- References
- Representing Meaning in Mind: When Predicate Argument Structures Meet Mental Representations
- Introduction
- Non-linguistic Information and the Construction of Meaning in Sentences
- Perception and Language Processes
- Theoretical Background and Experimental Design
- Perception and Comprehension Processes
- When Linguistic and Non-linguistic Information Meet to Build Up Meaning
- Method
- Results and Discussion
- References
- Beyond the Panopticon Framework: Privacy, Control and User Generated Content
- Introduction
- Literature Review
- Method
- Results and Discussion
- Conclusions
- References
- Micro and Macro Spatial Dimensions of New Media in Five European Countries
- Introduction
- Literature Review
- Method
- Results
- The First Spatial Shell: House Size
- The First Spatial Shell: House Size
- The Second Spatial Shell: City Size
- The Third and Fourth Spatial Shells: European Macro Areas and Countries
- Analysis of the Effects of the Four Spatial Shells Considered Altogether
- Final Discussion and Conclusion
- References
- Nonverbal Expressions of Turn Management in TV Interviews: A Cross-Cultural Study between Greek and Icelandic
- Introduction
- Data Collection
- Annotation Process
- Turn Taking in Political Interviews
- Data Analysis
- Non-verbal Turn Management: Frequency
- Non-verbal Turn Management: Preferred Modalities
- Discussion
- References
- II - Synchrony through Verbal and Nonverbal Signals
- Interpretation and Generation of Dialogue with Multidimensional Context Models
- Introduction
- Context and Dialogue Interpretation in DIT
- Dialogue Semantics
- Multifunctionality and Multidimensionality
- Dimensions
- Communicative Functions
- Content and Structure of a Dialogue Context Model
- A Catalogue of Context Information
- Representation Structures
- Applications
- Semantic and Pragmatic Dialogue Analysis
- Annotation
- Dialogue System Design
- Functional Markup of Embodied Conversational Agents
- Perspectives and Future Work
- References
- Coordination, Not Control, Is Central to Movement
- Introduction
- Coordination and Coordinative Structures
- Synchronous Speech
- Embodied Task Dynamics
- Dynamics and Autonomy
- References
- Speech, Gaze and Head Motion in a Face-to-Face Collaborative Task
- Introduction
- Experiment
- Procedure
- Technical Setup
- Results
- Analyses
- Completion Time
- Search Time and Location Time
- Number of Cubes
- Head Motion
- Conclusions and Discussion
- Future Work
- References
- Neuroelectric Methodologies for the Study of the Economic Decisions in Humans
- Introduction
- Material and Methods
- Experiment 1
- Experiment 2
- Results
- Experiment 1
- Experiment 2
- Discussion
- References
- An Evaluation Study on Speech Feature Densities for Bayesian Estimation in Robust ASR
- Introduction
- Mathematical Background
- The Bayes Estimation Framework
- Statistical Analysis of Speech Feature Densities
- Non-linear Least Squares
- Estimation Results
- Enhancement Algorithms
- Short-Time Spectral Amplitude Approaches
- Mel-frequency Domain Approaches
- Log-Mel and MFCC Domain Approaches
- ASR Results
- Conclusions
- References
- Naturalness, Adaptation and Cooperativeness in Spoken Dialogue Systems
- Introduction
- Naturalness of Interface Language
- Adaptive Dialogue Strategies
- Step 1: Modeling the State of Interaction
- Step 2: Defining Dialogue Strategy Components
- Step 3: Defining Dialogue Strategies
- An Example
- Conclusion
- References
- Towards Semantic Multimodal Video Annotation
- Introduction
- Multimodal Video Annotation
- Video Annotation Tools
- Annotation Schemas
- Semantic Web
- Human Emotion Ontology
- Faceted Browsing Paradigm
- Semantic Video Annotation Tools
- Video Annotation Survey
- Results Discussion
- A Roadmap towards Semantic Multimodal Annotation
- Conclusions
- References
- The Effect of Subharmonic Stimuli on Singing Voices
- Introduction
- The Relationship between Subharmonics and Voice Quality from the Point of View of Opera Singers and Voice Coaches
- Experiments and Initial Results
- Trial Setup and Signal Processing Analysis
- Example Results
- Discussion
- Summary and Conclusions
- References
- Speech Modeling Using the Complex Cepstrum
- Introduction
- Speech Vocoder Based on the Complex Cepstrum
- Segmentation
- Complex Spectrum Speech Analysis
- Vocal Tract Model
- Speech Synthesis
- Conclusion
- References
- Problems of the Automatic Emotion Recognitions in Spontaneous Speech
- An Example for the Recognition in a Dispatcher Center
- Introduction
- System Description
- Acoustical Pre-processing
- Speech Corpora
- Testing of the System
- Discussion
- Conclusions
- References
- Slovak Language Model from Internet Text Data
- Introduction
- Process of Text Gathering and Building SLMs
- Speech Recognition Setup
- Experimental Results
- Discussion
- Conclusions
- References
- Automatic Recognition of Emotional State in Polish Speech
- Introduction
- Database
- Automatic Recognition
- Feature Extraction
- Classification Algorithms
- Experimental Results
- Conclusions
- References
- Categorical Perception of Consonants and Vowels: Evidence from a Neurophonetic Model of Speech Production and Perception
- Introduction
- The Neurophonetic Model
- Method
- Results
- Discussion
- References
- The MultiLis Corpus - Dealing with Individual Differences in Nonverbal Listening Behavior
- Introduction
- Related Work
- Data Collection
- Setup
- Procedure
- Tasks
- Measures
- MultiLis Corpus
- Annotations
- Individual Differences in the MultiLis Corpus
- Conclusions and Future Work
- References
- Comparing the Rhythmical Characteristics of Speech and Music - Theoretical and Practical Issues
- Introduction
- Rhythmical Composition of Music and Speech
- Comparing Research Tasks
- Onset Detection as Example
- Database
- Onset Detection
- Evaluation
- Results
- Conclusions
- References
- The Ability of Children with Mild Learning Disabilities to Encode Emotions through Facial Expressions
- Introduction
- Purpose and Goals of the Study
- Methodology
- Subjects
- Results
- External Validation of the Assessment Methodology
- Differential Ability to Produce the Basic States
- Inter-relations between Abilities
- Exploring for Age - Gender and School Performance Issues
- Discussion
- Relations between Ability to En/Decode and Other Cognitive/Behavioral Skills
- Age and Gender Issues
- The Ability to Encode/Decode Emotions and Its Neurophysiological Substrates
- Conclusions and Future Studies
- References
- Designing a Hungarian Multimodal Database - Speech Recording and Annotation
- Introduction
- Material Design
- Personnel Requirements - Database Speakers
- Technical Requirements - Studio Equipment
- Database Contents
- Speech Annotation
- Labels of Intonational Phrase Types, Expressed Attitudes and Discourse Types
- Transcription Levels
- Second Passing
- Future Steps
- References
- Conveying Directional Gaze Cues to Support Remote Participation in Hybrid Meetings
- Introduction
- Transmission of Gaze Behavior in Mediated Communication
- Experimental Conditions and Hypotheses
- Standard 2D Videoconferencing Interface
- Integrated 3D Videoconferencing Interface
- Hypotheses
- User Study - Method
- Participants and Experimental Design
- Group Decision Tasks
- Procedure
- Measures
- Results
- Group Process and Satisfaction with the Outcome
- Social Presence
- Turn-Taking, Usability, Recognition of Non-verbal Cues and Gaze Direction
- Visual Focus of Attention
- Group Interviews
- Discussion
- Conclusions
- References
- Affect Recognition in Real Life Scenarios
- Introduction
- System Architecture
- The PROMETHEUS Database
- Experiments and Results
- Conclusion
- References
- Understanding Parent-Infant Behaviors Using Non-negative Matrix Factorization
- Introduction
- Home Movies Corpus and Annotation
- Corpus
- Annotation
- Multi-modal Interaction
- Parent-infant Interaction Analysis
- Statistical Interaction Characterization
- Clustering
- Experimental Results
- Conclusions
- References
- Learning and Knowledge-Based Sentiment Analysis in Movie Review Key Excerpts
- Introduction
- Metacritic - A Movie Review Database
- On-line Knowledge Sources
- Valence Estimation Using On-line Knowledge Sources
- Fallback Mechanisms
- Filtering and Classification
- Evaluation
- Parameter Tuning
- Out of Vocabulary Resolution
- Bag of N-Grams vs. On-line Knowledge Sources
- Error Analysis
- Estimating Mixed Sentiment
- Information Fusion
- Review Score Prediction
- Conclusion and Outlook
- References
- Author Index
