Multilingual and Multimodal Information Access Evaluation
Second International Conference of the Cross-Language Evaluation Forum, CLEF 2011, Amsterdam, The Netherlands, September 19-22, 2011, Proceedings
Published on 25. September 2011
X, 143 pages
978-3-642-23708-9 (ISBN)
Description
This book constitutes the refereed proceedings of the Second International Conference on Multilingual and Multimodal Information Access Evaluation, in continuation of the popular CLEF campaigns and workshops that have run for the last decade, CLEF 2011, held in Amsterdem, The Netherlands, in September 2011.
The 14 revised full papers presented together with 2 keynote talks were carefully reviewed and selected from numerous submissions. The papers accepted for the conference included research on evaluation methods and settings, natural language processing within different domains and languages, multimedia and reflections on CLEF. Two keynote speakers highlighted important developments in the field of evaluation: the role of users in evaluation and a framework for the use of crowdsourcing experiments in the setting of retrieval evaluation.
More details
Other editions
Additional editions
Pamela Forner | Julio Gonzalo | Jaama Kekäläinen
Multilingual and Multimodal Information Access Evaluation
Second International Conference of the Cross-Language Evaluation Forum, CLEF 2011, Amsterdam, The Netherlands, September 19-22, 2011, Proceedings
Book
09/2011
1st Edition
Springer
€53.49
Shipment within 7-9 days
Content
- Title
- Preface
- Organization
- Table of Contents
- Keynote Addresses
- "Would You Trust Your IR System to Choose Your Date?" Re-thinking IR Evaluation in the 21st Century
- Overview
- Crowdsourcing for Information Retrieval Experimentation and Evaluation
- Methodologies and Lessons
- Building a Cross-Language Entity Linking Collection in Twenty-One Languages
- Introduction
- Document Collections
- Choosing Names
- Generating English Ground Truth
- Name Projection
- Collection Statistics
- Using the Test Collection
- Conclusion
- References
- Search Snippet Evaluation at Yandex: Lessons Learned and Future Directions
- Introduction
- Related Work
- Snippet Evaluation at Yandex
- Eye Tracking Experiment
- Manual Assessment
- Automated Quality Measures
- A/B Testing
- Conclusions and Future Research
- References
- Towards a Living Lab for Information Retrieval Research and Development
- Introduction
- From Standard Test Collections to Living Labs
- A System Architecture for Living Labs
- A Living Lab for Online Shopping
- Discussion and Future Plans
- References
- A Comparison of Evaluation Metrics for Document Filtering
- Introduction
- State of the Art
- Task Definition and Formal Constraints
- Measure Properties
- Utility-Based Metrics
- Informativeness-Based Measures
- Class-Oriented Measures: Precision and Recall
- Conclusions
- References
- Language and Processing
- Filter Keywords and Majority Class Strategies for Company Name Disambiguation in Twitter
- Introduction
- State of the Art
- WePS-3: Test Collection and Systems
- A Fingerprint Visualization to Compare Systems
- Hypotheses Testing
- Upper Bound Performance of Filter Keywords
- Upper Bound Performance of Winner-Takes-All Strategy
- Upper Bound Combination of Strategies
- Automatic Discovery of Filter Keywords
- Term Features
- Keyword Discovery
- Discussion and Conclusions
- References
- Automatic Annotation of Bibliographical References for Descriptive Language Materials
- Introduction
- Problem Statement
- The Present Dataset
- Related Work
- A DNF Approach
- Experiments
- Experiment Design
- Results
- Discussion
- Conclusion
- References
- A Language-Independent Approach to Identify the Named Entities in Under-Resourced Languages and Clustering Multilingual Documents
- Introduction
- Related Work
- Proposed Approach
- Phase-1: NE Identification
- Phase-2: Multilingual Document Clustering Based on Named Entities (MDCNE)
- Experimental Evaluation
- Discussion
- Conclusion and Future Work
- References
- Multilingual Question-Answering System in Biomedical Domain on the Web: An Evaluation
- Introduction
- Question Answering Systems: Beyond Information Retrieval
- Method Section
- Results Section
- Conclusions
- References
- Simulation of Within-Session Query Variations Using a Text Segmentation Approach
- Introduction
- Automatically Generating Query Reformulations
- Reformulation Effect on Retrieval
- Experiments
- Conclusions
- References
- Visual and Context
- Assessing the Scholarly Impact of ImageCLEF
- Introduction
- The ImageCLEF Evaluation Campaign
- Bibliometric Analysis Method
- The Dataset of ImageCLEF Publications
- Results for ImageCLEF Publications 2004-2010
- Conclusions
- References
- The Importance of Visual Context Clues in Multimedia Translation
- Introduction
- Background and Motivation
- The Meteor MT Evaluation System
- Experimental Design
- Results
- Conclusion
- References
- To Re-rank or to Re-query: Can Visual Analytics Solve This Dilemma?
- Introduction
- Related Work
- The Formal Model
- The Prototype
- Experimentation
- Conclusion and Future Work
- References
- Evaluation Methods for Rankings of Facetvalues for Faceted Search
- Introduction
- Motivation
- Two Evaluation Metrics
- Normalized Discounted Cumulated Gain
- Normalized Recursive Discounted Cumulated Gain
- Conclusions
- References
- Improving Query Expansion for Image Retrieval via Saliency and Picturability
- Introduction and Motivation
- Salient Semantic Analysis
- Flickr Picturability
- Automatic Query Expansion via Saliency and Picturability
- Candidate Generation
- Candidate Selection
- Empirical Evaluation
- Discussion
- Related Work
- References
- Author Index
