Comparative Evaluation of Focused Retrieval
9th International Workshop of the Inititative for the Evaluation of XML Retrieval, INEX 2010, Vught, The Netherlands, December 13-15, 2010, The Netherlands, Revised Selected Papers
Published on 30. August 2011
XII, 398 pages
978-3-642-23577-1 (ISBN)
Description
This book constitutes the thoroughly refereed post-workshop proceedings of the 9th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2009, held in Vught, The Netherlands, in December 2009. The aim of the INEX 2010 workshop was to bring together researchers in the field of XML IR who participated in the INEX 2010 campaign. During the past year, participating organizations contributed to the building of large-scale XML test collections by creating topics, performing retrieval runs and providing relevance assessments. The workshop concluded the results of this large-scale effort, summarized and addressed issues encountered, and devised a work plan for the future evaluation of XML retrieval systems. The 37 full papers presented were carefully reviewed and selected from 42 submissions; they are organized into sections corresponding to the nine tracks of the workshop, investigating various aspects of XML retrieval, from book search to entity.
More details
Other editions
Additional editions
Shlomo Geva | Jaap Kamps | Andrew Trotman
Comparative Evaluation of Focused Retrieval
9th International Workshop of the Inititative for the Evaluation of XML Retrieval, INEX 2010, Vught, The Netherlands, December 13-15, 2010, The Netherlands, Revised Selected Papers
Book
09/2011
1st Edition
Springer
€53.49
Shipment within 5-7 days
Content
- Title
- Preface
- Organization
- Table of Contents
- Ad Hoc Track
- Overview of the INEX 2010 Ad Hoc Track
- Introduction
- Ad Hoc Retrieval Track
- Tasks
- Submission Format
- Evaluation Measures
- Ad Hoc Test Collection
- Corpus
- Topics
- Judgments
- Questionnaires
- Ad Hoc Retrieval Results
- Participation
- Relevant in Context Task
- Restricted Relevant in Context Task
- Restricted Focused Task
- Efficiency Task
- Significance Tests
- Analysis of Reading Effort
- Relevant in Context
- Analysis of Article Retrieval
- Article Retrieval: Relevance Judgments
- Discussion and Conclusions
- References
- The Potential Benefit of Focused Retrieval in Relevant-in-Context Task
- Introduction
- Metrics and Results
- Cumulating the Localizing Effort
- Results
- Analysis Based on Three Document Retrieval Scenarios
- Discussion and Conclusions
- References
- ENSM-SE and UJM at INEX 2010: Scoring with Proximity and Tag Weights
- Introduction
- Influence Functions
- Structure, Elements and Logical Elements
- Influence Function of a Term to a Position
- Influence Function of a Query to a Position
- Score of an Element
- Weighting Tags and Modulating Influence Function Shapes
- Weighting Tags
- Modulating Influence Function Shapes
- Experiments
- Building Runs
- Results
- Conclusion
- References
- LIP6 at INEX'10: OWPC for Ad Hoc Track
- Introduction
- Learning Base
- Basic Features
- Context Features
- OWPC, Our Learning to Rank Model
- Learning Step
- Ranking Prediction
- Experiments - Results
- Efficiency/thorough Task
- Restricted Focused Task
- Conclusion
- References
- A Useful Method for Producing Competitive Ad Hoc Task Results
- Introduction
- Dynamic Element Retrieval and the Ad Hoc Tasks
- Producing the Element Set
- Producing Task-Specific Output
- Early Results
- Experiments and Results
- Thorough Task
- Focused Task
- Relevant-in-Context Task
- Analysis and Observations
- References
- Relaxed Global Term Weights for XML Element Search
- Introduction
- XML Element Search Techniques and Related Studies
- An Expansion from Document Search to Element Search
- Existing Methods of Calculating Global Weights
- Relaxing Restriction of Document Structure in a Query
- Classification of Similar Path Expressions
- Inverse Combination Frequency
- Inverse Aggregated-Path Frequency
- Inverse Set Frequency
- Experiments
- Experiments for Integrating Path Expressions
- Analyzing the Number of XML Elements in Classes
- Evaluation on Search Accuracies
- Conclusion
- References
- Searching the Wikipedia with Public Online Search Engines
- Introduction
- Related Work
- Running Online Search Engines
- Bing
- Yahoo!
- Wikisearch
- From Result Pages to Run Submissions
- Restricted Focused
- Restricted Relevant in Context
- Summary
- Results
- Conclusion
- References
- Extended Language Models for XML Element Retrieval
- Introduction
- Pre-processing and Indexing
- Query Formulation
- Retrieval Models and Strategies
- Full Document Retrieval
- Snippet Retrieval
- Results
- Runs
- Measured as Document Retrieval
- Measured as Snippet Retrieval
- Analysis
- Conclusion
- References
- Book Track
- Overview of the INEX 2010 Book Track: Scaling Up the Evaluation Using Crowdsourcing
- Introduction
- Participating Organizations
- Summary of Participants' Approaches
- The Book Corpus
- Search Tasks
- Best Books to Reference (BB) Task
- Prove It (PI) Task
- Test Topics for the Search Tasks
- INEX Topics
- Crowdsourced Topics
- Topic Selection
- Relevance Assessments
- Gathering Relevance Labels from INEX Participants
- Crowdsourcing Relevance Labels
- Collected Relevance Data
- Analysis of Crowdsourced Relevance Labels
- Official Qrels
- Evaluation Measures and Results for the Search Tasks
- Best Books Task Evaluation
- Prove It Task Evaluation
- The Structure Extraction (SE) Task
- The Active Reading Task (ART)
- Conclusions and Plans
- References
- LIA at INEX 2010 Book Track
- Introduction
- Word Hyphenation Correction
- Best Books for Reference
- Retrieval Model
- Baselines
- Contextual Query Expansion Using Wikipedia
- Using the Stanford Parser
- Applying the Book Track's Contextual Query Expansion to the Ad Hoc Track
- Results
- Conclusions
- References
- The Book Structure Extraction Competition with the Resurgence Software for Part and Chapter Detection at Caen University
- Introduction
- Our Book Structure Extraction Method
- Challenges
- Strategy
- Calibrating the System
- Experiment
- Results
- Discussion
- Reflections on the Experiment
- Reflections on Evaluation Measures
- Proposals
- Conclusion
- References
- Focus and Element Length for Book and Wikipedia Retrieval
- Introduction
- Ad Hoc Track
- Indexing
- Category Evidence
- Runs
- Thorough Evaluation
- Focused Task
- Relevant in Context Task
- Restricted Relevant in Context Task
- Book Track
- Prove It Run Analysis
- Best Book Results
- Prove It Results
- Conclusion
- Bibliography
- Combining Page Scores for XML Book Retrieval
- Introduction
- The Retrieval Algorithms and Fusion Operators
- TREC2 Logistic Regression Algorithm
- Blind Relevance Feedback
- Coordination Level Matching
- Result Combination Operators
- Database and Indexing Issues
- Indexing the Books XML Database
- INEX 2010 Book Track Runs
- Conclusions and Future Directions
- References
- OUC's Participation in the 2010 INEX Book Track
- Introduction
- Indexing and Retrieval Strategies
- Runs and Results
- Discussion
- Conclusion
- References
- Data Centric Track
- Overview of the INEX 2010 Data Centric Track
- Introduction
- The Task
- INEX Data Centric Track Collection
- Document Collection
- Topics
- Submission Format
- Submitted Runs
- Assessment and Evaluation
- Results
- Conclusions
- DCU and ISI@INEX 2010: Adhoc and Data-Centric Tracks
- Introduction
- Document Retrieval
- Preprocessing
- Language Model (LM) Term Weighting
- XML Element Retrieval
- Simplified Hierarchical Language Model
- Adhoc Track Experiments and Results
- Thorough Task
- Focused Task
- Data Centric Track Experiments and Results
- Using Negative Information Present in Verbose Queries
- Conclusions and Future Work
- References
- Automatically Generating Structured Queries in XML Keyword Search
- Introduction
- Related Work
- StruX
- Overview
- Input Transformation
- Candidate Predicate Selection
- Keywords Matching Element Names
- Indexing
- Experimental Setup
- Experimental Results
- Conclusions
- References
- UPF at INEX 2010: Towards Query-Type Based Focused Retrieval
- Introduction
- INEX Topic Classification
- Dimensions
- Data Classification
- Relevance Assessments Analysis
- Experiments
- Ad-Hoc Track Experiments
- Data-Centric Track Experiments
- Discussion and Conclusions
- References
- BUAP: A First Approach to the Data-Centric Track of INEX 2010
- Introduction
- Description of the System
- Data Processing
- Experimental Results
- Conclusions
- References
- Interactive Track
- Overview of the INEX 2010 Interactive Track
- Introduction
- Tasks
- Participating Groups
- Research Design
- Search System
- Document Corpus
- Online Questionnaires
- Relevance Assessments
- "Shopping" Basket
- Logging
- Experimental Procedure
- Results and Future Plans
- References
- Using Eye-Tracking for the Evaluation of Interactive Information Retrieval
- Introduction
- System Description
- Eye-Tracking for Interactive Information Retrieval Systems
- A Model for Interactive Information Retrieval
- User Study @ iTrack 2010
- Conclusion and Outlook
- References
- Link the Wiki Track
- Overview of the INEX 2010 Link the Wiki Track
- Introduction and Motivation
- Te Ara Test Set
- Runs
- Submission Format
- Assessment Tool
- Metrics
- Results
- Conclusions and Further Work
- References
- University of Otago at INEX 2010
- Introduction
- Related Work
- The Link-the-Wiki Track
- The Ad Hoc Track
- The Efficiency Track
- System Overview
- Indexer
- Query Evaluation
- The Link-the-Wiki Track
- Differences between Wikipedia and Te Ara
- Adapting to the Differences in Te Ara
- Algorithm
- Results
- Assessment Tool
- The Ad Hoc Track
- Learning Stemmers
- Refining Stemmers
- Experimental Results
- The Efficiency Track
- Experiments
- Results
- The Data Centric Track
- Conclusion and Future Work
- The Link-the-Wiki Track
- The Ad Hoc Track
- The Efficiency Track
- References
- Question Answering Track
- Overview of the INEX 2010 Question Answering Track (QA@INEX)
- Introduction
- Baseline System: Restricted Focused IR System
- Features
- Evaluation on the 2010 Restricted Focus Ad-Hoc Task
- Sets of Questions and References
- Encyclopedic vs. General Questions
- Short Type Questions
- Long Type Questions
- Format Submission
- Reference Texts
- Evaluation of Long Answers
- Results
- Plans for Next QA Track 2011 Edition
- Merging Short and Long Answer Task
- Towards Contextualization of Questions
- Conclusion
- References
- The GIL Summarizers: Experiments in the Track QA@INEX'10
- Introduction
- The ResúmeME Summarizer
- The GIL-UNAM-3 Summarizer
- Query Modification
- Experiments Settings and Results
- Conclusions
- References
- The Cortex Automatic Summarization System at the QA@INEX Track 2010
- Introduction
- The CORTEX System
- Experiments Settings and Results
- Conclusions
- References
- The REG Summarization System with Question Reformulation at QA@INEX Track 2010
- Introduction
- The REG System
- Terms and Name Entity Extraction
- Experiments Settings and Results
- Conclusions
- References
- Relevance Feedback Track
- Overview of the INEX 2010 Focused Relevance Feedback Track
- Introduction
- Focused Relevance Feedback
- Evaluation
- Task
- Overview
- Submission Format
- Results
- Submissions
- Evaluation
- Conclusion
- References
- Exploring Accumulative Query Expansion for Relevance Feedback
- Introduction
- Related Work
- System Setup
- Results and Analysis
- Conclusions and Future Work
- References
- Combining Strategies for XML Retrieval
- Introduction
- Ad Hoc Track
- Maximal Lowest Common Ancestor (MAXLCA)
- Distribution Measurements
- Learn to Optimize the Parameters
- Comparison Results
- Data Centric Track
- Related Work
- Extracting the Most Representative Attributes
- Relevance Feedback Track
- Rocchio Algorithm
- Revised Rocchio Algorithm
- Criterion Weight Adjustment
- References
- Web Service Discovery Track
- Overview of the INEX 2010 Web Service Discovery Track
- Introduction
- WSDL Collection
- Topics
- Submissions
- Assessment
- Evaluation and Results
- Conclusion
- Semantics-Based Web Service Discovery Using Information Retrieval Techniques
- Introduction
- Related Work
- Discovery Approach
- Pre-processing
- Semantic Analysis
- Linking Analysis
- System Integration
- Data Set
- Evaluation
- Conclusions and Future Work
- References
- The BUAP Participation at the Web Service Discovery Track of INEX 2010
- Introduction
- Description of the Presented Approaches
- Experimental Results
- Conclusions
- References
- XML Retrieval More Efficient Using Double Scoring Scheme
- Introduction
- Related Works
- BM25F Function Overview
- XML Retrieval Model
- Inverted File Definition
- The Leaf Node Scoring Function
- Double Scoring Function
- Double Scoring on BM25 (BM25W)
- Score Sharing Function
- Experiment Setup
- INEX Collections
- Ad Hoc Track
- Data Centric Track
- Web Service Discovery Track
- Conclusions
- References
- XML Mining Track
- Overview of the INEX 2010 XML Mining Track: Clustering and Classification of XML Documents
- Introduction
- Corpus
- Categories
- Clustering Task
- Clustering Evaluation Measures
- Clustering Participants, Submissions and Evaluations
- Classification Task
- Classification Evaluation Measures
- Classification Participants, Submissions and Evaluations
- Conclusion
- References
- An Iterative Clustering Method for the XML-Mining Task of the INEX 2010
- Introduction
- Description of the Two Approaches
- The K-Means Based Clustering Approach
- Experimental Results
- Conclusions
- References
- PKU at INEX 2010 XML Mining Track
- Introduction
- System Overview
- Document Model for Categorization (SLVM)
- BM25 Formula
- Chi-Square Feature Selection
- Parameters Tuning
- Pruning the Tree
- WordNet
- Using the Link Information to Improve the Result
- Other Tuning
- The Clustering Methods
- INEX 2010 Results
- System Settings
- Results
- Conclusions
- References
- Author Index
