U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by: Ian Soboroff (Fed)

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 26 - 50 of 61

Information Retrieval on the Blogosphere

July 30, 2012
Author(s)
Ian M. Soboroff, Rodrygo Santos, Craig Macdonald, Richard McCreadie, Iadh Ounis
Blogs have recently emerged as a new open, rapidly evolving and reac- tive publishing medium on the Web. Rather than managed by a central entity, the content on the blogosphere—the collection of all blogs on the Web—is produced by millions of independent

Overview of the TREC 2011 Web Track

November 15, 2011
Author(s)
Ian M. Soboroff, Nick Craswell, Charles L. Clarke, Gordon Cormack
The TREC Web Track explores and evaluates Web retrieval technology over large collections of Web data. In its current incarnation, the Web Track has been active for two years. For TREC 2010, the track includes three tasks: 1) an adhoc retrieval task, 2) a

A Comparative Analysis of Cascade Measures for Novelty and Diversity

February 10, 2011
Author(s)
Ian M. Soboroff, Charles L. Clarke, Nick Craswell, Azin Ashkan
Traditional editorial effectiveness measures, such as nDCG, remain standard for Web search evaluation. Unfortunately, these traditional measures can inappropriately reward re- dundant information and can fail to reflect the broad range of user needs that

The Effect of Assessor Errors on IR System Evaluation

July 19, 2010
Author(s)
Ben Carterette, Ian Soboroff
Recent efforts in test collection building have focused on scaling back the number of necessary relevance judgments and then scaling up the number of search topics. Since the largest source of variation in a Cranfield-style experiment comes from the topics

Test Collection Diagnosis and Treatment

June 15, 2010
Author(s)
Ian M. Soboroff
Test collections are a mainstay of information retrieval research. Since the 1990s, large reusable test collections have been developed in the context of community evaluations such as TREC, NTCIR, CLEF, and INEX. Recently, advances in pooling practice as

Blog Track Research at TREC

June 1, 2010
Author(s)
Ian M. Soboroff, Craig Macdonald, Rodrygo Santos, Iadh Ounis
The TREC Blog track aims to explore information seeking behaviour in the blogosphere, by building reusable test collections for blog-related search tasks. Since, its advent in TREC 2006, the Blog track has led to much research in this growing field, and

Overview of the TREC-2009 Blog Track

May 3, 2010
Author(s)
Ian M. Soboroff, Craig Macdonald, Iadh Ounis
In TREC 2009, the Blog track has been markedly revamped with the use of a new and larger sample of the blogosphere, cal led Blogs08, which has a 13-month timespan covering a period ranging from 14th January 2008 to 10th February 2009, and the introdu c-

Overview of the TREC 2008 Blog Track

January 12, 2010
Author(s)
Ian M. Soboroff, Iadh Ounis, Craig Macdonald
The Blog track explores the information seeking behaviour in the blogosphere. The track was introduced in 2006 [1], with a main pilot search task, namely the opinion-finding task. In TREC 2007 [2], the track investigated two main tasks inspired by the

Overview of the TREC 2008 Enterprise Track

January 1, 2010
Author(s)
Ian M. Soboroff, Krisztian Balog, Nick Craswell, Arjen de Vries, Paul Thomas, Peter Bailey
The goal of the enterprise track is to conduct experiments with enterprise data that reflect the experiences of users in real organizations. This year, we continued with the CERC collection introduced in TREC 2007. Topics were developed in conjunction with

Is spam an issue for opinionated blog post search?

July 19, 2009
Author(s)
Ian M. Soboroff, Craig Macdonald, Iadh Ounis
In opinion-finding, the retrieval system is tasked with re- trieving not just relevant documents, but those that also express an opinion towards the query target entity. This task has been studied in the context of the blogosphere by groups participating

A Guide to the RIA Workshop Data Archive

July 18, 2009
Author(s)
Ian M. Soboroff
During the course of the Reliable Information Access (RIA) workshop, a data archive was created to hold the outputs of the many experiments being done. This archive was designed to serve both as an organizational structure to support the researchers at the

Overview of the TREC 2007 Blog Track

December 17, 2008
Author(s)
Ian Soboroff, Craig Macdonald, Iadh Ounis
The goal of the Blog track is to explore the information seeking behaviour in the blogosphere. It aims to create the required in- frastructure to facilitate research into the blogosphere and to study retrieval from blogs and other related applied tasks

Overview of the TREC 2007 Enterprise Track

December 17, 2008
Author(s)
Ian M. Soboroff, Peter Bailey, Nick Craswell, Arjen de Vries
The goal of the enterprise track is to conduct experiments with enterprise data that reflect the experiences of users in real organizations. This year, the track has introduced a new corpus with the goal to be more representative of real-world enterprise

Limits of Opinion-Finding Baseline Systems

July 21, 2008
Author(s)
Ian M. Soboroff, Craig Macdonald, Ben He, Iadh Ounis
In opinion-finding, the retrieval system is tasked with re- trieving not just relevant documents, but which also express an opinion towards the query target entity. Most opinion- finding systems are based on a two-stage approach, where initially the system

Relevance assessment: are judges exchangeable and does it matter?

July 21, 2008
Author(s)
Ian M. Soboroff, Peter Bailey, Nick Craswell, Alan Smeaton, Emine Yilmaz, Paul Thomas
We investigate to what extent people making relevance judgments for a reusable IR test collection are exchangeable. We consider three classes of judge: gold standard judges, who are topic origi- nators and are experts in a particular information seeking

On The TREC Blog Track

April 30, 2008
Author(s)
Iadh Ounis, Craig Macdonald, Ian Soboroff
The rise of blogging as a new grassroots publishing medium and the many interesting peculiarities that characterize blogs compared to other genres of documents opened up several new interesting research areas in the information retrieval field. The Blog

The TREC 2006 Terabyte Track

March 24, 2008
Author(s)
Stefan Buttcher, Charles L. Clarke, Ian Soboroff
The primary goal of the Terabyte Track is to develop an evaluation methodology for terabyte-scale document collections. In addition, we are interested in efficiency and scalability issues, which can be studied more easily in the context of a larger

Overview of the TREC 2006 Enterprise Track

February 25, 2008
Author(s)
Ian M. Soboroff, Arjen de Vries, Nick Craswell
The goal of the enterprise track is to conduct experiments with enterprise data --- intranet pages, email archives, document repositories --- that reflect the experiences of users in real organizations, such that for example, an email ranking technique

Overview of the TREC 2006 Blog Track

November 27, 2007
Author(s)
Iadh Ounis, Maarten de Rijke, Craig Macdonald, Gilad Mishne, Ian Soboroff
The Blog track began this year, with the aim to explore the information seeking behaviour in the blogosphere. For this purpose, a new large-scale test collection, namely the TREC Blog06 collection, has been created. In the first pilot run of the track in

A Comparison of Pooled and Sampled Relevance Judgments

August 29, 2007
Author(s)
Ian M. Soboroff
Test collections are most useful when they are reusable, that is, when they can be reliably used to rank systems that did not contribute to the pools. Pooled relevance judgments for very large collections may not be reusable for two reasons: they will be

The TREC 2005 Terabyte Track

August 27, 2007
Author(s)
Charles L. Clarke, Falk Scholer, Ian Soboroff
The Terabyte Track explores how retrieval and evaluation techniques can scale to terabyte-sized collections, examining both efficiency and effectiveness issues. TREC 2005 is the second year for the track. The track was introduced as part of TREC 2004, with

Problems with Kendall's Tau

July 23, 2007
Author(s)
Mark Sanderson, Ian Soboroff
Test collections are most useful when they are reusable, that is, when they can be reliably used to rank systems that did not contribute to the pools. Pooled relevance judgments for very large collections may not be reusable for two reasons: they will be

Reliable Information Retrieval Evaluation With Incomplete and Biased Judgements

July 23, 2007
Author(s)
Stefan Buttcher, Charles L. Clarke, Peter C. Yeung, Ian Soboroff
Information retrieval evaluation based on the pooling method is inherently biased against systems that did not contribute to the pool of judged documents. This may distort the results obtained about the relative quality of the systems evaluated and thus

Bias and the Limits of Pooling for Large Collections

July 17, 2007
Author(s)
C E. Buckley, Darrin L. Dimmick, Ian Soboroff, Ellen M. Voorhees
Modern retrieval test collections are built through a process called pooling in which only a sample of the entire document set is judged for each topic. The idea behind pooling is to find enough relevant documents such that when unjudged documents are