Overview of the NTCIR-17 FairWeb-1 Task

Published

February 13, 2024

Author(s)

Sijie Tao, Nuo Chen, Tetsuya Sakai, Zhumin Chu, Hiromi Arai, Ian Soboroff, Nicola Ferro, Maria Maistro

Abstract

This paper provides an overview of the NTCIR-17 FairWeb-1 Task. FairWeb-1 is an English web search task which seeks more than an ad-hoc web search task. Our task considers not only document relevance but also group fairness. We designed three types of search topics for this task: researchers (R), movies (M), and Youtube con- tents (Y). For each topic type, attribute sets are defined for consider- ing group fairness. We utilise a deduped version of the Chuweb21 corpus as the target corpus. We received 28 runs from six teams, in- cluding six runs from the organisers team. In this paper, we describe the task, the test collection construction and the official evalution results of the submitted runs.

Proceedings Title

Proceedings of the 17th NTCIR (NII Testbeds and Community for Informtion Access Research) Conference

Conference Dates

December 12-15, 2023

Conference Location

Tokyo, JP

Conference Title

NTCIR-17

Pub Type

Conferences

Download Paper

https://doi.org/10.20736/0002001318

Local Download

Keywords

information retrieval, fairness in ai

Citation

Tao, S. , Chen, N. , Sakai, T. , Chu, Z. , Arai, H. , Soboroff, I. , Ferro, N. and Maistro, M. (2024), Overview of the NTCIR-17 FairWeb-1 Task, Proceedings of the 17th NTCIR (NII Testbeds and Community for Informtion Access Research) Conference, Tokyo, JP, [online], https://doi.org/10.20736/0002001318, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=956867 (Accessed October 13, 2025)