Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Overview of the NTCIR-17 FairWeb-1 Task

Published

Author(s)

Sijie Tao, Nuo Chen, Tetsuya Sakai, Zhumin Chu, Hiromi Arai, Ian Soboroff, Nicola Ferro, Maria Maistro

Abstract

This paper provides an overview of the NTCIR-17 FairWeb-1 Task. FairWeb-1 is an English web search task which seeks more than an ad-hoc web search task. Our task considers not only document relevance but also group fairness. We designed three types of search topics for this task: researchers (R), movies (M), and Youtube con- tents (Y). For each topic type, attribute sets are defined for consider- ing group fairness. We utilise a deduped version of the Chuweb21 corpus as the target corpus. We received 28 runs from six teams, in- cluding six runs from the organisers team. In this paper, we describe the task, the test collection construction and the official evalution results of the submitted runs.
Proceedings Title
Proceedings of the 17th NTCIR (NII Testbeds and Community for Informtion Access Research) Conference
Conference Dates
December 12-15, 2023
Conference Location
Tokyo, JP
Conference Title
NTCIR-17

Keywords

information retrieval, fairness in ai

Citation

Tao, S. , Chen, N. , Sakai, T. , Chu, Z. , Arai, H. , Soboroff, I. , Ferro, N. and Maistro, M. (2024), Overview of the NTCIR-17 FairWeb-1 Task, Proceedings of the 17th NTCIR (NII Testbeds and Community for Informtion Access Research) Conference, Tokyo, JP, [online], https://doi.org/10.20736/0002001318, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=956867 (Accessed April 17, 2024)
Created February 13, 2024