Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Challenges of Mathematical Information Retrieval in the NTCIR-11 Math Wikipedia Task



Howard S. Cohl, Abdou S. Youssef, Moritz Schubotz


Mathematical Information Retrieval concerns retrieving information related to a particular mathematical concept. The NTCIR-11 Math Task develops an evaluation test collection for document sections retrieval of scientific articles based on human generated topics. Those topics involve a combination of formula patterns and keywords. In addition, the optional Wikipedia Task provides a test collection for retrieval of individual mathematical formula from Wikipedia based on search topics that contain exactly one formula pattern. We developed a framework for automatic query generation and immediate evaluation. This paper discusses our dataset preparation, topic generation and evaluation methods, and summarizes the approaches of the participants, with a special focus on the Wikipedia Task.
Conference Dates
August 9-13, 2015
Conference Location
Conference Title
The 38th Annual ACM SIGIR Conference


evaluation, test collections, other applications, digital libraries, IR data, structured data, XML


Cohl, H. , Youssef, A. and Schubotz, M. (2015), Challenges of Mathematical Information Retrieval in the NTCIR-11 Math Wikipedia Task, The 38th Annual ACM SIGIR Conference, Santiago, -1, [online], (Accessed April 12, 2024)
Created August 8, 2015, Updated October 6, 2020