Sim4IR: Workshop on Simulation for Information Retrieval Evaluation
Co-located with SIGIR 2021 (Virtual Event)
July 15, 2021
Simulation techniques are not foreign to information retrieval. Simulation has been employed, for example, for constructing test collections and for model performance prediction and analysis in a broad array of information access scenarios. The need for simulation has become ever more apparent recently with the emergence of areas where other types of evaluation are infeasible. One such area is conversational information access, where human evaluation is both time and resource intensive at scale. Another example is provided by settings that do not allow sharing of data, e.g., because of privacy constraints, and therefore necessitate the creation of synthetic test collections.
Despite the apparent need, a standardized methodology for performance evaluation via simulation has not yet been developed. The goal of the Sim4IR workshop is to create a forum for researchers and practitioners to promote methodology development and more widespread use of simulation for evaluation by:
We invite submissions of regular papers, position papers, demonstrators as well as encore talks (featuring already published work).
Topics of interest include, but are not limited to: