28-31 August 2018
Bertinoro, Italy


Design of Experimental Search
Information REtrieval Systems

Sponsored by


Do you want to be a sponsor?
Check out our sponsorship opportunities!


A systems-oriented biennial conference, complementary in its mission to the mainstream Information Access and Retrieval conferences, emphasizing the innovative technological aspects of search and retrieval systems.

It gathers researchers and practitioners from both academia and industry to discuss the latest innovative and visionary ideas in the field.

It is a retreat-like event that favors discussion and brainstorming.

One goal is to give voice to topics which are not easy to discuss in other venues.

Another goal is to constitute a program with many invited lecturers and compelling evening activities.

Call for papers

DESIRES welcomes contributions focusing on implementation details, successful or failed reproducibility attempts, technological breakthroughs and
new uses of old ideas.

Running prototypes and war stories are more than welcome.

Industrial and "real-world" experiences play a central role in DESIRES. For instance, papers discussing how well-established research ideas are implemented and used in commercial products are encouraged.

DESIRES looks for original work that cannot be submmitted in parallel to another venue.

DESIRES is a single-track conference focused on discussion.
To encourage authors to submit only their best work, each person can be an author or co-author of only a single paper or demo.
That is, authors can submit only 1 (one) paper.

DESIRES invites 3 kinds of contributions:

Papers (up to 6 pages)
Papers usually lack rigorous frameworks, simulations of performance, or traditional evaluations but present a radical departure from conventional approaches that enables new applications. There will be a discussant for each paper that will read the paper in advance and prepare questions (plus questions from the audience).
Prototypes (up to 6 pages)
Prototype descriptions generally are a detailed report on implementation techniques, successes and mistakes, system design decisions, and/or operational considerations. Instrumentation and implementation of novel experiments are also considered.
Abstracts (1 page)
Ideas that are too half-baked for a paper or demo proposal are good candidates for an abstract. Any author of a paper or prototype demo may additionally submit one abstract. Abstracts are expected to have a single author. Accepted abstracts will be presented in a gong-show style.

Submissions must be in English, in PDF format, and should not exceed the specified length (including references and figures). Suitable LaTeX and Word templates are available from the ACM Website; please use the sigconf ACM proceedings template.

The accepted (and presented) papers will be published as CEUR-WS proceedings freely available on-line (Indexed by DBLP and Scopus).
Modified or incremental versions of the papers included in the DESIRES proceedings can be submitted to other venues.

DESIRES adopts a single-blind peer review process, so you do not need to anonymize your paper.

Submit here

Important dates

Submission deadline: 15 April 2018 23 April 2018, AoE

Notifications: 30 May 2018 07 June 2018

Camera-ready: 11 July 2018

Conference: 28-31 August 2018


DESIRES 2018 will be held at the
University Residential Centre of Bertinoro (Ce.U.B.)

The Ce.U.B complex is a former fortress (dating back to the 10th century) and an historical landmarks considered one of the most important monumental complex of the Region.

The Ce.U.B. offers the possibility to accommodate up to 86 people in single rooms and provides a big canteen where the participants can share a meal and a chat.

All the participants should reside at the Ce.U.B. in order to maximize participation and ideas exchange. More information about registration procedures and costs will be provided after the submission deadline.

How to reach Ce.U.B.


DESIRES 2018 is organized under the Bertinoro international Center for informatics (BiCi) umbrella.
BiCi is an association whose mission is to foster cutting-edge research and advanced education in Computer Science.

General Chairs

Omar Alonso (Microsoft, USA)
Gianmaria Silvello (University of Padua, Italy)

Advisory Board

Maristella Agosti (University of Padua, Italy)
Ricardo Baeza-Yates (NTENT, USA; UPF, Spain; Univ. de Chile)
Bruce Croft (University of Massachusetts, USA)
Susan Dumais (Microsoft, USA)
Norbert Fuhr (University of Duisburg-Essen, Germany)
Donna Harman (NIST, USA)
Kalervo Jarvelin (University of Tampere, Finland)
Marc Najork (Google, USA)
Jan Pedersen (eBay, USA)
Gerhard Weikum (MPI, Germany)

Program Committee

James Allan (University of Massachusetts, USA)
Barla Cambazoglu (NTENT, USA)
Diego Ceccarelli (Bloomberg, UK)
Jeffrey Dalton (University of Glasgow, UK)
Claudia Hauff (Delft University of Technology, The Netherlands)
Nicola Ferro (University of Padua, Italy)
Evgenyi Gabrilovich (Google, USA)
Krishna Gummadi (MPI-SWS, Germany)
Vasileios Kandylas (Microsoft, USA)
Bhaskar Mitra (Microsoft, UK)
Nicola Montecchio (Spotify, Germany)
Edgar Meij (Bloomberg, UK)
Jussi Karlgren (Gavagai & KTH, Sweden)
Emre Kiciman (Microsoft, USA)

Andreas Rauber (Vienna University of Technology, Austria)
Tony Russell-Rose (2dSearch, UK)
Tetsuya Sakai (Waseda University, Japan)
Mark Sanderson (RMIT, Australia)
Sunita Sarawagi (IIT Bombay, India)
Ian Soboroff (NIST, USA)
Paul Thomas (Microsoft, USA)
Andrew Trotman (eBay)
Arjen de Vries (Radboud University, The Netherlands)
Ingmar Weber (QCRI, Qatar)
Wouter Weerkamp (904 Labs, The Netherlands)
Cong Yu (Google Research, USA)
Justin Zobel (University of Melbourne, Australia)


Omar Alonso and Gianmaria Silvello: DESIRES 2018 - Proceedings of the First Biennial Conference on Design of Experimental Search & Information Retrieval Systems, Bertinoro, Italy, August 28-31, 2018. CEUR Workshop Proceedings 2167, CEUR-WS.org 2018


Program Overview

Welcome Dinner: Giardino del Rivellino

Social Dinner: Ca de Be


The first keynote speaker is Jimmy Lin, Professor and the David R. Cheriton Chair in the David R. Cheriton School of Computer Science at the University of Waterloo.

Title of the keynote:
Computing without Servers, V8, Rocket Ships, and Other Batsh*t Crazy Ideas in Data Systems

The second keynote speaker is Diego Ceccarelli, Software engineer at Bloomberg LP based in London, UK.

Title of the keynote:
Learning to Rank at Bloomberg - From Theory to Production

The third keynote speaker is Shane Culpepper, Vice-Chancellor's Principal Research Fellow, Director of the Centre for Information Discovery, and Professor at RMIT University in the School of Science.

Title of the keynote:
Balancing ​Efficiency and Effectiveness ​Trade-offs in Large Scale Multi-Stage Search Engines

Detailed Schedule

Click the title to download the pre-print version of the papers

Wednesday 29 August 2018

Session 1: Systems

Emre Kiciman and Jorgen Thelin
Answering What If, Should I, and Other Expectation Exploration Queries Using Causal Inference over Longitudinal Data
Johannes Kiesel, Arjen P. De Vries, Matthias Hagen, Benno Stein and Martin Potthast
WASP: Web Archiving and Search Personalized

Session 2: Models

Jussi Karlgren and Pentti Kanerva
Hyperdimensional utterance spaces
Jeffrey Dalton and John Foley
Search Agent Model: A Conceptual Framework for Search by Algorithms and Agent Systems
Hamed Zamani and Bruce Croft
Joint Modeling and Optimization of Search and Recommendation

Gong-Show I

Matt Lease
Fact Checking and Information Retrieval
Dennis Dosso
Keyword Search on RDF Graphs
Shane Culpepper
Single Query Optimisation is the Root of All Evil
Rodger Benham
Topic-Specific Off-line Web Search
Maram Barifah
Building Interaction Profiles for Better Search Tools in DLs
Benjamin Hättasch
Towards Interactive Summarization of Large Document Collections
Stefano Marchesin
Implicit-Explicit Representations for Case-Based Retrieval

Thursday 30 August 2018

Session 3: Experimentation

Mucahid Kutlu, Tyler McDonnell, Aashish Sheshadri, Tamer Elsayed and Matthew Lease
Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably
Monica Marrero
APONE: Academic Platform for ONline Experiments
Craig Macdonald, Richard Mccreadie and Iadh Ounis
Agile Information Retrieval Experimentation with Terrier Notebooks

Session 4: Querying and Retrieval

Rodger Benham, Shane Culpepper, Luke Gallagher, Xiaolu Lu and Joel Mackenzie
Towards Efficient and Effective Query Variant Generation
Eugene Yang, Dave Lewis, Ophir Frieder, David Grossman and Roman Yurchak
Retrieval and Richness when Querying by Document

Industry Panel

The goal of the panel is to discuss topics that are of importance in industrial settings and may not be of significance in academia. Moreover, another goal is to examine problems, emergent trends, and their implications for the future of the field.

Emre Kiciman, Microsoft
David Lewis, Cyxtera
Mark Najork, Google
Tony Russel Rose, UXLabs

Gong-Show II

Marc Najork
Training On-Device Ranking Models from Cross-User Interactions in a Privacy-Preserving Fashion
Giorgio Maria Di Nunzio
Finding all the Needles in the Haystack. A System to Estimate the Costs of e-Discovery and Systematic Reviews
Chris Stahlhut
Searching Arguments in German with ArgumenText
David Lewis
We Don't Know S***** (Search)
Alberto Purpura
Non-negative Matrix Factorization for Topic Modeling
Jan-Christoph Klie
INCEpTION: Interactive machine-assisted annotation

Friday 31 August 2018

Session 5: Novel Applications

Sindunuraga Rikarno Putra, Kilian Grashoff, Felipe Moraes and Claudia Hauff
On the Development of a Collaborative Search System
Chris Stahlhut, Christian Stab and Iryna Gurevych
Pilot Experiments of Hypothesis Validation Through Evidence Detection for Historians
Tony Russell-Rose and Phil Gooch
2dSearch: a visual approach to search strategy formulation


To register to DESIRES 2018 please fill in the form available at the following URL: Registration Form For additional information please contact Roberta Partisani

The registration fees include conference costs, full board accommodation in single or double room at the University Residential Center of Bertinoro or hotel nearby for 3 nights (in: August 28 – out: August 31), meals and coffee breaks, social dinner, museum visit, shuttle service from Bologna airport to the course venue and back

Single room Double room*
Regular 550€ 480€
Student 420€ 360€

* if you’d like to share a double room, please specify in the field note of your booking, your roommate. Otherwise the organizing secretariat will arrange the combination. If you travel with an accompanying person please contact Roberta Partisani before starting the registration procedure.

It is possible to pay via bank transfer or credit card.

There are a limited number of seats for DESIRES 2018 and they are assigned on a first come first served basis. At least one author per paper must register to the event.