Dynamic Search Models and Applications
Yang, Grace Hui
Dynamic search is an information retrieval task that involves a sequence of queries for a complex information need (e.g. searching for one-week tour plans in Italy). It is characterized by rich user-system interactions and temporal dependency between queries and between consecutive user behaviors. Dynamic search is a trial-and-error process, which matches well with the family of Reinforcement Learning (RL) algorithms: the RL algorithms learn from repeated, varied attempts which are continued until success. The learner/agent learns from its dynamic interactions with the world. These similarities inspire us to model dynamic search using RL frameworks.In particular, we model dynamic search as a dual-agent stochastic game, one of the standard variants of Partially Observable Markov Decision Processes (POMDPs), where the user agent and the search engine agent work together to jointly maximize their long-term rewards. In the framework, users' search statuses, such as exploitation, exploration, struggle, etc. are modeled as hidden states, which can only be estimated through user interaction data. In each search iteration, one search algorithm is picked from a set of candidates to maximize a reward function. Our work provides a general framework to model dynamic search. It enables the use of Reinforcement Learning algorithms for optimizing retrieval results. The experiments on the Text REtrieval Conference (TREC) 2011-2014 Session Track datasets show a statistically significant improvement over the state-of-the-art dynamic search algorithms.The design of states, actions, and rewards is quite flexible when using POMDPs in dynamic search. In the thesis, we also examine all available design choices from related work and compare their retrieval accuracy and efficiency. Our analysis reveals the effects of these choices, which enables us to recommend practical design choices. The finding again proves that modeling dynamic search using POMDPs is promising, however, it also shows that this approach is computationally demanding.To improve the efficiency of the above dynamic search models, we propose another RL framework, direct policy learning, which finds optimal policies for the best search engine actions directly from what is observed in the user and search engine interactions via gradient descent. The proposed framework greatly reduces the model complexity over the POMDP framework. It is also a flexible design, which includes a wide range of features describing the rich interactions in dynamic search. The framework is shown to be highly effective and efficient on the TREC Session Tracks.In addition to the dynamic search frameworks, we propose predictive models to detect user struggling states in search. Most prior work uses effort-based features to detect user struggle. However, recent studies suggest that there might be a gap between user effort and user struggle. In this work, we take a psychological perspective to bridge this gap. We demonstrate that after removing the biases introduced by different searcher motivations, user struggle can be tightly connected to user effort.At last, we implement a dynamic search tool to support the annotation task for the TREC Dynamic Domain Tracks. This serves as our first trial of implementing dynamic search algorithm and struggling detection in practice. The results show that our algorithm is also effective in real-life settings.The research in this thesis is among the first in this field and serves as one step towards solving dynamic search and developing real-world applications.
MetadataShow full item record
Showing items related by title, author, creator and subject.
Wu, Hongkai (Georgetown University, 2016)Dynamic search is an information retrieval task, in which information systems retrieve documents for a user’s multiple queries. Each query starts a search iteration and aims to fulfill part of the user’s information need. ...