Retrieval Systems for Structured Data: the critical missing piece for grounding LLM-driven query interfaces in factual data

Whether analytical questions are answered by querying relational databases or in dialog with the latest Large Language Model (LLM): a key component for ensuring trustworthy answers is the retrieval of relevant data to answer the question at hand. In this talk, we will discuss how retrieval systems for structured data are key in grounding LLM-driven query interfaces in factual and domain-specific data stored in relational databases and data lakes. We present insights surfaced with our novel TARGET benchmark for analyzing Table Retrieval for Generative Tasks such as Question Answering and Text-to-SQL. For example, with TARGET, we find that the popular BM25 retrieval algorithm is not as robust for retrieval over structured data as over text documents. Our analysis also points out that systems using LLM-based embeddings can perform well but suffer from variations across tasks and datasets of varying difficulties, highlighting the need for more research in this direction. We conclude with early insights into a lightweight multi-table retrieval method which reduces the semantic gap between queries and tables for retrieval.

Speaker: Madelon Hulsebos, researcher

Register to attend in person, or see weblink for connection information online