Publications

2025

1. Rethinking Dataset Discovery with DataScout

   Rachel Lin, Bhavya Chopra, Wenjing Lin, and 3 more authors

   UIST ’25: Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, 2025
2. How well do LLMs reason over tabular data, really?

   Cornelius Wolff, and Madelon Hulsebos

   In Proceedings of the 4th Table Representation Learning Workshop, 2025

   Abs DOI

   Large Language Models (LLMs) excel in natural language tasks, but less is known about their reasoning capabilities over tabular data. Prior analyses devise evaluation strategies that poorly reflect an LLM’s realistic performance on tabular queries. Moreover, we have a limited understanding of the robustness of LLMs towards realistic variations in tabular inputs. Therefore, we ask: Can general-purpose LLMs reason over tabular data, really?, and focus on two questions 1) are tabular reasoning capabilities of general-purpose LLMs robust to real-world characteristics of tabular inputs, and 2) how can we realistically evaluate an LLM’s performance on analytical tabular queries?Building on a recent tabular reasoning benchmark, we first surface shortcomings of its multiple-choice prompt evaluation strategy, as well as commonly used free-form text metrics such as SacreBleu and BERT-score. We show that an LLM-as-a-judge procedure yields more reliable performance insights and unveil a significant deficit in tabular reasoning performance of LLMs. We then extend the tabular inputs reflecting three common characteristics in practice: 1) missing values, 2) duplicate entities, and 3) structural variations. Experiments show that the tabular reasoning capabilities of general-purpose LLMs suffer from these variations, stressing the importance of improving their robustness for realistic tabular inputs.

3. Metadata Matters in Dense Table Retrieval

   Daniel Gomm, and Madelon Hulsebos

   In ELLIS workshop on Representation Learning and Generative Models for Structured Data, 2025

2024

1. TARGET: Benchmarking Table Retrieval for Generative Tasks

   Xingyu Ji, Aditya Parameswaran, and Madelon Hulsebos

   In NeurIPS 2024 Third Table Representation Learning Workshop, 2024
2. “It Took Longer than I was Expecting:” Why is Dataset Search Still so Hard?

   Madelon Hulsebos, Wenjing Lin, Shreya Shankar, and 1 more author

   2024
3. SchemaPile: A Large Collection of Relational Database Schemas

   Till Döhmen, Radu Geacu, Madelon Hulsebos, and 1 more author

   Proceedings of the ACM on Management of Data, 2024

2023

1. Observatory: Characterizing Embeddings of Relational Tables

   Tianji Cong, Madelon Hulsebos, Zhenjie Sun, and 2 more authors

   PVLDB, 2023
2. GitTables: A large-scale corpus of relational tables

   Madelon Hulsebos, Çağatay Demiralp, and Paul Groth

   Proceedings of the ACM on Management of Data, 2023