Abstract
Frontier language models in agentic pipelines lead text-to-SQL benchmarks, at a high inference cost. Small specialised models would avoid that cost, but training them needs data that reflects the scale, semantics and structure of real databases. SQaLe is a semi-synthetic text-to-SQL dataset built on 9,259 real-world schemas from SchemaPile. Our generation pipeline extends each schema, fills it with synthetic rows, generates questions and answers them with an agent, and validates every stage by execution. The result is 1.4 million natural-language questions paired with SQL, over schemas far larger than in any existing dataset. Training a 2B parameter model on SQaLe with GRPO alone raises its execution accuracy on an independent test set from 38.7% to 66.3%. It does so by learning effectively how to explore the dataset. It finds the relevant tables in large schemas and reads the values its answer needs.
Why SQaLe
Small specialised models avoid the inference cost of frontier agentic pipelines. Training them needs data that looks like the databases they will be used on, and the common text-to-SQL training sets use small schemas. Yet, BIRD’s training set has significantly fewer samples, while SynSQL lacks realistic schemas. A model trained on them does not learn to search for the tables a question needs effectively.
SQaLe starts from 9,259 real-world schemas from SchemaPile, extends each one with tables in its own style and fills it with synthetic rows. Its schemas have a median of 113 tables and 538 columns, while also delivering enough questions and SQL queries to make for an effective training dataset.
The dataset
- question–SQL pairs
- 1.4M
- real-world schemas
- 9,259
- distinct SQL queries
- 176,761
- median tables per schema
- 113
- median columns per schema
- 538
- synthetic rows
- 109M
Each record pairs a natural-language question with a schema and a gold SQL query that has been run against the populated database. Every question exists in 8 phrasings, the original and seven rewrites in different styles, and all eight share one SQL query. One record is laid out in full on its own page, with the schema it was written against.
The dataset is published on Hugging Face in two parts that join on schema_id. trl-lab/SQaLe-2-text-to-SQL-Queries holds the questions in all eight phrasings, the gold SQL and its result. trl-lab/SQaLe-2-text-to-SQL-Schemas holds the DDL and the generated rows of each database.
The result
To isolate the effect of the data, each of the three runs starts from the same Qwen3.5-2B checkpoint and trains with GRPO on one corpus: SQaLe, SynSQL-2.5M or BIRD train (MSQaLe, MSynSQL and MBIRD). MSQaLe answers as well as untrained models four times its size. It sits 28 points above the best untrained model of its own size, 52% against 24%, and it takes an 8B general-purpose model to reach the same accuracy. Qwen3.5-27B leads at 63%.
Accuracy against model size, 100 moderate SQaLe test questions
All 25 models as a table
Training on SQaLe
Apart from the training data and the strength of a length curriculum, the base model, environment, tools, reward, steps, batch size and evaluation are identical. There are no distilled traces, no teacher and no test-time scaffolding.2
On the SQaLe test set (Figure 2), MSQaLe reaches 66.3%, sitting far above the base model and also outperforming MBIRD and MSynSQL by considerable margins.
Execution accuracy (%) by training corpus
What a small model learns
The three trained models differ only in their training data, so differences in how they behave come from the data. We look at four of those differences.
Exploration
The environment caps rounds, not tool calls, so the only way for the model to see more of a database is to ask more in each round. Over training it learns to do that: tool calls per episode roughly triple, while the number of rounds stays near six. Three real episodes are replayed round by round, on schemas of 106 to 239 tables.
Inside one episode, over GRPO training on SQaLe
Schema size
Accuracy falls for every model as the schema grows, and MSQaLe leads at every size, from 72.7% with only the gold tables to 66.3% on the full schema. With only the gold tables present, all three models open every gold table in at least 95% of episodes. On the full schema, MSQaLe opens every gold table in 89.7% of episodes, again outperforming MBIRD and MSynSQL. BIRD and SynSQL training schemas have a median of 5 and 10 tables, so models trained on them never have to search.
Populated tables
Before submitting, MSQaLe has seen 88% of the string literals its answer depends on. MSynSQL has seen 55%, MBIRD 40% and the untrained model 35%. SynSQL’s tables hold a median of 2 rows, and SQaLe’s hold 69.
Schema size
Share of episodes in which the model opened every gold table (%) at three schema sizes
All nine values as a table
String literals seen before submitting
Share of the string literals the answer depends on that the model saw in tool output (%)
Domains
On SQaLe test questions whose schemas lie outside BIRD’s domains, MSQaLe leads MBIRD by 9.9 points. Inside BIRD’s domains the lead is 2.8 points.3
How it compares
How much searching a model has to learn depends on how large its training schemas are. Figure 5 compares schema size across four datasets. The median SQaLe schema has 113 tables and 538 columns. The next largest medians are EHRSQL’s 13.5 tables and 92 columns, measured on 2 schemas. SQaLe also has the most foreign-key relations, 1,196,078 across its 9,259 schemas.
23.4% of SQaLe queries are nested, against 7.7% in BIRD, and 40% of the queries that join chain multiple joins, against 26% in BIRD. 3.1% of queries touch five or more tables, up to 20. No BIRD or EHRSQL query touches more than four.
How it is built
Schemas that size do not come ready to use. The pipeline runs in five stages, and every stage is validated by execution. Value synthesis repairs rows that break key constraints, and answering sends failed or rejected queries back to the agent.
Show how it is builtHide how it is built
- Schema collection and extension. SchemaPile provides real-world schemas from permissively licensed GitHub repositories. A tool-using agent annotates each of the 14,597 source repositories with a domain description. An LLM then extends each schema with tables that keep its naming conventions, normalisation level and foreign-key style.
- Value synthesis. Tables are filled in foreign-key dependency order. For each table the LLM writes a Python function from its DDL, its original rows, the allowed foreign-key values and the domain description. Fact and junction tables get more rows and a skewed key distribution, and every table is checked for primary-key uniqueness and referential integrity.
- Question generation. A connected subgraph of up to 20 tables is sampled along foreign keys, together with sample rows. The LLM writes questions that need every table in it, at three difficulty levels (simple, moderate, hard), with every literal grounded in the data.
- Style variation. Each question is rewritten into the seven styles shown in the example record. All eight versions share the gold SQL.
- Agentic answering and judging. An agent explores the live database with tools (
list_tables,describe_table,sample_rows,distinct_values,run_query) and commits withsubmit_sql. Execution errors go back to the agent. An LLM judge checks the result against the question and sends rejected queries back for a rewrite.1
Cite
If you use SQaLe, please cite the paper, Towards Specialized Text-to-SQL Models with SQaLe (PDF), and the earlier workshop paper.
Notes
- On 158 BIRD dev questions (225 judge calls), 94.5% of the queries the judge accepts are aligned with the question. The judge agrees with human labels 86.2% of the time (κ = 0.68).↩
- The schema is withheld. The model explores the database through tools, the five used in answering plus
foreign_keysandjoin_path. The reward is tiered (correct result, then executes, then parses, then nothing), and partial credit never ranks a wrong query above a correct one.↩ - The domain split covers 108 schemas. That sample is too small to call the difference between the two gaps significant.↩