Submission by AWS Quick team: https://aws.amazon.com/quicksuite/
Contributors: Ravi Shankar, Leah Riley, Adi Kalyanpur
BIRD leaderboard: https://bird-bench.github.io/
If you find our work helpful, feel free to give us a cite.
@techreport{qsql2025aws,
title={Q-SQL: AWS, Memory with RL for Structured Data },
author={Shankar, Ravi and Riley, Leah and Kalyanpur, Adi},
institution={AWS Quick Team},
year={2025},
month={December},
url={<https://aws.amazon.com/quicksuite/>},
note={BIRD Benchmark Submission}
}
✅ Consensus @K Results (Execution-Based)
| Version | Correct / Total | Accuracy | Δ vs Gemini |
|---|---|---|---|
| Spektr-SQL(Amazon Ads) | 1104 / 1532 | 72.06 % | -0.56 pts |
| Gemini-2.5 Pro | 1113 / 1533 | 72.63 % | N/A |
| Q-SQL(AWS: Quick Science) | 1119 / 1533 | 72.99 % | +0.36 pts |
📊 Accuracy based on Count of Candidates(Higher is better):
| K (Candidates) | Accuracy (%) | Correct | Gold OK |
|---|---|---|---|
| 1 | 70.55 | 1078 | 1528 |
| 2 | 71.69 | 1099 | 1533 |
| 3 | 71.80 | 1100 | 1532 |
| 4 | 72.54 | 1112 | 1533 |
| 5 | 72.54 | 1112 | 1533 |
| 6 | 72.02 | 1104 | 1533 |
| 7 | 72.73 | 1115 | 1533 |
| 8 | 72.73 | 1115 | 1533 |
| 9 | 72.60 | 1113 | 1533 |
| 10 | 72.80 | 1116 | 1533 |
| 11 | 73.39 | 1125 | 1533 |
| 12 | 72.93 | 1118 | 1533 |
| 13 | 72.86 | 1117 | 1533 |
| 14 | 72.93 | 1118 | 1533 |
| 15 | 72.99 | 1119 | 1533 |