Key result
Demonstrated practical NL-to-SQL interaction quality across four LLMs using human evaluation metrics.
Why it matters
- Non-technical users often face barriers when querying structured databases directly.
- Natural-language access can democratize analytics and reduce operational dependency on SQL experts.
Approach
- Built a chatbot pipeline that maps user questions to SQL queries and verbalizes returned results.
- Benchmarked multiple LLM backends with human-centric quality evaluation criteria.
Results
- Validated feasibility of conversational database access for real workflows.
- Provided a comparative baseline for improving reliability and usability in NL database assistants.
Abstract
In today’s data-driven world, the ability to access and interpret information is crucial for informed decision-making. However, technical complexities in database management often create barriers for non-experts. This paper presents a prototype of a chatbot leveraging large language models (LLMs) to bridge this gap. The chatbot interprets natural language questions about structured databases and translates them into SQL queries, retrieving relevant data and converting results back into natural language. The system was tested with four different LLMs, evaluated by human evaluation on a number of metrics: accuracy, relevance, fluency, completeness, cohesion, and naturalness. The results show the usefulness of the approach and how it can make data interaction more user-friendly by democratizing data access, reducing human workload, and empowering users to focus on strategic tasks.