Comparison: detailed results

Per-statement breakdowns, reproduction instructions, and methodology for the competitive comparison.

Generated on arm64-darwin with syntaqlite 0.1.0 on 2026-03-14.

To reproduce these results: tools/run-comparison --setup && tools/run-comparison --all. See the script for requirements and tool versions.

Parser details

Ground truth

Every test statement validated against sqlite3 (via EXPLAIN):

40/40 statements validated by sqlite3.

Per-statement results

Legend: PASS = correctly parses valid SQL, FAIL = rejects valid SQL, FP = accepts invalid SQL

Scoreboard

Speed details

• bench.sql: 117 lines, 3,545 bytes
• bench_30x.sql: 3510 lines, 106,350 bytes

bench.sql (1×)

bench_30x.sql (30×)

Formatter details

Ground truth

40/40 statements validated by sqlite3.

Round-trip validation

For each formatter: does the formatted output produce identical EXPLAIN bytecode to the original? This verifies semantic preservation, not just syntactic validity. "CORRUPT" means the bytecode differs or EXPLAIN fails.

Scoreboard

Corruption details

Speed details

• bench.sql: 117 lines, 3,545 bytes
• bench_30x.sql: 3510 lines, 106,350 bytes

bench.sql (1×)

bench_30x.sql (30×)

Slow tools (single timed run)

Validator details

Diagnostic quality showcase

A realistic query with 2 subtle errors. How does each tool report them?

Query (CTE declares 3 columns but SELECT produces 2; typo ROUDN):

WITH
  monthly_stats(month, revenue, order_count) AS (
    SELECT
      STRFTIME('%Y-%m', o.created_at) AS month,
      SUM(o.total) AS revenue
    FROM orders o
    WHERE o.status = 'completed'
    GROUP BY STRFTIME('%Y-%m', o.created_at)
  )
SELECT
  ms.month,
  ms.revenue,
  ms.order_count,
  ROUDN(ms.revenue / ms.order_count, 2) AS avg_order
FROM monthly_stats ms
ORDER BY ms.month DESC
LIMIT 12;

syntaqlite

Static semantic analysis — offline, no database needed. Finds both errors in one pass:

error: table 'monthly_stats' has 2 values for 3 columns
  --> /var/folders/rx/t6_rqmqx0f15l7kgp7yjhcbc0000gn/T/tmpico33e5u.sql:30:3
   |
30 |   monthly_stats(month, revenue, order_count) AS (
   |   ^~~~~~~~~~~~~
warning: unknown function 'ROUDN'
  --> /var/folders/rx/t6_rqmqx0f15l7kgp7yjhcbc0000gn/T/tmpico33e5u.sql:42:3
   |
42 |   ROUDN(ms.revenue / ms.order_count, 2) AS avg_order
   |   ^~~~~
   = help: did you mean 'round'?

sqlite3

Runtime execution — stops at first error:

Error: in prepare, table monthly_stats has 2 values for 3 columns

sqlite-runner-lsp

Runtime via LSP — wraps sqlite3, same single error:

(no diagnostics)

sql-lint

Structural checks only:

/var/folders/rx/t6_rqmqx0f15l7kgp7yjhcbc0000gn/T/tmpxbgeg7p3.sql:1 sql-lint was unable to lint the following query "WITH...

Per-case error detection

Schema: users, orders, products, order_items. Ground truth: sqlite3.

Scoreboard

Speed details

• bench.sql: 117 lines, 3,545 bytes (+ schema preamble)
• bench_30x.sql: 3510 lines, 106,350 bytes (+ schema preamble)

bench.sql (1×)

bench_30x.sql (30×)

LSP details

Tested capabilities

Each server is started, sent a test file, and probed for completion, hover, diagnostics, and formatting via the LSP protocol.

¹ sqls requires a live database connection. Completion and hover results come from the connected database schema, not static analysis. Without a database, these features return no results.

Completion depth

Diagnostic detail

What each server reports for SELEC * FROM users; (syntax error):

syntaqlite

1:1 error syntax error near 'SELEC'

sqls

(no diagnostics)

sql-language-server

1:2 error Expected "$", "(", "--", "/*", "ALTER", "CREATE TABLE", "CREATE", "DELETE", "DROP TABLE", "DROP VIEW", "DROP", "INSERT", "REPLACE", "SELECT", "UPDATE", "WITH", "return", [ \t\n\r], or end of input but "S" found.

Speed details

Time to start server, send document, receive diagnostics, and exit: