Warp SQL

Run SQL from anything that speaks HTTP. Get Arrow back.

Warp SQL is an open protocol for running SQL against a database server over plain HTTP. Query results, query parameters and bulk uploads travel as Apache Arrow IPC streams (application/vnd.apache.arrow.stream), so every Arrow library reads them directly. Control messages are small JSON documents. A browser fetch, curl or any language’s standard HTTP library is a complete client: there is no gRPC, no protobuf and no generated code.

It does for HTTP what Arrow Flight SQL does for gRPC: a small set of operations that any engine can implement and any HTTP client can call. Its metadata endpoints return Flight SQL’s result schemas, so driver code written for Flight SQL carries over.

Status

Version 1, specification 1.0.1. Version 1 is stable: it changes only by addition (new endpoints, fields, capabilities and error codes), so a client written for 1.0.0 keeps working with every later 1.x server. See Versioning and compatibility.

Why

Plain HTTP

Every operation works over HTTP/1.1 through an ordinary reverse proxy. Nothing relies on HTTP/2 streams, trailers or server push, and no request has to outlive a proxy timeout.

Arrow all the way

Results, Arrow query parameters and uploads are Arrow IPC streams, with optional lz4_frame or zstd buffer compression. No row-by-row conversion on either side.

Short queries stay short

One POST runs a query and streams its result. Long queries are submitted, long-polled and fetched later, and a dropped download resumes from the batch where it stopped.

Wide when it pays

A large result can be read as several partitions over separate connections at once, in order when the query has one. A large upload can be sent as several parts and committed as one transaction.

Security in the core

Credentials are bearer tokens, never cookies, so cross-site request forgery is impossible by construction. Every handle is bound to the identity that created it and checked on every use. Mutual TLS and browser sign-in are in the protocol.

Engine-neutral and open

Apache 2.0 licensed. Nothing in the protocol depends on one engine, and a black-box conformance suite tells an implementation when it is done.

A quick look

Sign in, open a session, run a query and read the Arrow result. These commands run against warp-duckdb, the reference server, listening on 127.0.0.1:31337 with user joe:

WARP=http://127.0.0.1:31337
TOKEN=$(curl -s $WARP/warp/v1/auth/token -H 'Content-Type: application/json' \
  -d '{"grant_type": "password", "username": "joe", "password": "s3:cret"}' | jq -r .access_token)
SESSION=$(curl -s $WARP/warp/v1/sessions -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{}' | jq -r .session_id)
curl -s $WARP/warp/v1/sessions/$SESSION/statements -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -H 'Accept: application/vnd.apache.arrow.stream' \
  -d '{"sql": "SELECT 42 AS answer, ? AS greeting", "parameters": ["hello"]}' -o result.arrows
python3 -c "import pyarrow.ipc as ipc; print(ipc.open_stream('result.arrows').read_all().to_pylist())"

[{'answer': 42, 'greeting': 'hello'}]

Every server describes itself at GET /warp (no credentials needed), so a client knows which optional features it may use:

curl -s http://127.0.0.1:31337/warp | jq .

{
  "protocol": "warp-sql",
  "protocol_versions": [
    "1"
  ],
  "server": {
    "name": "warp-duckdb",
    "version": "1.0.1 (DuckDB …)"
  },
  "capabilities": [
    "prepared_statements",
    "transactions",
    "ingest",
    "partitioned_ingest",
    "long_poll",
    "ipc_compression_zstd",
    "ipc_compression_lz4_frame",
    "result_partitions",
    "openapi"
  ],
  "auth": {
    "grant_types": [
      "password"
    ]
  },
  "links": {
    "openapi": "/warp/v1/openapi.json"
  }
}

How it works

A long query: submit, long-poll, fetch. A short one is a single POST that streams its result.

The flows walk through every operation with a sequence diagram and commands you can run.

Where to go

The specificationThe protocol's definition: every endpoint, header, document and error code. Version 1, specification 1.0.1.

OpenAPI referenceEvery JSON endpoint and schema as an OpenAPI 3.1 document, in Swagger UI. Servers also serve it at GET /warp/v1/openapi.json.

FlowsSign-in, sessions, queries, partitions, prepared statements, ingest, cancel, errors and shutdown, step by step.

ConformanceA black-box test suite any server can run, and what it does not test yet.

ImplementationsServers and clients that speak Warp SQL: ADBC (Go, Python), JDBC, JavaScript and C++.

EndpointsAll 38 endpoints of version 1 on one page, each with its section and OpenAPI operation.