Cutting Our P99 Latency in Half with Connection Pooling

Our API felt fine in the median and miserable at the tail. P50 was a healthy 22ms; P99 sat at 340ms. The culprit wasn't query time — it was connection setup.

The smoking gun

Every request grabbed a new connection, did a TLS handshake, authenticated, and then ran a 3ms query. Under load, Postgres spent more time on startup packets than on actual work.

// Before: a new client per request (don't do this)
async function handler(req: Request) {
  const client = new Client(DATABASE_URL);
  await client.connect();
  const rows = await client.query("SELECT 1");
  await client.end();
  return rows;
}

Adding a pool

We dropped in a pooler with a conservative max size and let it recycle connections:

const pool = new Pool({ connectionString: DATABASE_URL, max: 20, idleTimeoutMillis: 30_000 });
 
async function handler() {
  const { rows } = await pool.query("SELECT 1");
  return rows;
}

The pool size that matters is not "how many users" — it's "how many concurrent queries your database can actually serve well." We tuned max down, not up.

Results

P99 dropped to 160ms and CPU on the database fell 18%. The lesson: measure the tail, and never pay for a handshake you can amortize.