fix(db): pool safety timeouts to stop login hangs (connection exhaustion)
Production outage: logins (and eventually all requests) hung forever. Root cause: the pg Pool was max:10 with NO connectionTimeoutMillis, so once all 10 connections were busy/stuck, every new query waited indefinitely for a free client — the "login spinner that never ends". The reject path stayed fast (one brief query); the success path needs several writes and blocked on acquiring a connection. Server-side only (reproduced in a desktop browser). Add fail-fast + self-healing pool config (all env-tunable): - connectionTimeoutMillis (10s): callers error instead of hanging on a saturated pool. - statement_timeout (60s): kill a runaway query so it can't pin a connection. - idle_in_transaction_session_timeout (60s): reclaim a client left idle in a hung transaction (the classic slow pool leak). - max 10 -> 20 for headroom; pool 'error' handler so a broken idle client can't crash the process. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
4baace83bc
commit
0129489746
1 file changed
+21
-1
+21
-1
@@ -10,7 +10,27 @@ const path = require('path');
|
|||||||
// to match the SQLite backend. Otherwise `expires_at < Date.now()` would compare a string to a number.
|
// to match the SQLite backend. Otherwise `expires_at < Date.now()` would compare a string to a number.
|
||||||
types.setTypeParser(20, (v) => (v === null ? null : parseInt(v, 10)));
|
types.setTypeParser(20, (v) => (v === null ? null : parseInt(v, 10)));
|
||||||
|
|
||||||
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 10 });
|
// Pool sizing + SAFETY TIMEOUTS. The old `max:10` with no connectionTimeoutMillis meant that once all 10
|
||||||
|
// connections were in use (a leaked/stuck client, a long query, or just a login burst) EVERY new query waited
|
||||||
|
// FOREVER for a free client — the "login spinner that never ends" outage. Now:
|
||||||
|
// • connectionTimeoutMillis — a saturated pool makes callers ERROR fast instead of hanging indefinitely.
|
||||||
|
// • statement_timeout — a runaway query is killed, so it can't pin a connection permanently.
|
||||||
|
// • idle_in_transaction_session_timeout — reclaims a client left "idle in transaction" (a hung tx()),
|
||||||
|
// which is the classic way a pool slowly leaks to zero.
|
||||||
|
// • max raised to 20 for more headroom.
|
||||||
|
// All tunable via env so they can be adjusted without a code change.
|
||||||
|
const pool = new Pool({
|
||||||
|
connectionString: process.env.DATABASE_URL,
|
||||||
|
max: Number(process.env.PG_POOL_MAX || 20),
|
||||||
|
connectionTimeoutMillis: Number(process.env.PG_CONN_TIMEOUT_MS || 10000),
|
||||||
|
idleTimeoutMillis: Number(process.env.PG_IDLE_TIMEOUT_MS || 30000),
|
||||||
|
statement_timeout: Number(process.env.PG_STATEMENT_TIMEOUT_MS || 60000),
|
||||||
|
idle_in_transaction_session_timeout: Number(process.env.PG_IDLE_TX_TIMEOUT_MS || 60000),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Don't let an unexpected idle-client error crash the process (pg emits 'error' on broken idle connections);
|
||||||
|
// the pool evicts the bad client and the next query gets a fresh one.
|
||||||
|
pool.on('error', (e) => { try { console.warn('[pg] idle client error:', e && e.message); } catch (_) {} });
|
||||||
|
|
||||||
// Repos use '?' placeholders (SQLite style); Postgres wants $1,$2,… — replace positionally. Safe because
|
// Repos use '?' placeholders (SQLite style); Postgres wants $1,$2,… — replace positionally. Safe because
|
||||||
// no literal '?' appears inside any SQL string literal in this codebase.
|
// no literal '?' appears inside any SQL string literal in this codebase.
|
||||||
|
|||||||
Reference in new issue
Block a user