An organizer has 50 credits. They open two tabs and send 40 invitations' worth from each at the same moment. Both requests succeed. The balance now says 10, and 80 credits of paid features went out for the price of 50.
Nobody did anything wrong, and the code that allowed it looks perfectly reasonable. It reads the balance, checks there's enough, and writes the new balance. Each step is correct on its own. Run two of them at the same time and the business has given away a feature it charges for, and the ledger no longer agrees with itself.
At HOFWAMS, organizers spend credits on platform features: guest invitations, reminders, the parts of an event that cost us money to send. Spends don't only come from someone clicking a button. A reminder job, a batch of invitations and an organizer in two tabs can all spend from the same balance within the same few milliseconds. This post is how the ledger makes overspending impossible, not unlikely, and why the fix is smaller than the problem.
The code below is simplified from HOFWAMS, which uses NestJS with Prisma on Postgres: names shortened, error handling trimmed, the shape kept.
The bug hiding in obvious code
Here is the version almost everyone writes first:
const { available } = await db.creditBalance.findUnique({ where: { userId } });
if (available < amount) throw new InsufficientCredits();
await db.creditBalance.update({
where: { userId },
data: { available: available - amount },
});The problem is the gap between the read on the first line and the write on the third. It's only a few milliseconds, but a second request can read the balance inside it.
spend.ts (the obvious version)
const { available } = await db.creditBalance.findUnique({ where: { userId } });if (available < amount) throw new InsufficientCredits();await db.creditBalance.update({ where: { userId }, data: { available: available - amount },});
Two requests each want to spend 40 credits. The balance is 50.
Read all 5 steps
- Two requests each want to spend 40 credits. The balance is 50.
- Request A reads the balance: 50.
- Before A writes anything, request B reads it too. Still 50.
- Both checks pass, because both saw 50.
- Both write 50 minus 40. The balance says 10, and 80 credits were spent.
This has a name: a lost update. Two transactions read the same value, each computes a new one from it, and the second write silently overwrites the first. Neither request saw an error, because from where each one stood, nothing went wrong.
Why the usual fixes fall short
Most first attempts at a fix move the problem rather than remove it.
Approach | What it does | Why it isn't enough on its own |
|---|---|---|
A stricter check in code | Re-reads or double-checks before writing | The gap between read and write is still there, just smaller |
A lock in application memory | A mutex per user around the spend | Only works inside one process. A second server, or a background worker, has its own memory |
| Locks the balance row until the transaction ends | Works, but every code path that spends has to remember to lock first, and it costs an extra round trip |
| Postgres aborts one of two conflicting transactions | Works, but the loser fails with a serialization error that has to be caught and retried |
A conditional write | The check is part of the | Works, in one statement. This is what HOFWAMS does |
The first two don't hold up once there's more than one server. The next two are real solutions, used in plenty of good systems. The last one is simpler than both for a single balance, and the rest of this post is why.
Put the check inside the write
Instead of reading the balance and deciding in code, the spend is a single update that only succeeds while the balance covers the amount:
const { count } = await tx.creditBalance.updateMany({
where: {
userId,
availableCredits: { gte: amount },
},
data: {
availableCredits: { decrement: amount },
usedCredits: { increment: amount },
},
});
if (count === 0) throw new InsufficientCreditsException(); // 402 Payment RequiredIn SQL that's one statement, with the check in the WHERE clause:
UPDATE credit_balances
SET available_credits = available_credits - 40,
used_credits = used_credits + 40
WHERE user_id = $1
AND available_credits >= 40;Two things make this safe, and they're worth understanding rather than taking on trust.
First, the new value is computed by the database from the row as it is at the moment of writing, available_credits - 40, not from a number the application read earlier. There's no stale value to write back.
Second, Postgres handles two such updates to the same row in a specific way. Under its default isolation level, Read Committed, the second update waits for the first to commit, then re-checks its WHERE clause against the updated row before applying. HOFWAMS doesn't set an isolation level, so this default is what it relies on. With a balance of 50, the first spend of 40 commits and leaves 10. The second re-checks 10 >= 40, matches no row, and changes nothing.
credit-ledger.service.ts (simplified)
const { count } = await tx.credits.updateMany({ where: { userId, available: { gte: amount } }, data: { available: { decrement: amount } },});if (count === 0) throw new PaymentRequired(); // 402
Two tabs spend 40 credits at the same moment.
Read all 5 steps
- Two tabs spend 40 credits at the same moment.
- Both reach the ledger together.
- Each is one conditional write: spend only if 40 is still available.
- The first write wins. The balance drops to 10.
- The second matches no row, so it gets a 402. Nothing overspends.
So "did it work?" is answered by how many rows the update touched. One row means the credits were spent. Zero means the balance wasn't there at the moment of writing, and the API answers 402 Payment Required.
Zero rows only means "not enough credits" because the row is known to exist. A WHERE that matches nothing can also mean there's no row to match. So before every spend, the ledger makes sure the account's balance row exists, creating an empty one if needed, and only then runs the conditional update.
The status code is a deliberate convention, not a standard meaning: HTTP defines 402 only as "reserved for future use". HOFWAMS uses it for "not enough credits" so the app can send the organizer straight to buy more, and the response names its error and carries the shortfall. Clients branch on that error name, not on 402 alone, because another feature returns a 402 of its own.
There's one subtlety the code is careful about. Zero rows can mean "never had enough" or "lost a race with another spend". The error records which one it looked like, by re-reading the balance afterwards, but nothing is allowed to decide whether to retry from that: by the time the error is built, the balance may have changed again. The database's answer at the moment of writing is the only one that counts.
One writer, one transaction
A conditional write only protects the balance if nothing else writes to it. In HOFWAMS, one service, the credit ledger, is the only code path that ever changes the available balance. Features ask it to spend; they never touch the balance row themselves.
That rule is an application convention, written in the billing module's documentation. Postgres doesn't know about it. A new feature, a maintenance script or a hurried fix could still issue an unconditional update, and nothing in the database would stop it. The next section is about what would.
Every spend also writes two more rows, in the same transaction as the debit:
- a ledger row: the amount, the balance before and after, a description, and an idempotency key;
- a usage row: which feature spent the credits, and how many.
Because the three writes commit together or not at all, the ledger never shows a spend that didn't happen, and the balance never moves without a ledger row explaining why.
The before and after balances on the ledger row are taken from the database, not from anything the application read earlier. After its conditional update, the transaction reads the balance back. Its own update still holds the row's lock, so no other transaction can change the balance between the two, and the read shows exactly the result of this spend. The balance before is that figure plus the amount spent. In plain SQL, UPDATE ... RETURNING gives the same number in one statement.
Retries are narrow on purpose. If Postgres reports a write conflict or a deadlock, the whole transaction is retried from the start in a new transaction, up to three attempts, a second apart. A clash on an idempotency key is never retried. It means the same operation is already recorded, so the code reads that winning row and returns its result.
Decisions in the write, invariants in the schema
HOFWAMS deliberately has no CHECK (available_credits >= 0) constraint on the balance table. The billing module's documentation argues that the conditional updates are the sole authority on solvency, so adding the constraint "would only be redundant, not a correctness gap".
There's a stronger case for having both, because they do different jobs. The conditional update makes a business decision: this spend is or isn't allowed, and the caller gets a clean 402 when it isn't. The constraint protects an invariant: no balance is ever negative, whichever code path, script or future feature tries to write it. The first only holds while every writer remembers it. The second holds regardless.
ALTER TABLE credit_balances
ADD CONSTRAINT available_credits_nonnegative CHECK (available_credits >= 0);The constraint doesn't replace the conditional update. Without the condition, an overspend would surface as a database error instead of a 402 with the shortfall in it. With both, the normal path gives the clean answer, and the constraint is there for the day someone forgets. Database permissions can go further, so that the application's ordinary role can't update the balance column directly at all.
The general rule: use conditional writes to enforce business decisions, and database constraints to protect invariants.
Grants: the same money, arriving twice
Credits come in as well as going out: a welcome grant when someone signs up, a purchase confirmed by a Stripe webhook. Here the danger isn't two spends at once but the same grant arriving twice. Stripe, like most payment providers, delivers webhooks at least once, so the same "payment succeeded" event can turn up again.
Every grant carries an idempotency key: a value that identifies the operation, so doing it twice has the same effect as doing it once. The important word is derived. The key is built from what the grant is for, never generated at random:
const welcomeKey = `welcome:${userId}`;
const purchaseKey = `credit-purchase:${purchaseId}`;A retry, a redelivered webhook or a backfill script all compute the same key for the same grant. The key is stored on the ledger row under a unique constraint, so a second insert with the same key fails, and the code returns the first grant's result instead of granting again. The same purchase key is also sent to Stripe as its idempotency key when the checkout is created, so Stripe deduplicates on its side too. That's a short-term guard, not a record: Stripe can remove keys once they're 24 hours old. The unique constraint in HOFWAMS's own database is what keeps the protection permanent.
Purchases get one more guard, and it's the same idea as the spend. A purchase starts PENDING. The webhook handler claims it with a conditional update, and grants the credits in the same transaction:
const { count } = await tx.creditPurchase.updateMany({
where: { id: purchase.id, paymentStatus: "PENDING" },
data: { paymentStatus: "SUCCEEDED", completedAt: new Date() },
});
if (count === 0) return; // another delivery already claimed it
await ledger.grantWithin({ tx, userId, amount, idempotencyKey: purchaseKey });Only one delivery can move a purchase out of PENDING. If anything fails after the claim, the whole transaction rolls back, the purchase is PENDING again, and Stripe's next delivery picks it up.
Corrections add rows
When credits need to go back, for example when a batch of invitations is cancelled before it's sent, nothing in the ledger is edited. A refund is a new row that restores the credits, with its own derived key, so the same refund can't be applied twice.
It's worth being precise about how far that goes. The refund's own update is conditional too, but on the account as a whole: it only applies while the account's recorded usage covers the amount, and one that doesn't is refused with a 409. That stops a refund inventing credits out of nothing. It doesn't, on its own, stop two different refunds for the same spend from adding up to more than that spend, because the account's other spending can cover them.
In HOFWAMS, one refund per spend comes from how refunds are created rather than from the ledger. Each refund key is derived from the spend it reverses, and the amount is read back from that spend's ledger row, so the paths that refund a whole spend can't do it twice. Where a spend can be refunded in parts, each part is capped at its own share. The stronger design puts the bound in the database: a refunded-amount counter on the spend, raised by a conditional update that only succeeds while the total stays within the original.
The original spend stays exactly as it was. The ledger is a history of what happened, and the balance is what that history adds up to.
Prove it against a real database
A concurrency bug you haven't reproduced is one you haven't fixed, and this one can't be reproduced with a mocked database client. Only a real Postgres re-checks available_credits >= n after a concurrent update commits. So the ledger's concurrency tests run against Postgres:
it("drains a balance to exactly zero, never below it", async () => {
const userId = await seedUserWithBalance(500);
const spends = Array.from({ length: 10 }, (_, i) =>
ledger.spend({ userId, product: "EVENT_REMINDER", quantity: 100, idempotencyKey: `reminder:${userId}:${i}` }),
);
const outcome = await raceSpends(spends); // Promise.all, counting successes and 402s
expect(outcome).toEqual({ succeeded: 5, insufficient: 5, otherErrors: [] });
expect(await availableCredits(userId)).toBe(0);
});Ten spends of 100 credits fire at a balance of 500 at once. Exactly five succeed, exactly five get a 402, and the balance ends at zero. The same file covers two 900-credit spends against 1,000 (one succeeds, 100 left), a spend of exactly the balance, a key replayed three times at once (one row), and refund bounds. The refund tests run one after another, though, not at once; a concurrent refund test is the next one worth adding.
The same shape at the door
Once you see the pattern, it turns up everywhere a check and a write are separate steps. At events, volunteers check guests in by scanning a QR code. Two scanners reading the same ticket at once could both check the guest in, so a ticket could be used twice.
registration-check-in.service.ts (simplified)
const { count } = await db.registration.updateMany({ where: { id, checkedInAt: null }, data: { checkedInAt: new Date() },});if (count === 0) return { status: "already_checked_in" };
Two volunteers scan the same ticket at once.
Read all 5 steps
- Two volunteers scan the same ticket at once.
- Both requests arrive together.
- The update only matches while the ticket is still unchecked.
- One scan flips it.
- The other gets a clear answer instead of a second check-in.
The fix is the same shape. The update only matches while the ticket is still unchecked:
const { count } = await prisma.registration.updateMany({
where: { id: registrationId, checkedInAt: null, status: { in: ["CONFIRMED", "PENDING"] } },
data: { checkedInAt: new Date() },
});
if (count === 0) return { registration, alreadyCheckedIn: true };The scan that loses gets a clear "already checked in" answer instead of a second check-in. A unit test covers that branch by simulating a scan that loses the write; unlike the ledger, it doesn't race real requests against Postgres. Since the whole point is that the database settles the race, an integration test that fires two scans at once would prove it the same way the ledger test does.
Where this stops working
A conditional write is the right tool when the rule is about one row: this balance, this ticket, this purchase. It has limits worth knowing before you reach for it everywhere.
Situation | What to use instead |
|---|---|
The rule spans several rows, like "the team's total spend stays under its budget" | Lock the rows involved with |
You need to know why a write failed, reliably | Accept that zero rows is the only reliable answer, and design the error around it |
Other code also writes the same column | Fix that first. The guarantee only holds with a single writer |
The check needs data from another service | The database can't check what it can't see; this needs a different design, such as a reservation |
What this guarantees
Within those limits, the guarantee is specific. While every write to a balance goes through the ledger's conditional update, two spends can never take an account below zero, however they interleave; the ledger records exactly the balance changes that happened; and replaying any grant, purchase or refund with the same key has no further effect. What it doesn't guarantee on its own is that every writer follows the rule, which is the job of the constraint above, or that separate refunds of one spend stay within it, which today rests on how refunds are created.
The rule
Every fix in this post is the same move: take a check that happened in application code a few milliseconds before a write, and make it part of the write. The database is the only party that sees every request at the moment it lands, so it's the only one that can enforce the rule without a gap.
If an invariant matters, make the database enforce it at the moment of writing, not your code a few milliseconds earlier.
Further reading
- PostgreSQL: transaction isolation and Read Committed, explicit locking and
FOR UPDATE, unique constraints - Prisma:
updateManyand transactions - Stripe: handling duplicate webhook events and idempotent requests