Lab L10 · Java + Go

Implementing a durable idempotent operation

Build a retry-safe add(scope, key, amount) on a counter stored in SQLite. The first call with a given (scope, key) increments the counter and stores the resulting value as that request's response, in the same transaction. A repeat with the same key and amount returns the stored response without incrementing again; the same key with a different amount is rejected as a conflict.

The task

A mobile app sends “top up my wallet by 50”. The server commits the credit, but the response is lost to a network drop, so the app retries. Without protection the wallet is credited twice. The fix: the client attaches an idempotency key (one UUID per user action), and the server records “key → response” atomically with the business change. A retry finds the record and replays the original answer. scope namespaces keys (e.g. per client), so two clients can reuse the same key safely.

The idempotency record and the business change commit or roll back together. That atomicity is what makes a retry after a lost response safe.

What you implement

public final class DurableCounter {
    public DurableCounter(Path sqliteFile) throws SQLException;        // creates tables if missing
    public long add(String scope, String key, long amount) throws SQLException;  // counter value after this request
}

business(id INTEGER PRIMARY KEY CHECK(id = 1), value INTEGER NOT NULL)        -- one row, starts at 0
requests(scope TEXT, key TEXT, amount INTEGER, response INTEGER, PRIMARY KEY(scope, key))

Rules

Empty scope or key, or amount <= 0 → IllegalArgumentException.
Inside one transaction (BEGIN IMMEDIATE takes the write lock up front): look up (scope, key). Found with the same amount → commit and return the stored response. Found with a different amount → roll back and throw IllegalArgumentException (“key conflict”).
Not found: read the counter, add with Math.addExact, update the counter, insert the request row with the new value as its response, commit, return it.
Any failure (overflow, SQL error, conflict) rolls back: neither table changes.
The same key in a different scope is a different request.
Durable: a new DurableCounter on the same file sees earlier counters and responses. Concurrent callers (threads or connections) apply each key exactly once.
The response is replayed exactly, even if the counter has since moved on.

Worked example

add(u1, k1, 50) → 50. Retry add(u1, k1, 50) → 50, counter unchanged.
add(u1, k2, 10) → 60. add(u2, k1, 5) → 65 (different scope).
add(u1, k1, 70) → IllegalArgumentException (same key, different amount).
add(u1, k1, 50) → still 50: the stored response, not the current counter.

Files and commands

Read the tests first, then replace each TODO in the starter until they pass. Java starter: DurableCounter.java · tests: DurableCounterTest.java. Go starter: counter.go · tests: counter_test.go.

cd java
./gradlew test --tests 'labs.idempotency.*'

cd go
go test -race -count=1 ./idempotency

Reveal the reference solutions

Java: DurableCounter.java
Go: counter.go

When to use

Use this lab to implement a repeatable local database mutation when a client may retry after losing the response. A request consists of a nonempty scope, nonempty idempotency key, and positive signed-64-bit integer amount. The operation increments one business counter and retains the resulting response under the scoped key. Repeating the same amount returns the original response; reusing the key with another amount is a conflict. Other operations may subsequently change the counter without changing that retained response.

The example uses a real SQLite file and transaction, not a concurrent map. Both languages share the table schema and integer contract. It demonstrates a single-host transactional boundary, not PostgreSQL isolation equivalence or exactly-once effects across external APIs. Do not put a remote payment call between the local statements and claim the transaction covers it.

Steps

Scope the key and payload. Use the pair of scope and key as a database primary key. Compare the exact integer amount rather than hashing an ambiguous serialized payload. A richer request needs a stable canonical representation and an explicit fingerprint/version policy. Scopes must be derived from authenticated tenant or operation identity by the caller.
Acquire the write transaction before checking. Start an immediate SQLite transaction. This serializes competing writers before the lookup, avoiding two callers both observing an absent key and applying the effect. A busy timeout is finite; lock timeout remains a caller-visible failure, not permission to invent a new key.
Mutate and retain atomically. Read the counter, check addition for signed overflow, update it, and insert the completed request response in the same transaction. Commit only after both writes succeed. Every failure path rolls back the transaction. No visible in-progress row is needed because the business work stays wholly inside this database transaction.
Reason about crash windows. Failure before commit leaves neither business change nor completed response. Failure after commit but before response delivery leaves both. A retry with the same key can retrieve the outcome. An ambiguous commit error requires outcome recovery with that key, not an assumption that nothing happened. Storage durability still depends on SQLite's documented filesystem and hardware assumptions.
Test concurrency and reopen. Run duplicates through two independently opened stores against one file and require every response to equal the first outcome. Reopen the file and retry the original request, then submit a different key to confirm the effect happened once. These tests cover connection-level contention and reopening, not an OS-kill or power-loss experiment.
Inject rollback failure. Install a test trigger that rejects the retained-response insert after the business update. Require the call to fail, then apply another operation and verify the counter did not include the failed amount. The trigger makes the mutation/response boundary testable without pretending a happy-path retry covers it.

Checklist

PostgreSQL adapter and recall

Keep the local Go and Java SQLite references for a quick exercise. The paired postgres packages add READ COMMITTED transactions using a locked singleton business row before key lookup, a unique scoped key, retained responses, and outbox insertion in the same commit. Shared SQL fixtures and integration tests cover duplicate connections, rollback hooks, subprocess termination before/after commit, and PostGIS predicates. Run the explicit integration commands in the examples README; unit success does not imply those database checks ran.

Rebuild the duplicate branch from memory. Variation: terminate the client after commit but before response. Acceptance: retry with the same key returns the stored response and does not apply the amount again. No automatic retention cleanup is supplied; indefinite retention preserves the stated retry contract at an unbounded storage cost.

Scope and key form a unique retained identity
Changed payloads conflict rather than replaying an unrelated response
Business effect and retained response commit or roll back together
Concurrent duplicates, restart/reopen, and injected rollback are tested
Retention and external-side-effect limitations are explicit

Signals

Working: duplicate callers receive the same retained response after reopening, and an injected insert failure leaves the business counter unchanged.

Going wrong: the key is checked outside the transaction, business state commits before the response row, or cleanup silently permits an old operation to execute again. This example retains records indefinitely and has no request-size quotas or expiry; production retention must match the promised retry horizon and growth budget.

Implementing Per-Key Request Coalescing, because sharing in-flight work is not restart-safe deduplication.