Saga State & Compensation
The deep mechanics behind the saga — where the orchestrator keeps state so it survives a crash, how compensation unwinds committed steps in reverse, and how to thread one step's output to the next without coupling the services.
The Data chapter drew the happy path of a saga. This appendix is the three hard parts that picture glosses over: where the state lives, how compensation unwinds a failure, and how step B gets data that only step A produced — without coupling B to A.
Where the saga keeps its state
The orchestrator must survive a crash mid-flow, so in-memory state is a
non-starter — a pod restart would strand every in-flight saga with no idea what to
compensate. The workhorse option is a DB-backed state machine: one row per saga
instance holding its status, a step_index, and a context blob, with you
owning the transitions. Crucially, the state update and the next command commit
together in one transaction — the same outbox discipline from the Data chapter,
applied to the saga’s own progress.
// SagaOrchestrator.java — DB-backed state machine; advance one step per call
private static final List<String> STEPS =
List.of("charge_payment", "reserve_stock", "book_shipping");
@Transactional // state + command commit together
public void advance(String sagaId) {
Saga s = repo.findForUpdate(sagaId); // row lock (SELECT … FOR UPDATE)
if (!"RUNNING".equals(s.getStatus())) return; // idempotent: already done/failed
String step = STEPS.get(s.getStepIndex());
var result = invoke(step, s.getContext()); // call the owning service
s.getContext().put(step, result); // accumulate output into context
s.setStepIndex(s.getStepIndex() + 1);
if (s.getStepIndex() == STEPS.size()) s.setStatus("COMPLETED");
}
// SagaOrchestrator.java — Quarkus; DB-backed state machine
private static final List<String> STEPS =
List.of("charge_payment", "reserve_stock", "book_shipping");
@Transactional // state + command commit together
public void advance(String sagaId) {
Saga s = repo.findForUpdate(sagaId); // row lock
if (!"RUNNING".equals(s.status)) return; // idempotent: already done/failed
String step = STEPS.get(s.stepIndex);
var result = invoke(step, s.context); // call the owning service
s.context.put(step, result); // accumulate output into context
s.stepIndex++;
if (s.stepIndex == STEPS.size()) s.status = "COMPLETED";
}
// SagaOrchestrator.cs — DB-backed state machine; advance one step per call
private static readonly string[] Steps =
["charge_payment", "reserve_stock", "book_shipping"];
public async Task AdvanceAsync(string sagaId)
{
await using var tx = await db.BeginTransactionAsync();
var s = await db.GetSagaForUpdateAsync(sagaId); // row lock
if (s.Status != "RUNNING") return; // idempotent
var step = Steps[s.StepIndex];
var result = await InvokeAsync(step, s.Context); // call the owning service
s.Context[step] = result; // accumulate output
s.StepIndex++;
if (s.StepIndex == Steps.Length) s.Status = "COMPLETED";
await tx.CommitAsync(); // state + command commit together
}
STEPS = ["charge_payment", "reserve_stock", "book_shipping"]
async def advance(saga_id: str):
async with db.transaction(): # state + command commit together
s = await db.get_saga(saga_id, for_update=True) # row lock
if s.status != "RUNNING":
return # idempotent: already done/failed
step = STEPS[s.step_index]
result = await invoke(step, s.context) # call the owning service
s.context[step] = result # accumulate output into context
s.step_index += 1
if s.step_index == len(STEPS):
s.status = "COMPLETED"
// saga_orchestrator.cc — DB-backed state machine; advance one step per call
constexpr std::array steps = {"charge_payment", "reserve_stock", "book_shipping"};
void advance(PGconn* conn, std::string_view saga_id) {
PQclear(PQexec(conn, "BEGIN")); // libpq: state + command commit together
auto s = get_saga_for_update(conn, saga_id); // row lock (SELECT … FOR UPDATE)
if (s.status != "RUNNING") return; // idempotent: already done/failed
auto step = steps[s.step_index];
auto result = invoke(step, s.context); // call the owning service
s.context[step] = result; // accumulate output into context
++s.step_index;
if (s.step_index == steps.size()) s.status = "COMPLETED";
update_saga(conn, s);
PQclear(PQexec(conn, "COMMIT"));
}
// saga_orchestrator.go — DB-backed state machine; advance one step per call
var steps = []string{"charge_payment", "reserve_stock", "book_shipping"}
func advance(ctx context.Context, pool *pgxpool.Pool, sagaID string) error {
tx, _ := pool.Begin(ctx) // state + command commit together
defer tx.Rollback(ctx)
s, _ := getSagaForUpdate(ctx, tx, sagaID) // row lock
if s.Status != "RUNNING" {
return nil // idempotent: already done/failed
}
step := steps[s.StepIndex]
result, _ := invoke(ctx, step, s.Context) // call the owning service
s.Context[step] = result // accumulate output into context
s.StepIndex++
if s.StepIndex == len(steps) {
s.Status = "COMPLETED"
}
updateSaga(ctx, tx, s)
return tx.Commit(ctx)
}
Because advance is idempotent and the row is locked, resuming after a crash is
trivial: on startup, scan for sagas still RUNNING and call advance again. It
picks up exactly where it left off.
Compensation unwinds a failed saga
There is no cross-service rollback — when book_shipping fails, the charge_payment
that already committed in the payment service is real money. So compensation
runs a defined semantic inverse for each step that completed — charge → refund,
reserve → release — in reverse order, and only for the steps that actually
finished.
static final Map<String,String> COMPENSATIONS = Map.of( // inverse per step
"charge_payment", "refund_payment",
"reserve_stock", "release_stock",
"book_shipping", "cancel_shipping");
@Transactional
public void compensate(String sagaId) {
Saga s = em.find(Saga.class, sagaId, LockModeType.PESSIMISTIC_WRITE);
// undo only the steps that actually completed, newest first
for (String step : reversed(STEPS.subList(0, s.getStepIndex()))) {
invoke(COMPENSATIONS.get(step), s.getContext()); // semantic inverse
}
s.setStatus("COMPENSATED");
}
static final Map<String,String> COMPENSATIONS = Map.of( // inverse per step
"charge_payment", "refund_payment",
"reserve_stock", "release_stock",
"book_shipping", "cancel_shipping");
@Transactional
public void compensate(String sagaId) {
Saga s = Saga.findById(sagaId, LockModeType.PESSIMISTIC_WRITE);
for (String step : reversed(STEPS.subList(0, s.stepIndex))) { // newest first
invoke(COMPENSATIONS.get(step), s.context); // semantic inverse
}
s.status = "COMPENSATED";
}
static readonly Dictionary<string,string> Compensations = new() // inverse per step
{
["charge_payment"] = "refund_payment",
["reserve_stock"] = "release_stock",
["book_shipping"] = "cancel_shipping",
};
public async Task CompensateAsync(string sagaId, CancellationToken ct)
{
await using var tx = await db.Database.BeginTransactionAsync(ct);
var s = await db.Sagas.FromSqlInterpolated(
$"SELECT * FROM sagas WHERE id={sagaId} FOR UPDATE").SingleAsync(ct);
// undo only completed steps, newest first
foreach (var step in s.CompletedSteps().Reverse())
await Invoke(Compensations[step], s.Context, ct); // semantic inverse
s.Status = "COMPENSATED";
await tx.CommitAsync(ct);
}
COMPENSATIONS = { # inverse for each forward step
"charge_payment": "refund_payment",
"reserve_stock": "release_stock",
"book_shipping": "cancel_shipping",
}
async def compensate(saga_id: str):
async with db.transaction():
s = await db.get_saga(saga_id, for_update=True)
# undo only the steps that actually completed, newest first
for step in reversed(STEPS[: s.step_index]):
await invoke(COMPENSATIONS[step], s.context) # semantic inverse
s.status = "COMPENSATED"
// Compensations are a separate boost::sml branch in OrderSaga's table:
// "comp_stock"_s + event<StockReleased> = "comp_pay"_s,
// "comp_pay"_s + event<PaymentRefunded> = X, // done
// The same advance() loop drives forward steps and compensations;
// invoke_next() dispatches on the current state name, newest-first.
Task<> compensate(sml::sm<OrderSaga>& sm, Saga& s) {
for (auto& step : reversed(s.completed_steps())) // newest first
co_await invoke(inverse_of(step), s.context); // semantic inverse
s.status = "COMPENSATED";
}
// compensations — the semantic inverse for each forward step
var compensations = map[string]string{
"charge_payment": "refund_payment",
"reserve_stock": "release_stock",
"book_shipping": "cancel_shipping",
}
func compensate(ctx context.Context, pool *pgxpool.Pool, sagaID string) error {
tx, err := pool.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx)
s, err := getSagaForUpdate(ctx, tx, sagaID) // row lock
if err != nil {
return err
}
// undo only the steps that actually completed, newest first
done := steps[:s.StepIndex]
for i := len(done) - 1; i >= 0; i-- {
if err := invoke(ctx, compensations[done[i]], s.Context); err != nil { // inverse
return err
}
}
s.Status = "COMPENSATED"
return tx.Commit(ctx)
}
Compensations themselves can fail, so each must be idempotent and retryable — a refund that times out gets retried until it sticks, and a saga stuck compensating is an alert, not a silent loss.
When step B needs data only step A produced
A common source of bad designs: the orchestrator calls A, then B — but B needs a
value only A produced (a reservation_id, a warehouse region). The tempting wrong
answer is to have B call A directly — which recreates the exact service-to-service
coupling the architecture exists to avoid, and hides a dependency the orchestrator
can’t see.
The right answer is already in the code: A’s result goes into the saga’s
context blob (s.context[step] = result), and the orchestrator passes that
context into B. The data flows through the orchestrator, the only component that
legitimately knows the whole flow. B stays unaware of A, and because the context is
persisted with the saga state, it survives a crash too — resume rehydrates it for
free.
Cross-check it yourself
Force the unhappy path. Run the saga with book_shipping rigged to fail, and
confirm the database shows release_stock then refund_payment executing in that
order — and that cancel_shipping does not run, because shipping never completed.
Then kill the orchestrator pod mid-flow and restart it: the RUNNING saga should
resume from its persisted step_index and context, not start over. Reverse-order
inverses over only the completed steps, plus a clean resume, is the saga actually
working.
Verification status: verified — examples/17-sagas/ passes 11/11 checks in each of the six languages on Docker Engine (2026-10-10)
(orchestrated saga happy path, compensation in reverse order on failure, skipped compensation for steps that never completed, resume of a RUNNING saga after an orchestrator restart).