Saga State & Compensation

The deep mechanics behind the saga — where the orchestrator keeps state so it survives a crash, how compensation unwinds committed steps in reverse, and how to thread one step's output to the next without coupling the services.

⏱ 20 minutesAppendix DDeep-dive appendices

The Data chapter drew the happy path of a saga. This appendix is the three hard parts that picture glosses over: where the state lives, how compensation unwinds a failure, and how step B gets data that only step A produced — without coupling B to A.

Where the saga keeps its state

The orchestrator must survive a crash mid-flow, so in-memory state is a non-starter — a pod restart would strand every in-flight saga with no idea what to compensate. The workhorse option is a DB-backed state machine: one row per saga instance holding its status, a step_index, and a context blob, with you owning the transitions. Crucially, the state update and the next command commit together in one transaction — the same outbox discipline from the Data chapter, applied to the saga’s own progress.

Four ways a saga can keep its state. In-memory: a dict or object only, lost on pod restart, never for real sagas. DB state machine: one row per saga instance with status and context JSON, and you own the transitions. Event-sourced: append step events and rebuild by replay, with full audit history. Workflow engine: Temporal or Camunda, durable execution with retries and timers built in. Whatever the store, the state transition and the outbound command must commit together.
Figure D.1 — Where saga state can live; the DB-backed state machine is the workhorse, but the commit-together rule is the same for all
// SagaOrchestrator.java — DB-backed state machine; advance one step per call
private static final List<String> STEPS =
    List.of("charge_payment", "reserve_stock", "book_shipping");

@Transactional                                   // state + command commit together
public void advance(String sagaId) {
    Saga s = repo.findForUpdate(sagaId);         // row lock (SELECT … FOR UPDATE)
    if (!"RUNNING".equals(s.getStatus())) return; // idempotent: already done/failed
    String step = STEPS.get(s.getStepIndex());
    var result = invoke(step, s.getContext());    // call the owning service
    s.getContext().put(step, result);             // accumulate output into context
    s.setStepIndex(s.getStepIndex() + 1);
    if (s.getStepIndex() == STEPS.size()) s.setStatus("COMPLETED");
}
// SagaOrchestrator.java — Quarkus; DB-backed state machine
private static final List<String> STEPS =
    List.of("charge_payment", "reserve_stock", "book_shipping");

@Transactional                                   // state + command commit together
public void advance(String sagaId) {
    Saga s = repo.findForUpdate(sagaId);         // row lock
    if (!"RUNNING".equals(s.status)) return;     // idempotent: already done/failed
    String step = STEPS.get(s.stepIndex);
    var result = invoke(step, s.context);        // call the owning service
    s.context.put(step, result);                 // accumulate output into context
    s.stepIndex++;
    if (s.stepIndex == STEPS.size()) s.status = "COMPLETED";
}
// SagaOrchestrator.cs — DB-backed state machine; advance one step per call
private static readonly string[] Steps =
    ["charge_payment", "reserve_stock", "book_shipping"];

public async Task AdvanceAsync(string sagaId)
{
    await using var tx = await db.BeginTransactionAsync();
    var s = await db.GetSagaForUpdateAsync(sagaId);   // row lock
    if (s.Status != "RUNNING") return;                // idempotent
    var step = Steps[s.StepIndex];
    var result = await InvokeAsync(step, s.Context);  // call the owning service
    s.Context[step] = result;                         // accumulate output
    s.StepIndex++;
    if (s.StepIndex == Steps.Length) s.Status = "COMPLETED";
    await tx.CommitAsync();                           // state + command commit together
}
STEPS = ["charge_payment", "reserve_stock", "book_shipping"]

async def advance(saga_id: str):
    async with db.transaction():                 # state + command commit together
        s = await db.get_saga(saga_id, for_update=True)   # row lock
        if s.status != "RUNNING":
            return                               # idempotent: already done/failed
        step = STEPS[s.step_index]
        result = await invoke(step, s.context)   # call the owning service
        s.context[step] = result                 # accumulate output into context
        s.step_index += 1
        if s.step_index == len(STEPS):
            s.status = "COMPLETED"
// saga_orchestrator.cc — DB-backed state machine; advance one step per call
constexpr std::array steps = {"charge_payment", "reserve_stock", "book_shipping"};

void advance(PGconn* conn, std::string_view saga_id) {
  PQclear(PQexec(conn, "BEGIN"));                // libpq: state + command commit together
  auto s = get_saga_for_update(conn, saga_id);   // row lock (SELECT … FOR UPDATE)
  if (s.status != "RUNNING") return;             // idempotent: already done/failed
  auto step = steps[s.step_index];
  auto result = invoke(step, s.context);         // call the owning service
  s.context[step] = result;                      // accumulate output into context
  ++s.step_index;
  if (s.step_index == steps.size()) s.status = "COMPLETED";
  update_saga(conn, s);
  PQclear(PQexec(conn, "COMMIT"));
}
// saga_orchestrator.go — DB-backed state machine; advance one step per call
var steps = []string{"charge_payment", "reserve_stock", "book_shipping"}

func advance(ctx context.Context, pool *pgxpool.Pool, sagaID string) error {
	tx, _ := pool.Begin(ctx)                     // state + command commit together
	defer tx.Rollback(ctx)
	s, _ := getSagaForUpdate(ctx, tx, sagaID)    // row lock
	if s.Status != "RUNNING" {
		return nil                               // idempotent: already done/failed
	}
	step := steps[s.StepIndex]
	result, _ := invoke(ctx, step, s.Context)    // call the owning service
	s.Context[step] = result                     // accumulate output into context
	s.StepIndex++
	if s.StepIndex == len(steps) {
		s.Status = "COMPLETED"
	}
	updateSaga(ctx, tx, s)
	return tx.Commit(ctx)
}

Because advance is idempotent and the row is locked, resuming after a crash is trivial: on startup, scan for sagas still RUNNING and call advance again. It picks up exactly where it left off.

Compensation unwinds a failed saga

There is no cross-service rollback — when book_shipping fails, the charge_payment that already committed in the payment service is real money. So compensation runs a defined semantic inverse for each step that completed — charge → refund, reserve → release — in reverse order, and only for the steps that actually finished.

Forward path: charge_payment then reserve_stock then book_shipping, which fails. On failure the saga compensates in reverse: release_stock then refund_payment — only the steps that completed, newest first
Figure D.2 — On failure, run each completed step's inverse, newest first
static final Map<String,String> COMPENSATIONS = Map.of(   // inverse per step
    "charge_payment", "refund_payment",
    "reserve_stock",  "release_stock",
    "book_shipping",  "cancel_shipping");

@Transactional
public void compensate(String sagaId) {
    Saga s = em.find(Saga.class, sagaId, LockModeType.PESSIMISTIC_WRITE);
    // undo only the steps that actually completed, newest first
    for (String step : reversed(STEPS.subList(0, s.getStepIndex()))) {
        invoke(COMPENSATIONS.get(step), s.getContext());  // semantic inverse
    }
    s.setStatus("COMPENSATED");
}
static final Map<String,String> COMPENSATIONS = Map.of(   // inverse per step
    "charge_payment", "refund_payment",
    "reserve_stock",  "release_stock",
    "book_shipping",  "cancel_shipping");

@Transactional
public void compensate(String sagaId) {
    Saga s = Saga.findById(sagaId, LockModeType.PESSIMISTIC_WRITE);
    for (String step : reversed(STEPS.subList(0, s.stepIndex))) {  // newest first
        invoke(COMPENSATIONS.get(step), s.context);               // semantic inverse
    }
    s.status = "COMPENSATED";
}
static readonly Dictionary<string,string> Compensations = new()  // inverse per step
{
    ["charge_payment"] = "refund_payment",
    ["reserve_stock"]  = "release_stock",
    ["book_shipping"]  = "cancel_shipping",
};

public async Task CompensateAsync(string sagaId, CancellationToken ct)
{
    await using var tx = await db.Database.BeginTransactionAsync(ct);
    var s = await db.Sagas.FromSqlInterpolated(
        $"SELECT * FROM sagas WHERE id={sagaId} FOR UPDATE").SingleAsync(ct);
    // undo only completed steps, newest first
    foreach (var step in s.CompletedSteps().Reverse())
        await Invoke(Compensations[step], s.Context, ct);          // semantic inverse
    s.Status = "COMPENSATED";
    await tx.CommitAsync(ct);
}
COMPENSATIONS = {                                # inverse for each forward step
    "charge_payment": "refund_payment",
    "reserve_stock":  "release_stock",
    "book_shipping":  "cancel_shipping",
}

async def compensate(saga_id: str):
    async with db.transaction():
        s = await db.get_saga(saga_id, for_update=True)
        # undo only the steps that actually completed, newest first
        for step in reversed(STEPS[: s.step_index]):
            await invoke(COMPENSATIONS[step], s.context)   # semantic inverse
        s.status = "COMPENSATED"
// Compensations are a separate boost::sml branch in OrderSaga's table:
//   "comp_stock"_s + event<StockReleased>   = "comp_pay"_s,
//   "comp_pay"_s   + event<PaymentRefunded> = X,          // done
// The same advance() loop drives forward steps and compensations;
// invoke_next() dispatches on the current state name, newest-first.
Task<> compensate(sml::sm<OrderSaga>& sm, Saga& s) {
  for (auto& step : reversed(s.completed_steps()))         // newest first
    co_await invoke(inverse_of(step), s.context);          // semantic inverse
  s.status = "COMPENSATED";
}
// compensations — the semantic inverse for each forward step
var compensations = map[string]string{
	"charge_payment": "refund_payment",
	"reserve_stock":  "release_stock",
	"book_shipping":  "cancel_shipping",
}

func compensate(ctx context.Context, pool *pgxpool.Pool, sagaID string) error {
	tx, err := pool.Begin(ctx)
	if err != nil {
		return err
	}
	defer tx.Rollback(ctx)

	s, err := getSagaForUpdate(ctx, tx, sagaID) // row lock
	if err != nil {
		return err
	}
	// undo only the steps that actually completed, newest first
	done := steps[:s.StepIndex]
	for i := len(done) - 1; i >= 0; i-- {
		if err := invoke(ctx, compensations[done[i]], s.Context); err != nil { // inverse
			return err
		}
	}
	s.Status = "COMPENSATED"
	return tx.Commit(ctx)
}

Compensations themselves can fail, so each must be idempotent and retryable — a refund that times out gets retried until it sticks, and a saga stuck compensating is an alert, not a silent loss.

When step B needs data only step A produced

A common source of bad designs: the orchestrator calls A, then B — but B needs a value only A produced (a reservation_id, a warehouse region). The tempting wrong answer is to have B call A directly — which recreates the exact service-to-service coupling the architecture exists to avoid, and hides a dependency the orchestrator can’t see.

The right answer is already in the code: A’s result goes into the saga’s context blob (s.context[step] = result), and the orchestrator passes that context into B. The data flows through the orchestrator, the only component that legitimately knows the whole flow. B stays unaware of A, and because the context is persisted with the saga state, it survives a crash too — resume rehydrates it for free.

The orchestrator owns the saga context. It calls service A, which returns a reservation_id and warehouse region; that output is persisted into the saga context (reservation_id r-77, region eu-1) written after A and read before B; the orchestrator then calls service B with the context, which supplies the reservation_id B needs. If the orchestrator crashes between persisting A's output and calling B, on restart it loads the saga context, sees A is done, and resumes at B without re-calling A.
Figure D.3 — A's output flows to B through the orchestrator's persisted context — B never calls A, and a crash resumes cleanly

Cross-check it yourself

Force the unhappy path. Run the saga with book_shipping rigged to fail, and confirm the database shows release_stock then refund_payment executing in that order — and that cancel_shipping does not run, because shipping never completed. Then kill the orchestrator pod mid-flow and restart it: the RUNNING saga should resume from its persisted step_index and context, not start over. Reverse-order inverses over only the completed steps, plus a clean resume, is the saga actually working.


Verification status: verified — examples/17-sagas/ passes 11/11 checks in each of the six languages on Docker Engine (2026-10-10) (orchestrated saga happy path, compensation in reverse order on failure, skipped compensation for steps that never completed, resume of a RUNNING saga after an orchestrator restart).