Transactional outbox pattern in .NET: publish without losing messages

Your order service saves an order and publishes OrderPlaced. Those two lines look like one business action:

await db.SaveChangesAsync(cancellationToken);
await bus.PublishAsync(orderPlaced, cancellationToken);

They are not one action. They are two writes to two systems with a crash-shaped gap between them. If the database commit succeeds and the process dies before publish, the order exists but inventory and fulfillment never hear about it. Reverse the lines and a published event can describe an order that the database failed to save.

No try/catch closes that gap. A retry can help, but after a timeout the service may not know which write succeeded. The transactional outbox pattern solves the atomicity problem by moving the first version of the message into the same database transaction as the business data.

One transaction, two rows

Instead of publishing directly, the request stores an outbox record:

public sealed class OutboxMessage
{
    public Guid Id { get; init; }
    public DateTimeOffset OccurredAt { get; init; }
    public required string Type { get; init; }
    public required string Payload { get; init; }
    public DateTimeOffset? ProcessedAt { get; set; }
    public int Attempts { get; set; }
    public string? LastError { get; set; }
}

The command adds the order and its event to one DbContext:

public async Task<Guid> PlaceOrderAsync(
    PlaceOrder command,
    CancellationToken cancellationToken)
{
    var order = Order.Place(command.CustomerId, command.Lines);
    var integrationEvent = new OrderPlaced(
        order.Id,
        order.CustomerId,
        order.Total);

    _db.Orders.Add(order);
    _db.OutboxMessages.Add(new OutboxMessage
    {
        Id = Guid.NewGuid(),
        OccurredAt = DateTimeOffset.UtcNow,
        Type = "orders.order-placed.v1",
        Payload = JsonSerializer.Serialize(integrationEvent)
    });

    await _db.SaveChangesAsync(cancellationToken);
    return order.Id;
}

For relational providers, SaveChanges uses a transaction for the batch. Either both rows commit or neither does. The request can now return without depending on the broker being online.

01

Save order

business row

→

02

Save event

outbox row

→

03

Commit once

same transaction

→

04

Publish later

retrying worker

atomic database transaction

at-least-once delivery

The outbox does not make two systems share a transaction. It makes the database remember what the broker still needs to learn.

The outbox has not made the database and broker atomic. It has removed the dangerous dual write. The database is now the durable source of messages that still need publishing.

Store enough context to operate it

Type and Payload are enough for a demo. Production rows should also answer the questions an operator asks during an incident: which business entity produced this, which request caused it, which contract version is inside it, and can I follow the work across traces?

public sealed class OutboxMessage
{
    public Guid Id { get; init; }
    public required string Type { get; init; }
    public required string Payload { get; init; }
    public required string AggregateType { get; init; }
    public required string AggregateId { get; init; }
    public required string CorrelationId { get; init; }
    public string? TraceParent { get; init; }
    public DateTimeOffset OccurredAt { get; init; }
    public DateTimeOffset? AvailableAt { get; set; }
    public DateTimeOffset? ProcessedAt { get; set; }
    public int Attempts { get; set; }
}

AvailableAt gives retries a durable backoff schedule. AggregateId lets you inspect or preserve ordering for one order without imposing global ordering. CorrelationId and TraceParent connect the original HTTP command to the later publisher and consumer, even though they run in different processes at different times.

Do not serialize an assembly-qualified CLR type name as the contract. Refactors would turn queued rows into unreadable data. Use a stable public name such as orders.order-placed.v1, deserialize it through an explicit registry, and treat the JSON as a deployed contract. Validate payload creation before committing the transaction; a poison message you knowingly stored is durable in the least helpful way.

The publisher's hot query also deserves an index. A partial index on pending rows, when the database supports it, keeps the scan proportional to the backlog rather than the lifetime history:

CREATE INDEX ix_outbox_pending
ON outbox_messages (available_at, occurred_at)
WHERE processed_at IS NULL;

That index complements cleanup. It does not excuse retaining millions of processed payloads in the primary table forever.

The publisher is intentionally boring

A background worker repeatedly claims pending rows, publishes them, and marks them processed. The important property is not cleverness; it is that every interruption leaves a state the next run can understand.

public async Task PublishBatchAsync(CancellationToken cancellationToken)
{
    var messages = await _db.OutboxMessages
        .Where(x => x.ProcessedAt == null)
        .OrderBy(x => x.OccurredAt)
        .Take(100)
        .ToListAsync(cancellationToken);

    foreach (var message in messages)
    {
        try
        {
            await _bus.PublishAsync(
                message.Type,
                message.Payload,
                message.Id,
                cancellationToken);

            message.ProcessedAt = DateTimeOffset.UtcNow;
        }
        catch (Exception exception) when (!cancellationToken.IsCancellationRequested)
        {
            message.Attempts++;
            message.LastError = exception.Message;
        }

        await _db.SaveChangesAsync(cancellationToken);
    }
}

That is the shape, not a production-ready leasing algorithm. With multiple workers, rows need to be claimed so two instances do not continuously publish the same batch. Depending on the database, teams use a short lease column, an atomic update that returns claimed rows, or row locks with a skip-locked option. Keep transactions around claiming and state changes short; do not hold a database transaction open across a slow broker call.

Delivery is at least once

There is one failure window the outbox deliberately cannot remove:

1. The worker publishes the message successfully.
2. The worker crashes before setting ProcessedAt.
3. The next worker publishes the same row again.

The event is not lost, but it can be duplicated. That makes an outbox an at-least-once delivery mechanism, not an exactly-once spell. The stable outbox Id should travel as the message ID so a consumer can record which messages it has handled.

CREATE TABLE processed_messages (
    consumer_name text NOT NULL,
    message_id uuid NOT NULL,
    processed_at timestamptz NOT NULL,
    PRIMARY KEY (consumer_name, message_id)
);

The consumer inserts that ID and applies its business change in one local transaction. A duplicate hits the unique key and becomes a no-op. Some handlers are naturally idempotent, such as setting a shipment status to a specific value. Others, such as incrementing a balance or sending an email, need explicit deduplication.

This is the message-processing version of idempotency keys in API design: retries are normal, so repeated intent must not multiply the effect.

The hard parts are operational

The table is easy. A dependable outbox needs decisions for the unglamorous cases:

• Backoff: a broker outage should not turn into a hot loop hammering both systems.
• Poison messages: after a bounded number of attempts, quarantine a message and alert instead of letting it block the batch forever.
• Ordering: only promise ordering where the domain requires it, usually within one aggregate or partition key. Global ordering destroys useful concurrency.
• Schema evolution: orders.order-placed.v1 is a contract. Old rows and old consumers may exist during a deployment, so add fields compatibly and version breaking changes.
• Cleanup: archive or delete processed rows in bounded batches after the retention window.
• Observability: use structured logs for delivery outcomes and measure oldest pending age, pending count, publish latency, attempts, and quarantined count. A worker can be "healthy" while the backlog is six hours old.

The Microsoft architecture guidance describes the same invariant: business data and the event are persisted atomically, then a separate process publishes unhandled entries. The storage and broker may change; the failure model does not.

When not to use it

Do not add an outbox to an operation that does not cross a consistency boundary. A single database write needs no messaging ceremony. A notification that is genuinely best-effort may not justify a durable pipeline. And if the platform already provides a transactionally integrated change feed, that may be the outbox mechanism rather than a table you poll yourself.

Use the pattern when losing an event would make two parts of the business disagree and the write cannot participate in one shared transaction. That is common in microservices, but it should be a conscious trade: more storage, a publisher, duplicate handling, cleanup, and monitoring in exchange for closing the lost-message gap.

The publisher still needs the right delivery topology. The message queue vs event bus guide separates one logical work owner from event fan-out, including the common design where every event subscription has its own competing worker pool.

The architecture skill is seeing the gap before production finds it. Katabench's architecture material focuses on boundaries like this, and the microservices Labs let you build the pattern inside a running system where checks can interrupt the workflow and verify the recovery path.