Lexora
A production Rust backend with an event-driven core: transactional outbox, background workers and a React front end, running on Cloud Run.
- Ownership
- Commissioned and co-developed product. Red Sentra continues to support it. Ownership arrangements and the partner stay private.
- Availability
- Live product. Source is private.
- Stack
- Rust / React / Cloud Run / Background workers
The problem
A product where one user action has to trigger several downstream effects, and none of them are allowed to quietly not happen. The naive version, doing the side effects inline with the request, loses work every time something fails halfway through.
The system
A Rust backend built around an event-driven core. A state change and the events it produces are committed together through a transactional outbox, and background workers pick those events up and do the slow work outside the request. A React front end sits on top, and the whole thing runs on Cloud Run.
My role
Backend architecture and implementation of the event-driven core, the outbox and worker design, and the production operations story around it.
The hard part
Delivery guarantees. Between writing to the database and telling the rest of the system about it there is a window where a crash loses the event. Closing that window without reaching for a distributed transaction is essentially the whole design.
Architecture decisions
- At-least-once delivery with idempotent consumers. Duplicates are much cheaper to handle than losses, so workers are written to tolerate seeing the same event twice.
- PostgreSQL is the queue. Kafka would have added an operational surface and a second failure domain to reason about, and it would still have needed an outbox to be atomic with the transaction it describes. On this system the broker buys nothing the table does not already give. LISTEN/NOTIFY was rejected as the primary mechanism for a plainer reason: notifications are not durable, so a worker that is down misses them outright.
- Only side effects are eventually consistent. Authorization, assignment persistence and learning state stay synchronous, so a learner is never shown a schedule that has not durably committed. The costs of the queue are accepted rather than hidden: polling adds roughly two seconds of latency when the system is idle, and the outbox table grows until a retention job exists to trim it.
Where it stands
In production. Red Sentra continues to support and extend it.