Event-Driven Architecture in Practice: From Message Queues to Event Sourcing — An Engineering Decision Guide

Bottom line: EDA is a tool, not a silver bullet

Event-Driven Architecture (EDA) has been widely discussed, but “decouple with a message queue” is only the shallowest layer. The real questions are: when do you actually need event-driven? Which message broker? Is Event Sourcing necessary? Are Saga and CQRS inseparable companions or independent tools?

This article breaks EDA down into three decision layers:

Layer	Problem	Core Decision
Pattern layer	Should I use EDA?	EDA vs request-response, event type definition
Architecture layer	How to organize events?	Event Sourcing vs message queues, CQRS, Saga
Technology layer	Which implementation?	Kafka vs RabbitMQ vs Pulsar, serialization, idempotency

---

1. Pattern layer: When EDA fits

1.1 When to choose event-driven

EDA’s core value is not speed but decoupling — the sender doesn’t need to know who processes the event or whether processing succeeds. Good candidates:

Scenario	Example	Benefit
Cross-service orchestration	Order → inventory → shipping	Each step is asynchronous, no blocking
State broadcast	Profile update → sync to search/cache/recommendation	One publication, N independent consumers
Data pipeline	Clickstream → cleaning → analytics → reporting	Each stage scales independently
External integration	Webhook → validation → transformation → persistence	Isolate external instability from core flow

1.2 When NOT to use EDA

• Strong consistency: Payment deduction, inventory locking — EDA’s eventual consistency here requires additional compensation (Saga), drastically increasing complexity.
• Small team, small system: 3-5 services with HTTP calls is more practical than introducing a message queue. Get it working first, introduce EDA when you hit a bottleneck.
• Simple CRUD: If your business has no complex event flow, a message queue only adds operational overhead.

---

2. Architecture layer: Three core patterns

2.1 Message Queue — The basic EDA

The classic pattern: producers publish messages to a queue, consumers pull and process. The core mechanism is point-to-point — each message is consumed by exactly one consumer.

┌────────┐  publish  ┌──────────┐  pull   ┌────────┐
│Producer│ ────────→ │  Queue   │ ──────→ │Consumer│
└────────┘           └──────────┘         └────────┘

Best for: task distribution (image processing, email sending), async RPC callbacks. RabbitMQ is the representative implementation.

2.2 Event Stream — For data pipelines

The event stream is fundamentally a log model — events are durably stored in partitions, consumers maintain their own offsets, and can replay historical events.

     ┌──────────┐  ┌──────────┐  ┌──────────┐
     │Consumer A│  │Consumer B│  │Consumer C│
     └────┬─────┘  └────┬─────┘  └────┬─────┘
          │             │             │
     ┌────▼─────────────▼─────────────▼────┐
     │           Event Stream              │
     │   (Partition 0, 1, 2, ...)          │
     └──────────────────────────────────────┘
     ┌──────────────────────────────────────┐
     │             Producer(s)              │
     └──────────────────────────────────────┘

Key differences from message queues:

Dimension	Message Queue (RabbitMQ)	Event Stream (Kafka)
Message lifecycle	Deleted after consumption	Retained until expiry
Consumption model	Competing (one message, one consumer)	Broadcast (each message, all consumers)
Replay capability	None	Strong (offset reset)
Throughput	Tens of thousands/sec	Millions/sec
Typical use	Task distribution, async RPC	Log aggregation, event processing, data pipelines

2.3 Event Sourcing + CQRS — For complex business state

Event Sourcing does not store current state; it stores the stream of state-changing events. To get the current state, replay all events.

   ┌──────┐  event  ┌──────────┐  project  ┌──────────┐
   │Command│ ──────→ │Event Store│ ────────→ │ Read DB  │
   └──────┘          └──────────┘           └──────────┘
       │                  │                      │
       ▼                  ▼                      ▼
  validate rules    append-only write       serve queries

CQRS (Command Query Responsibility Segregation) is often paired with Event Sourcing but is not required. Using CQRS alone (separate read/write databases) solves most “read vs write performance conflict” problems. Event Sourcing adds:

• Complete audit log: every state change is traceable
• Time travel: reconstruct any historical state
• Event-driven integration: other systems subscribe to the event stream

But the cost is significant:

• Eventual consistency: read models lag behind the write model
• Event schema versioning: schemas must evolve with backward compatibility
• Mental model shift: contradicts the “database holds current state” paradigm

---

3. Technology layer: Selection and implementation

3.1 Message broker selection

Feature	RabbitMQ	Apache Kafka	Apache Pulsar
Message model	Queue	Log	Queue + Log
Latency	Microseconds	Milliseconds	Milliseconds
Throughput	10K/sec	1M/sec	1M/sec
Message ordering	Ordered per queue	Ordered per partition	Ordered per shard
Ops complexity	Low	Medium	High
Learning curve	Low	Medium	High

Selection rules:

• Task distribution, RPC callbacks → RabbitMQ (low latency, simple, reliable)
• Event streams, log aggregation, data pipelines → Kafka (high throughput, persistence, replay)
• Mixed queue + stream → Pulsar (layered architecture, highest ops cost)
• New project, uncertain → start with RabbitMQ, evaluate Kafka when needed

3.2 Idempotency — EDA’s first line of defense

Messages in distributed systems can be delivered more than once (network glitches, consumer crashes, retries). Producers must guarantee at-least-once delivery; consumers must be idempotent (processing the same event multiple times produces the same result).

Idempotency implementations (simplest to most reliable):

1. Naturally idempotent: SET key = value, DELETE WHERE id = x
2. Deduplication table: use event ID as unique constraint, skip duplicates
3. Optimistic lock: version field check, retry on mismatch
4. State machine: only allow specific state transitions

3.3 Event schema management

Events are contracts between services. Poor schema management causes hard-to-debug production failures.

Approach	Characteristics	Best for
JSON (no schema)	Flexible, zero dependencies	Small teams, fast iteration
Avro + Schema Registry	Strong typing, compatibility checks	Medium/large, cross-team
Protobuf	Good performance, code generation	Teams with existing Protobuf infra
CloudEvents	Standard format, cross-platform	Multi-cloud, hybrid architecture

Rule: only add fields to events, never remove. New fields must have defaults. Consumers handle what they know and ignore what they don’t — this is the bottom line for forward compatibility in distributed systems.

---

4. Practical decision path

When facing “should we introduce a message queue?” in an architecture review, follow this sequence:

Do you actually need decoupling?
  ├─ No → HTTP calls, keep it simple
  └─ Yes → What type of events?
       ├─ Task distribution / work queues → RabbitMQ
       ├─ Event streams / broadcast → Kafka
       └─ Need Event Sourcing?
            ├─ No → CQRS + traditional DB is sufficient
            └─ Yes → Event Store + CQRS

---

Summary

EDA is a powerful toolbox, but each tool has its sweet spot:

Scenario	Recommended approach
Async task distribution between services	RabbitMQ + message queue
Cross-service state sync / broadcast	Kafka + event stream
Full audit trail for complex business state	Event Sourcing + CQRS
Distributed transaction coordination	Saga (orchestration + Kafka event stream)
Simple scenario, no middleware	HTTP callback + retry + dedup table

The single most important piece of advice: do not introduce EDA because it “might be useful”. Wait until you actually hit the problem — “this API is slow because it waits for three downstream services to finish” — and at that point, you’ll know exactly which layer needs which infrastructure.

---

Related reading

• Message Queue Comparison: RabbitMQ vs Kafka vs Pulsar — In-depth architecture comparison of three mainstream message brokers
• Microservice Decomposition: From Monolith to Services — Finding the right granularity and how EDA fits
• Monolith to Microservices Migration: 6 Proven Strategies — Event interception and CDC sync in migration practice
• API Gateway Selection Guide: Kong vs APISIX — Where the gateway layer ends and event-driven begins

Need an architecture review for your system? Get in touch →