Messaging Systems Overview

The previous chapter showed four ways to integrate applications and argued that messaging is the right default for loosely coupled systems. But “messaging” is a big idea — too big to implement as a single pattern. Hohpe and Woolf decompose it into six fundamental building blocks that every messaging architecture is made of. These six concepts are the vocabulary for everything that follows in this tutorial: every pattern in Parts 3 through 8 is a refinement, a specialization, or a composition of these building blocks.

This chapter introduces all six, shows how each one appears in the shipping domain, and demonstrates how Apache Camel models them. By the end, you’ll be able to look at any Camel route and identify the channels, messages, pipes, filters, routers, translators, and endpoints that compose it.

Figure 3.1 — The six building blocks of a messaging system

Download .excalidraw source Edit on excalidraw.com

Pattern 1: Message Channel

What it is

A Message Channel is a logical conduit that connects a sender to a receiver. When order-service emits an OrderPlaced event, it doesn’t send it to inventory-service — it sends it to a channel. Inventory-service (and notification-service, and any future consumer) reads from that channel independently. The channel is the decoupling mechanism: the sender knows the channel name, not the receiver; the receiver knows the channel name, not the sender.

In our stack, a channel is a Kafka topic or a Pulsar topic. The topic eip.orders.placed is a channel. So is eip.inventory.reserved, eip.payments.processed, and every other topic in the event catalog.

The two fundamental types

Channels come in two flavors, and the distinction matters for every pattern that follows:

Point-to-Point Channel — A message is consumed by exactly one receiver. If multiple consumers are listening, they compete: each message goes to one and only one of them. In Kafka, this is a topic with a single consumer group. In Pulsar, this is an exclusive or failover subscription.
Publish-Subscribe Channel — A message is delivered to every subscriber. Each consumer gets its own copy. In Kafka, this is a topic with multiple consumer groups (each group gets every message). In Pulsar, this is multiple subscriptions to the same topic.

Our shipping domain uses both:

Point-to-Point: inventory-service’s consumer group (inventory-service) competing across instances for OrderPlaced events — each order is processed by exactly one instance.
Publish-Subscribe: OrderPlaced delivered to both inventory-service and notification-service — different consumer groups, each getting every message.

We’ll explore these in depth in Part 3 (Messaging Channels).

How Camel models it

In Camel, a channel is a component endpoint URI. The component (kafka:, pulsar:, jms:, file:, direct:) determines the transport; the path determines the specific channel:

// Sending to a channel
from("direct:order-placed")
    .to("kafka:eip.orders.placed?brokers=localhost:9092");

// Receiving from a channel
from("kafka:eip.orders.placed?brokers=localhost:9092&groupId=inventory-service")
    .to("direct:process-order");

The from() and to() calls are how Camel connects to channels. Every route starts by consuming from one channel (from) and ends by producing to one or more channels (to). The channel abstraction means you can swap Kafka for Pulsar, JMS, or even a file directory by changing the URI — the route logic stays the same.

Pattern 2: Message

What it is

A Message is the unit of data that travels through a channel. It’s an atomic, self-contained packet: everything the receiver needs to understand and act on the sender’s intent is inside the message.

Every message has two parts:

Header — Metadata about the message: where it came from, where it’s going, what type of content it carries, when it was created, a unique identifier, a correlation ID. Headers are key-value pairs that travel with the message but aren’t part of the business data. Camel routes frequently read, set, and branch on headers.
Body — The payload: the business data itself. In our shipping domain, the body of an OrderPlaced message is the JSON (or Avro-serialized) order record with order_id, customer_id, item_sku, quantity, amount, and status.

Message types

Hohpe and Woolf identify three semantic types of message (we’ll cover them fully in Part 4):

Type	Intent	Example
Command Message	“Do this”	`ReserveInventory` — tells inventory-service to act
Document Message	“Here’s the data”	`OrderExport` — a batch of order records for accounting
Event Message	“This happened”	`OrderPlaced` — a fact about something that occurred

Our shipping domain uses event messages almost exclusively. This isn’t accidental — event-driven architectures are inherently more decoupled than command-driven ones, because the sender doesn’t dictate what the receiver should do; it just announces what happened.

How Camel models it

In Camel, a message is represented by the Exchange object, which contains:

Exchange.getIn() — The input message (headers + body).
Exchange.getMessage() — The current message (preferred in Camel 4.x; the getIn()/getOut() distinction is deprecated).
Exchange.getProperties() — Exchange-level properties (not transmitted to the channel; used for route-internal state).

from("kafka:eip.orders.placed?brokers=localhost:9092&groupId=inventory-service")
    .routeId("message-anatomy")
    // Reading headers
    .log("Kafka key: ${header.kafka.KEY}")
    .log("Kafka partition: ${header.kafka.PARTITION}")
    .log("Kafka offset: ${header.kafka.OFFSET}")
    // Reading and transforming the body
    .unmarshal().json(Map.class)
    .log("Order ${body[order_id]} for customer ${body[customer_id]}")
    // Setting a header for downstream routing
    .setHeader("orderPriority", simple("${body[amount]} > 1000 ? 'HIGH' : 'STANDARD'"))
    .log("Priority: ${header.orderPriority}");

Notice how Kafka metadata (key, partition, offset, topic, timestamp) arrives as Camel headers prefixed with kafka.. Every Camel component populates component-specific headers — the file component sets CamelFileName, the http component sets CamelHttpResponseCode, and so on. These headers are the mechanism by which transport-layer metadata flows into route logic.

Pattern 3: Pipes and Filters

Figure 3.1 — Pipes and Filters: each filter transforms the message

Download .excalidraw source Edit on excalidraw.com

What it is

Pipes and Filters is the architectural style that decomposes a message processing task into a sequence of independent, composable steps. Each step is a filter — a self-contained processing unit that receives a message, does one thing to it, and passes it along. The pipes are the channels that connect filters together.

This is the fundamental architecture of a Camel route. When you write:

from("kafka:orders") → unmarshal → validate → enrich → transform → to("kafka:processed")

…you’ve built a pipes-and-filters pipeline. Each step (unmarshal, validate, enrich, transform) is a filter. The arrows between them are pipes. Each filter is independent: it doesn’t know what came before it or what comes after. It just processes the message it receives and passes it along.

Why it matters

The power of pipes and filters is composability:

Reordering — You can rearrange the steps without changing their implementations. Move the validation filter before the enrichment filter? Just reorder the route.
Insertion — You can add a new step (say, logging or wire-tapping) without modifying existing ones.
Removal — You can skip a step (say, disable enrichment in a test environment) without affecting the rest of the pipeline.
Reuse — The same filter can appear in multiple pipelines. A JSON-to-Avro transformer can be used in the order pipeline and the payment pipeline.
Testability — Each filter can be unit-tested in isolation with a mock input and expected output.

How Camel models it

Every Camel route is a pipes-and-filters pipeline. The DSL’s method chaining (Java), step lists (YAML), or nested elements (XML) define the sequence of filters:

from("kafka:eip.orders.placed?brokers=localhost:9092&groupId=order-pipeline")
    .routeId("pipes-and-filters")
    // Filter 1: Deserialize
    .unmarshal().json(Map.class)
    // Filter 2: Validate
    .filter(simple("${body[amount]} > 0 && ${body[quantity]} > 0"))
    // Filter 3: Enrich with customer tier
    .process(exchange -> {
        Map<String, Object> body = exchange.getIn().getBody(Map.class);
        double amount = Double.parseDouble(body.get("amount").toString());
        body.put("tier", amount > 500 ? "GOLD" : "STANDARD");
        exchange.getIn().setBody(body);
    })
    // Filter 4: Transform to canonical format
    .marshal().json()
    // Pipe to the next channel
    .to("kafka:eip.orders.enriched?brokers=localhost:9092");

Each step in the chain is a filter. Camel manages the pipes between them — the internal in-memory handoff from one step to the next. When the final to() sends the message to a Kafka topic, that’s an external pipe connecting this pipeline to whatever consumes eip.orders.enriched next.

Pattern 4: Message Router

What it is

A Message Router is a filter that doesn’t transform the message — it reads the message and decides where to send it next. It’s a fork in the pipeline. The message comes in, the router inspects something (a header value, a body field, an external condition), and it forwards the message to one of several output channels.

Shipping domain example

When an order arrives, the system needs to route it based on the order amount:

Orders over $1,000 go to a priority processing channel.
Orders under $50 go to a small-order batch channel.
Everything else goes to standard processing.

How Camel models it

Camel’s Choice EIP is the core message router. It evaluates predicates in order and routes to the first match:

from("kafka:eip.orders.placed?brokers=localhost:9092&groupId=router-demo")
    .routeId("message-router")
    .unmarshal().json(Map.class)
    .choice()
        .when(simple("${body[amount]} > 1000"))
            .log("Priority order ${body[order_id]}: $${body[amount]}")
            .to("direct:priority-processing")
        .when(simple("${body[amount]} < 50"))
            .log("Small order ${body[order_id]}: $${body[amount]}")
            .to("direct:batch-processing")
        .otherwise()
            .log("Standard order ${body[order_id]}: $${body[amount]}")
            .to("direct:standard-processing")
    .end();

The choice() is just the simplest form. Part 5 covers the full family of routing patterns: Content-Based Router (this one), Message Filter, Dynamic Router, Recipient List, Splitter, Aggregator, Scatter-Gather, Routing Slip, and Process Manager. Each one is a specialized message router.

Pattern 5: Message Translator

What it is

A Message Translator is a filter that changes the message’s format, structure, or representation without changing its semantic meaning. It’s the adapter between systems that use different data formats.

Shipping domain example

Order-service emits events as JSON, but the legacy accounting system expects XML. The payment gateway returns responses in a proprietary flat-file format that needs to become JSON before other services can consume it. The shipping carrier API uses SOAP/XML, but our internal events are Avro-serialized. Every one of these boundaries is a translation.

How Camel models it

Camel provides translation at multiple levels:

Data format marshaling — marshal() / unmarshal() to convert between serialization formats (JSON, XML, Avro, CSV, Protobuf, YAML).
Expression-based transformation — transform() with Simple, JSONPath, XPath, or any Camel expression language.
Processor-based transformation — Custom Java code in a process() block for complex transformations.
Bean method invocation — bean() to delegate transformation to a POJO method.

from("direct:translate-order-to-xml")
    .routeId("message-translator")
    .unmarshal().json(Map.class)
    .process(exchange -> {
        Map<String, Object> order = exchange.getIn().getBody(Map.class);
        String xml = String.format("""
            <order>
              <orderId>%s</orderId>
              <customerId>%s</customerId>
              <itemSku>%s</itemSku>
              <quantity>%s</quantity>
              <amount>%s</amount>
              <status>%s</status>
            </order>
            """,
            order.get("order_id"), order.get("customer_id"),
            order.get("item_sku"), order.get("quantity"),
            order.get("amount"), order.get("status"));
        exchange.getIn().setBody(xml);
        exchange.getIn().setHeader(Exchange.CONTENT_TYPE, "application/xml");
    })
    .to("direct:accounting-xml-inbound");

In practice, Camel’s built-in data formats handle most translations without custom code. Converting JSON to XML is as simple as chaining unmarshal().json() followed by marshal().jacksonXml(). Part 6 covers the full family of transformation patterns: Message Translator, Envelope Wrapper, Content Enricher, Content Filter, Claim Check, Normalizer, and Canonical Data Model.

Pattern 6: Message Endpoint

What it is

A Message Endpoint is the code that connects an application to a messaging channel. It’s the glue between your application’s domain logic and the messaging system. The endpoint knows how to send messages to a channel (a producer endpoint) or receive messages from a channel (a consumer endpoint), handling all the protocol details — serialization, connection management, error handling, acknowledgment — so the application code doesn’t have to.

Shipping domain example

Every service in the shipping domain has at least one message endpoint:

Service	Consumer Endpoints	Producer Endpoints
order-service	`eip.payments.processed`, `eip.shipping.scheduled`	`eip.orders.placed`, `eip.orders.cancelled`
inventory-service	`eip.orders.placed`, `eip.orders.cancelled`	`eip.inventory.reserved`, `eip.inventory.insufficient`
payment-service	`eip.inventory.reserved`	`eip.payments.processed`, `eip.payments.failed`
shipping-service	`eip.payments.processed`	`eip.shipping.scheduled`, `eip.shipping.delivered`
notification-service	All event topics	(none — pure consumer)

How Camel models it

This is where Camel shines. Every Camel component is a factory for message endpoints. The component handles all the protocol-specific details; you just provide a URI:

// Consumer endpoint — connects inventory-service to the orders channel
from("kafka:eip.orders.placed"
        + "?brokers=localhost:9092"
        + "&groupId=inventory-service"
        + "&autoOffsetReset=earliest"
        + "&valueDeserializer=org.apache.kafka.common.serialization.StringDeserializer")
    .routeId("message-endpoint-consumer")
    .log("Received order: ${body}")
    .to("direct:process-inventory-check");

// Producer endpoint — connects inventory-service to the reserved channel
from("direct:inventory-reserved")
    .routeId("message-endpoint-producer")
    .to("kafka:eip.inventory.reserved"
        + "?brokers=localhost:9092"
        + "&key=${header.orderId}"
        + "&valueSerializer=org.apache.kafka.common.serialization.StringSerializer");

Camel has over 400 components — each one is a message endpoint connector. kafka:, pulsar:, jms:, http:, file:, sql:, redis:, grpc:, aws2-s3:, slack: — every one of these turns an external system into a message endpoint that can participate in a Camel route. This is Camel’s core value proposition: you don’t write integration plumbing, you configure endpoints.

The component catalog

You can explore the full catalog from the Camel CLI:

# Search for a component
camel search --component kafka

# Read documentation for a component
camel doc kafka

# List all available components
camel catalog component

How the six patterns compose

Every Camel route is a composition of these six building blocks. Here’s a complete route from the shipping domain with each building block labeled:

// Message Endpoint (consumer) — connects to a Message Channel
from("kafka:eip.orders.placed?brokers=localhost:9092&groupId=order-processor")
    .routeId("six-patterns-composed")
    // Message — the Exchange flowing through this route
    .unmarshal().json(Map.class)               // Message Translator
    // Pipes and Filters — each step is a filter in the pipeline
    .log("Processing order ${body[order_id]}")
    .choice()                                   // Message Router
        .when(simple("${body[amount]} > 1000"))
            .setHeader("processingTier", constant("PRIORITY"))
        .otherwise()
            .setHeader("processingTier", constant("STANDARD"))
    .end()
    .process(exchange -> {                      // Message Translator
        Map<String, Object> body = exchange.getIn().getBody(Map.class);
        body.put("processing_tier", exchange.getIn().getHeader("processingTier"));
        body.put("processed_at", System.currentTimeMillis());
    })
    .marshal().json()                           // Message Translator
    // Message Endpoint (producer) — sends to a Message Channel
    .to("kafka:eip.orders.processed?brokers=localhost:9092");

Thirteen lines of route DSL. Six patterns. One pipeline. This is why the EIP book starts with these building blocks — they’re the atoms from which every integration molecule is constructed.

Running the examples

Save any of the YAML routes above to a file and run them directly:

camel run message-router.yaml --dev

For routes that consume from Kafka, make sure the local stack is running (./scripts/setup-stack.sh). You can publish test messages to eip.orders.placed using the Kafka UI at http://localhost:8090 or from the command line:

podman exec eip-kafka /opt/kafka/bin/kafka-console-producer.sh \
  --bootstrap-server localhost:9092 \
  --topic eip.orders.placed

Then paste a JSON order:

{"order_id": 1, "customer_id": "CUST-100", "item_sku": "SKU-ABC-42", "quantity": 2, "amount": 149.99, "status": "PLACED"}

Watch the Camel logs to see the message flow through the pipeline — each log() step shows where the message is and what’s happening to it.

Common pitfalls

Confusing channels with endpoints. A channel is the path (the Kafka topic); an endpoint is the connection to that path (the Camel component configured with broker address, consumer group, serializer, etc.). Two different endpoints can connect to the same channel with different configurations — and in fact, that’s how point-to-point and pub-sub coexist on the same topic.

Thinking of pipes and filters as a design choice. It’s not — it’s how Camel works. Every route is a pipeline. The question isn’t whether to use pipes and filters; it’s how to decompose your processing into the right filters. If a single route has 40 steps, it’s probably doing too much — split it into multiple routes connected by direct: or seda: channels.

Over-routing. Not every decision needs a message router. If there’s only one path, just use the pipeline. A choice() with a single when() and no otherwise() is just a filter — use filter() instead.

References

Hohpe & Woolf, Enterprise Integration Patterns, Chapter 3: “Messaging Systems”
enterpriseintegrationpatterns.com — Messaging
Apache Camel — Component Reference
Apache Camel — EIP Reference
Apache Camel — Simple Expression Language

What you learned

Message Channel — the logical conduit (Kafka topic, Pulsar topic) that decouples sender from receiver.
Message — the headers + body that flow through channels; Camel represents them as Exchange objects.
Pipes and Filters — every Camel route is a pipeline of composable processing steps.
Message Router — a filter that inspects the message and directs it to one of several channels.
Message Translator — a filter that converts the message’s format without changing its meaning.
Message Endpoint — the Camel component that connects application code to a channel.

These six building blocks are the vocabulary for the rest of this tutorial. Next, we dive into the first building block in depth — Messaging Channels — exploring point-to-point, publish-subscribe, dead letter channels, guaranteed delivery, and more.

Verification status: verified — messaging patterns verified against Quarkus 3.37.0, Camel 4.20.0 on Podman (2026-07-11). Confirm: all Camel route examples compile and run with camel run against Camel 4.20; Kafka component header names (kafka.KEY, kafka.PARTITION, kafka.OFFSET) match actual Camel Kafka component behavior; Simple expression syntax for map access (${body[field]}) works as shown.*