Fan-out pattern

What is fan-out?

PublisherPublish once

user-eventsChannel

Email Service

Analytics

Provisioning

DLQOn failure

Fan-out is a messaging pattern where a single published message is delivered independently to every subscriber on a channel. Each subscriber receives its own copy of the message and processes it at its own pace. A failure in one subscriber has no effect on the others.

This pattern is fundamental to event-driven architectures. When something important happens in your system, multiple services often need to react, each in their own way.

When to use fan-out

Fan-out is the right choice when:

Notifications: A user signs up and you need to send a welcome email, create an analytics event, and provision their account, all at the same time.
Audit logging: Every mutation in your system should be recorded by an audit service, independent of the primary write path.
Analytics: Capture domain events for analytics pipelines without coupling your application logic to your analytics provider.
Cross-service events: Multiple downstream services need to react to the same business event without tight coupling between them.

Basic fan-out

The simplest fan-out setup involves one channel with multiple webhook subscribers. Each subscriber receives every message published to the channel.

setup.ts

import { NPayload } from "@npayload/node";

const np = new NPayload({
  instanceUrl: process.env.NPAYLOAD_INSTANCE_URL,
  token: process.env.NPAYLOAD_TOKEN,
});

// Create a channel for user events
const channel = await np.channels.create({
  name: "user-events",
  description: "All user lifecycle events",
});

// Subscribe three independent services
await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://email-service.example.com/hooks/npayload",
  name: "email-service",
});

await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://analytics.example.com/hooks/npayload",
  name: "analytics-service",
});

await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://provisioning.example.com/hooks/npayload",
  name: "provisioning-service",
});

Now every message published to user-events is delivered to all three services independently:

publish.ts

await np.messages.publish({
  channelId: channel.id,
  payload: {
    type: "user.signup",
    userId: "usr_abc123",
    email: "alice@example.com",
    plan: "pro",
  },
});
// All three subscribers receive this message independently

Selective fan-out with routing keys

Not every subscriber needs every message. Use routing keys to let subscribers filter which messages they receive, without creating separate channels for each event type.

selective-setup.ts

// Subscribe the email service only to user-facing events
await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://email-service.example.com/hooks/npayload",
  name: "email-service",
  routingKey: "user.*",
});

// Subscribe the billing service only to payment events
await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://billing.example.com/hooks/npayload",
  name: "billing-service",
  routingKey: "payment.*",
});

// Subscribe the audit service to everything
await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://audit.example.com/hooks/npayload",
  name: "audit-service",
  routingKey: "#",
});

publish-selective.ts

// Only email-service and audit-service receive this
await np.messages.publish({
  channelId: channel.id,
  routingKey: "user.signup",
  payload: { userId: "usr_abc123", email: "alice@example.com" },
});

// Only billing-service and audit-service receive this
await np.messages.publish({
  channelId: channel.id,
  routingKey: "payment.completed",
  payload: { orderId: "ord_xyz789", amount: 99.99 },
});

Fan-out with consumer groups

When a single subscriber service runs multiple instances for high availability, you need load balancing within that service. Consumer groups solve this by distributing messages across instances of the same service, while still fanning out across different services.

consumer-group-setup.ts

// Create a consumer group for the email service (3 instances)
await np.consumerGroups.create({
  channelId: channel.id,
  name: "email-service-group",
  strategy: "round-robin",
});

// Each email service instance joins the same group
// Instance 1
await np.consumerGroups.join({
  groupId: emailGroup.id,
  webhookUrl: "https://email-1.example.com/hooks/npayload",
});

// Instance 2
await np.consumerGroups.join({
  groupId: emailGroup.id,
  webhookUrl: "https://email-2.example.com/hooks/npayload",
});

// The analytics service still gets every message (no group)
await np.subscriptions.create({
  channelId: channel.id,
  webhookUrl: "https://analytics.example.com/hooks/npayload",
  name: "analytics-service",
});

With this setup, each published message is delivered to one email service instance (load-balanced) and to the analytics service (every message).

Scaling considerations

npayload supports 100+ subscribers per channel. Here are guidelines for scaling fan-out:

Scenario	Recommendation
Under 10 subscribers	Direct subscriptions work well
10 to 50 subscribers	Use routing keys to reduce unnecessary deliveries
50 to 100+ subscribers	Combine routing keys with consumer groups for load-balanced services
High-throughput channels	Use priority levels so critical consumers are served first

For channels with very high message rates, consider splitting by routing key prefix into separate channels. This gives you independent throughput limits per event category.

Error isolation

One of the key benefits of fan-out is that each subscriber's delivery is independent. If the email service is down, the analytics and provisioning services continue to receive messages normally.

npayload provides several mechanisms for handling subscriber failures:

Automatic retries: Failed webhook deliveries are retried with exponential backoff.
Dead letter queue: After exhausting retries, messages land in the channel's DLQ for manual inspection.
Circuit breaker: If a subscriber consistently fails, npayload opens a circuit breaker to stop sending to it temporarily, protecting both your service and npayload from wasted resources.

check-dlq.ts

// Check the DLQ for failed deliveries
const deadLetters = await np.dlq.list({
  channelId: channel.id,
  limit: 20,
});

for (const entry of deadLetters.items) {
  console.log(
    `Failed delivery to ${entry.subscriberName}: ${entry.failureReason}`
  );

  // Retry or discard
  if (entry.retryable) {
    await np.dlq.retry({ entryId: entry.id });
  }
}

Monitoring fan-out health

Track the health of your fan-out topology by monitoring subscription lag and delivery rates:

monitor.ts

// Get delivery stats for all subscribers on a channel
const stats = await np.channels.stats({
  channelId: channel.id,
});

for (const sub of stats.subscribers) {
  console.log(`${sub.name}: ${sub.pendingMessages} pending, ${sub.deliveryRate}/s`);

  if (sub.pendingMessages > 1000) {
    console.warn(`Subscriber ${sub.name} is falling behind`);
  }
}

Anti-patterns

Too many channels instead of routing keys

Creating a separate channel for every event type leads to management overhead and makes it harder to add new consumers. Instead, use one channel per domain with routing keys for event types.

// Bad: one channel per event type
await np.channels.create({ name: "user-signup" });
await np.channels.create({ name: "user-updated" });
await np.channels.create({ name: "user-deleted" });

// Good: one channel with routing keys
await np.channels.create({ name: "user-events" });
await np.messages.publish({
  channelId: userEventsChannel.id,
  routingKey: "user.signup",
  payload: { ... },
});

Chatty publishers

Publishing many small messages at high frequency creates unnecessary overhead. Batch related events when possible:

// Bad: one message per item
for (const item of cartItems) {
  await np.messages.publish({
    channelId: channel.id,
    payload: { type: "item.added", item },
  });
}

// Good: one message with all items
await np.messages.publish({
  channelId: channel.id,
  payload: { type: "cart.updated", items: cartItems },
});

Missing error handling

Always configure a DLQ on channels used for fan-out. Without one, failed deliveries are silently dropped after retries are exhausted.

Next steps

Subscriptions for details on subscription types and configuration
Webhooks for setting up reliable webhook delivery
Consumer groups for load-balanced message distribution

Was this page helpful?

Fan-out pattern

On this page