Enterprise NATS Messaging Cluster

NATSGoDockerAWSJetStreamMQTT

Architecture Overview

Hover over the highlighted elements in the diagram to reveal additional details.

Public Subnet C
Public Subnet A
Server A
           NATS AUTH container
The NATS container exposes 2 ports: - 4222: for client connections (pub & sub). This is accessed through the public subnet. - 6222: for cluster communication. This is accessed through the private subnet.The NATS container exposes 2 ports: - 4222: for client connections (pub & sub). This is accessed through the public subnet. - 6222: for cluster communication. This is accessed through the private subnet.
      NATS container
Public Subnet A
      NATS Container
The AUTH container validates the JWT token sent by web clients when connecting to NATS. Connections done with NKEYs are handled directly by the NATS container.The AUTH container validates the JWT token sent by web clients when connecting to NATS. Connections done with NKEYs are handled directly by the NATS container.
          NATS AUTH container
Server B
         
NATS container
         
NATS AUTH container
Server C
Cluster connection
(Port 6222)
Server connections are established using private IP, since all servers are in the same VPC.
Cluster connection
(Port 6222)
Cluster connection
(Port 6222)
Client connection
(Port 4222)
System client connects to NATS using NKEY. It receives updates from external data provider, formats the data & publishes it through NATS connection.System client connects to NATS using NKEY. It receives updates from external data provider, formats the data & publishes it through NATS connection.System client connects to NATS using NKEY. It receives updates from external data provider, formats the data & publishes it through NATS connection.
System Client
Client connection is established through public internet. Connection string: tls://nats.[domain].com:4222
Client connection
(Port 4222)
Web user connects to NATS using JWT token.Web user connects to NATS using JWT token.Web user connects to NATS using JWT token.
Web Client Y
Web Client Z
Client connection
(Port 4222)
Publish to sports.football.112233:
{time: 15, goals:[..]}
User subscribed to specific match IDUser subscribed to specific match ID
Received on
sports.football.112233:
{time: 15, goals:[..]}
User subscribed to specific sport with all matches.User subscribed to specific sport with all matches.
Received on
sports.football.*:
{time: 15, goals:[..]}
MQTT connection
thesports.com
(Sports updates provider)

Project Details

An enterprise-grade distributed messaging system I architected and led development for, designed to handle real-time sports updates from external MQTT feeds and distribute them to internal systems and web clients. The solution provides high availability, fault tolerance, and enterprise-level security through a 3-node NATS cluster deployment.

Business Requirements

  • Real-time sports data ingestion from thesports.com via MQTT
  • High-availability message distribution to internal applications
  • Secure real-time updates to authenticated web users
  • Fault-tolerant architecture with zero message loss
  • Scalable system supporting multiple client types

Architecture Design

  • 3-node NATS cluster across different AWS subnets for fault tolerance
  • JetStream persistence layer for message durability and replay capability
  • Go-based authentication service integration
  • Dockerized NATS containers for consistent deployment
  • Load balancing across cluster nodes for high availability
  • MQTT to NATS bridge for external data ingestion

Security Implementation

  • Private IP cluster communication for enhanced security
  • Multi-tenant security with separate NATS accounts per application
  • NKEY-based authentication for internal system clients
  • Custom Go-based NATS AUTH container for JWT verification
  • Fine-grained permissions and access controls
  • TLS encryption for all cluster and client communications