NGINX Proxy Stream - Using SNI (Server Name Indication) - Tridium Niagara

Tridium Niagara
1

Configuring NGINX for Client Connections

Overview

In this post, I’ll guide you through configuring NGINX to proxy client connections through a single port, 4911, for a cloud-based centralised server setup. This configuration helps manage multiple supervisors and remote JACEs or intermediate supervisors installed at various sites, connecting through a centralised NGINX server.

Setup Description

Several supervisors are running on a single machine (or multiple machines), each on different ports (4912 to 4914). Our goal is to use NGINX to route all client connections to these supervisors through a single port, 4911, while using domain names for each site.

Example Domains

We use distinct domains for each site pointing to the server address:

  • nsw.server
  • qld.server
  • act.server
  • (and so on…)

NGINX Configuration

Main Settings

worker_processes 1;

events {
    worker_connections 1024;
}

HTTP Settings
The HTTP block handles web traffic, manages SSL configurations, and forwards requests to the appropriate local supervisors based on the domain name:

http {
    server_names_hash_bucket_size 64;
    include mime.types;
    default_type application/octet-stream;

    sendfile on;
    keepalive_timeout 65;

    # SSL Settings
    ssl_certificate      /nginx/ssl/certs/your_cert.crt;
    ssl_certificate_key  /nginx/ssl/private/your_cert_private.pem;
    ssl_trusted_certificate /nginx/ssl/certs/your_cert_CA.crt;
    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;
    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;
}

Server Block for Each Site
Each server block listens on port 443 (HTTPS) and proxies requests to the appropriate local supervisor web server based on the domain name:

    # Example Server Block
    server {
        listen 443 ssl;
        server_name nsw.server;

        location / {
            proxy_pass http://localhost:83; #The internal port of the webjetty server from NSW
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

Repeat similar blocks for other domains (qld.server, act.server, etc.), pointing to the respective local ports.

Stream Settings
The stream block handles TCP traffic, enabling NGINX to proxy connections based on the domain names. This section explains how Server Name Indication (SNI) is used to achieve this:

:exclamation: Streaming using SNI is only avaliable from Niagara 4.13+. This feature must be enable in system.properties: org.bouncycastle.jsse.client.assumeOriginalHostName=true
If you do not enable this the SNI NGINX will not be able to extract the domain and direct traffic to the correct internal port.

stream {
    log_format stream '$remote_addr [$time_local] '
                      '$protocol $status $bytes_sent $bytes_received '
                      '$session_time "$ssl_preread_server_name"';
    access_log /nginx/logs/stream.log stream;
    error_log /nginx/logs/stream_error.log debug;

    map_hash_bucket_size 64;

    map $ssl_preread_server_name $upstream_name {
        nsw.server nsw;
        qld.server qld;
        act.server act;
        default nsw;
    }

    upstream nsw {
        server localhost:4912;
    }
    upstream qld {
        server localhost:4913;
    }
    upstream act {
        server localhost:4914;
    }

    server {
        listen 4911;
        ssl_preread on;
        proxy_pass $upstream_name;
    }
}

Explanation

  • HTTP Block: Manages web traffic and SSL settings, providing secure connections to the server. Each server block within the HTTP block listens for connections on port 443 (HTTPS) and forwards the requests to the corresponding supervisor based on the domain name.
  • Stream Block: Manages TCP traffic, using Server Name Indication (SNI) to determine the correct upstream server. SNI is an extension of the TLS protocol, which allows the client to specify the hostname it is trying to connect to at the start of the handshake process. This enables NGINX to use the $ssl_preread_server_name variable to determine which upstream server to route the connection to based on the domain name. By enabling ssl_preread, NGINX can extract the server name from the client’s initial TLS handshake and use it to route the connection appropriately. This allows all remote JACEs or intermediate supervisors to connect through a single port (4911), simplifying the configuration and management of connections.

Full NGINX Configuration Example

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    server_names_hash_bucket_size 64;
    include mime.types;
    default_type application/octet-stream;

    sendfile on;
    keepalive_timeout 65;

    # SSL Settings
    ssl_certificate      /nginx/ssl/certs/your_cert.crt;
    ssl_certificate_key  /nginx/ssl/private/your_private_key.pem;
    ssl_trusted_certificate /nginx/ssl/certs/your_CA.crt;
    ssl_session_cache    shared:SSL:1m;
    ssl_session_timeout  5m;
    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;

    # Server block for nsw.server
    server {
        listen 443 ssl;
        server_name nsw.server;

        location / {
            proxy_pass http://localhost:81;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

    # Server block for qld.server
    server {
        listen 443 ssl;
        server_name qld.server;

        location / {
            proxy_pass http://localhost:82;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

    # Server block for act.server
    server {
        listen 443 ssl;
        server_name act.server;

        location / {
            proxy_pass http://localhost:83;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

stream {
    log_format stream '$remote_addr [$time_local] '
                      '$protocol $status $bytes_sent $bytes_received '
                      '$session_time "$ssl_preread_server_name"';
    access_log /nginx/logs/stream.log stream;
    error_log /nginx/logs/stream_error.log debug;

    map_hash_bucket_size 64;

    map $ssl_preread_server_name $upstream_name {
        nsw.server nsw;
        qld.server qld;
        act.server act;
        default nsw;
    }

    upstream nsw {
        server localhost:4913;
    }
    upstream qld {
        server localhost:4914;
    }
    upstream act {
        server localhost:4914;
    }


    server {
        listen 4911;
        ssl_preread on;
        proxy_pass $upstream_name;
    }
}