Troubleshooting Notes — Nginx Reverse Proxy: server_name vs IP

Our system depends on a third-party service that restricts access by IP. For security reasons, our system validates certificates, so we use an Nginx reverse proxy to access that service. After the service migrated to the cloud, our system started experiencing issues.

The Nginx configuration file looks like this:

http {
    upstream backend.example.com {
        server backend1.example.com:443;
   }

    server {
        listen      80;
        server_name www.example.com;

        location /upstream {
            proxy_pass                    https://backend.example.com;
            proxy_ssl_certificate         /etc/nginx/client.pem;
            proxy_ssl_certificate_key     /etc/nginx/client.key;
            proxy_ssl_protocols           TLSv1 TLSv1.1 TLSv1.2;
            proxy_ssl_ciphers             HIGH:!aNULL:!MD5;
            proxy_ssl_trusted_certificate /etc/nginx/trusted_ca_cert.crt;

            proxy_ssl_verify        on;
            proxy_ssl_verify_depth  2;
        }
    }
}

Debug

When the production issue occurred, the first thing I checked was the configuration change history — no changes had been made to the config.

Next, I checked whether the certificates had expired — they had not.

I tried a curl request using the domain name — the third-party service responded normally and the certificate was valid.

We had made no changes on our end, and the third-party had done nothing more than a simple migration — yet things broke.

After thinking it over for a while: since nothing had changed on the Nginx side, the problem had to be on the third party’s end. The most direct approach would be to contact the third-party provider, but the communication channel is cumbersome and time-consuming. To fix the issue quickly, we decided to debug blindly.

Attempts to Fix

First attempt: Since curl requests were all working fine, I tried removing the upstream block and writing the address directly into proxy_pass. Still failed — the error log showed "https://1.1.1.1:443 TLS handshake failed". This suggested that proxy_pass was resolving the domain name to an IP address.

Second attempt: I tried curling the IP directly to access the service. Failed. This led to the conclusion that the IP and the domain name were pointing to two different services. At this point we had found the root cause.

Solution direction: During reverse proxying, access the upstream service by domain name rather than by IP.

After checking the documentation, I found the directive proxy_ssl_server_name, which defaults to off. The fix is to add the following to the configuration file:

proxy_ssl_server_name on;

Third attempt: Added proxy_ssl_server_name on to the configuration file. Success.

What Is server_name

Server Name Indication (SNI) is an extension to the TLS protocol. At the start of the handshake, it allows the client to indicate the hostname it is attempting to connect to. This enables a server to host multiple certificates on the same IP address and TCP port, thereby allowing a single IP address to serve multiple HTTPS websites without requiring all of them to share the same certificate.