NGINX反向代理https

架构说明

我的实践是构建 docs.cloud-atlas.io :

  • HTTPS反向代理部署在阿里云上: 因为阿里云有稳定的公网带宽

  • HTTP服务器部署在 Raspberry Pi Cluster : 部署在家庭内部的微型 Raspberry Pi 系统

    • 核心服务器完全是自己部署,可以自由扩展服务器集群规模而没有云计算的高昂费用

    • 所有软硬件都由自己打造,可以充分锻炼 一个人的数据中心 技术堆栈

    • 个人家庭宽带没有公网IP,通过 CTunnel 打通阿里云公网服务器到家庭内部集群的通道

安装Nginx

Debian 系安装Nginx
apt update
apt install nginx
Arch Linux 安装Nginx
# 需要安装certbot-nginx插件,以便能够让Certbot配置nginx
pacman -S nginx certbot-nginx

配置Nginx

备注

需要简单配置Nginx的服务域名 Nginx virtual host配置 这样后续执行 certbot 会自动修订配置完成TLS/SSL配置修订。

警告

如果服务器在墙内云上,如果没有备案, Cetbot 配置时反向访问HTTP 80端口会被云厂商拦截导致配置失败。请备案后执行,或者在海外服务器上配置后再迁移。

  • 配置 /etc/nginx/conf.d/cloud-atlas.io.conf ( 这个文件命名只需要以 .conf 结尾即可包含在 /etc/nginx/nginx.conf 中,具体根据nginx版本发行版提供的 /etc/nginx.conf 而定 ):

/etc/nginx/nginx.conf 包含的 Nginx virtual host配置 配置
server {
    listen 80;
    listen [::]:80;

    root /var/www/cloud-atlas.io;
    index index.html index.htm ;

    server_name cloud-atlas.io alias docs.cloud-atlas.io;

    location / {
        try_files $uri $uri/ =404;
    }
}
  • /var/www/cloud-atlas.io 目录下创建一个测试 index.html 内容如下:

测试 index.html
<html>
    <head>
        <title>Welcome to Cloud Atlas</title>
    </head>
    <body>
        <h1>Success!  The cloud-atlas.io server block is working!</h1>
    </body>
</html>

Let's Encrypt证书

备注

首先需要确保域名( docs.cloud-atlas.io )已经指向了反向代理服务器,也就是我的阿里云公网服务器IP

这个步骤非常重要: Let's Encrypt就是通过域名指向的服务器来确保分发证书是合法的

  • 安装 Certbot :

Debian 安装 Certbot
apt-get update
apt-get install software-properties-common
add-apt-repository ppa:certbot/certbot
apt-get update
apt-get install python3-certbot-nginx
CentOS / Rocky Linux 安装 Certbot
dnf install epel-release
dnf install certbot python3-certbot-nginx
# 如果是Apache,则使用
# dnf install certbot python3-certbot-apache
  • 运行 Certboot :

运行 certbot 为Nginx生成证书
certbot --nginx

执行的输出信息

运行 certbot 为Nginx生成证书
...
Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: cloud-atlas.io
2: docs.cloud-atlas.io
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1 2
Requesting a certificate for cloud-atlas.io and docs.cloud-atlas.io

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/cloud-atlas.io/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/cloud-atlas.io/privkey.pem
This certificate expires on 2025-03-24.
These files will be updated when the certificate renews.

Deploying certificate
Successfully deployed certificate for cloud-atlas.io to /etc/nginx/nginx.conf
Successfully deployed certificate for docs.cloud-atlas.io to /etc/nginx/nginx.conf
Congratulations! You have successfully enabled HTTPS on https://cloud-atlas.io and https://docs.cloud-atlas.io

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

检查 /etc/nginx/conf.d/cloud-atlas.io.conf 可以看到 certbot 修订了配置:

Certbot 自动修订了 /etc/nginx/nginx.conf 包含的 Nginx virtual host配置 配置
server {
    server_name cloud-atlas.io alias docs.cloud-atlas.io;

    root /var/www/cloud-atlas.io;
    index index.html index.htm ;

    location / {
        try_files $uri $uri/ =404;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/cloud-atlas.io/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cloud-atlas.io/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = docs.cloud-atlas.io) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = cloud-atlas.io) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen       80;
    server_name  cloud-atlas.io  alias  docs.cloud-atlas.io;
    return 404; # managed by Certbot
}

现在访问 https://cloud-atlas.io 或者 https://docs.cloud-atlas.io 就可以看到HTTPS的加密已经生效,并且证书是Let's Encrypt 签发的。

证书更新

certbot 提供了自动更新证书的能力,简单执行以下命令可以检查:

Let's Encrypt证书更新检查
# 测试证书更新
certbot renew --dry-run
  • 配置一个定时任务 sudo crontab -e :

配置自动更新Let's Encrypt证书
0 0,12 * * * python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew --quiet

警告

我还没有实践,待补充完善

反向代理

上述已经完成HTTPS基本配置,接下来修订转发规则:

修订转发规则
server {
    server_name cloud-atlas.io alias docs.cloud-atlas.io;

    #root /var/www/cloud-atlas.io;
    index index.html index.htm ;


    location / {
        #try_files $uri $uri/ =404;
        proxy_pass http://127.0.0.1:24180;
        proxy_http_version  1.1;
        proxy_cache_bypass  $http_upgrade;
        proxy_set_header Host              $host;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-Host  $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Port  $server_port;
    }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/cloud-atlas.io/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cloud-atlas.io/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = docs.cloud-atlas.io) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = cloud-atlas.io) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen       80;
    server_name  cloud-atlas.io  alias  docs.cloud-atlas.io;
    return 404; # managed by Certbot
}

注意,这里必须要传递 header ,也就是 proxy_set_header 传递参数非常关键,没有这些参数,后端WEB服务器无法分辨访问域名,也就无法使用 Nginx virtual host配置 来提供合适的返回。

后端服务器

后端服务器运行在一个 Raspberry Pi 主机 192.168.7.241 上,简单的Nginx配置实现一个WEB服务器配置:

后端服务器 nginx 配置
server {
	listen 80;
	listen [::]:80;

	server_name cloud-atlas.io alias docs.cloud-atlas.io;

	root /var/www/cloud-atlas.io;
	index index.html;

	location / {
		try_files $uri $uri/ =404;
	}
}

异常排查

我发现奇怪的问题,在 macOS 上使用 Safari 访问域名 https://docs.cloud-atlas.io/discovery ,有一定几率无法打开页面:

Safari访问网站随机断开连接
Safari Can't Open the Page

Safari can't open the page "https://docs.cloud-atlas.io/discovery/index.html"
because the server unexpectedly dropped the connection.
This somtimes occurs when the server is busy.
Wait for a few minutes, and then try again.

然而,服务器是非常空闲的。有时候刷新safari浏览器又能够看到页面

这个问题困扰我很久,最初我以为是阿里云外网访问的随机问题(但我觉得阿里云这样的头部云厂商不太可能存在这样的bug),所以我尝试更换操作系统(Linux更换到FreeBSD),问题依旧。

今天我更新了Sequioa 15.3版本macOS之后,我突然发现现在safari完全不能打开我的网站 https://docs.cloud-atlas.io/discovery/ ,晕倒...

但是,chrome浏览器访问是正常的 另外,验证了 Firefox 也可以正常访问网站。奇怪的是Safari不行...

这带来一个问题,iOS设备无法访问我的网站,毕竟这是非常主要的手机客户端

我注意到Safari浏览器访问网站时候,Nginxi日志 error.log 显示:

safari浏览器访问时nginx错误日志
==> error.log <==
2025/01/30 13:23:58 [error] 1882#100184: accept4() failed (53: Software caused connection abort)

参考 nginx faq: What does the following error mean in the log file: “accept() failed (53: Software caused connection abort) while accepting new connection on 0.0.0.0:80”? 说明:

这类错误是因为客户端在nginx能够处理之前关闭的连接。例如,当用户没有等待一个包含大量图像的页面完全加载,而是点击了另一个链接是,就会发生这种情况。在这种情况下,用户浏览器将关闭所有不需要的先前连接。 这是一个非关键错误

但是,为何safari客户端会快速关闭页面访问连接,而chrome却能够正常浏览?

警告

还需要排查...

参考