【nginx报错】upstream timed out (110: Connection timed out) while reading response header from upstream
城风漫漫 2024-10-10 13:37:01 阅读 80
最近在工作上写了一个接口，供ITSM流程调用。
调用这个接口会触发一个任务，处理一些单据数据。因为每天的单据数量不固定，所以这个任务的完成时间也不固定。
接口刚上线时，这个接口的任务完成时间在1分钟以内，直接调用不会有任何问题。
最近因为要处理的单据数量上来了，处理时间超过1分钟了，导致调用接口会有报错，但实际后台任务是可以跑完的。
例如，执行curl请求之后，报错信息如下：
[root@bk-07 ~]# curl http://172.16.1.11:83/sync_cteam_itr
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>The page is temporarily unavailable</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<style type="text/css">
/*<![CDATA[*/
body {
background-color: #fff;
color: #000;
font-size: 0.9em;
font-family: sans-serif,helvetica;
margin: 0;
padding: 0;
}
:link {
color: #c00;
}
:visited {
color: #c00;
}
a:hover {
color: #f50;
}
h1 {
text-align: center;
margin: 0;
padding: 0.6em 2em 0.4em;
background-color: #294172;
color: #fff;
font-weight: normal;
font-size: 1.75em;
border-bottom: 2px solid #000;
}
h1 strong {
font-weight: bold;
font-size: 1.5em;
}
h2 {
text-align: center;
background-color: #3C6EB4;
font-size: 1.1em;
font-weight: bold;
color: #fff;
margin: 0;
padding: 0.5em;
border-bottom: 2px solid #294172;
}
h3 {
text-align: center;
background-color: #ff0000;
padding: 0.5em;
color: #fff;
}
hr {
display: none;
}
.content {
padding: 1em 5em;
}
.alert {
border: 2px solid #000;
}
img {
border: 2px solid #fff;
padding: 2px;
margin: 2px;
}
a:hover img {
border: 2px solid #294172;
}
.logos {
margin: 1em;
text-align: center;
}
/*]]>*/
</style>
</head>
<body>
<h1><strong>nginx error!</strong></h1>
<div class="content">
<h3>The page you are looking for is temporarily unavailable. Please try again later.</h3>
<div class="alert">
<h2>Website Administrator</h2>
<div class="content">
<p>Something has triggered an error on your
website. This is the default error page for
<strong>nginx</strong> that is distributed with
Fedora. It is located
<tt>/usr/share/nginx/html/50x.html</tt></p>
<p>You should customize this error page for your own
site or edit the <tt>error_page</tt> directive in
the <strong>nginx</strong> configuration file
<tt>/etc/nginx/nginx.conf</tt>.</p>
</div>
</div>
<div class="logos">
<a href="http://nginx.net/"><img
src="/nginx-logo.png"
alt="[ Powered by nginx ]"
width="121" height="32" /></a>
<a href="http://fedoraproject.org/"><img
src="/poweredby.png"
alt="[ Powered by Fedora ]"
width="88" height="31" /></a>
</div>
</div>
</body>
</html>
一开始怀疑是nginx层面因上游的uwsgi超过1分钟没返回，然后自动报错返回，故先抛开nginx，直接请求uwsgi的接口，测试发现还是有报错，如下：
[root@bk-07 welink]# curl http://127.0.0.1:5007/sync_cteam_itr
curl: (52) Empty reply from server
查看nginx的日志，如下：
[root@bk-07 welink]# tail /var/log/nginx/access.log
172.16.1.11 - - [29/Nov/2023:00:16:08 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:18:34 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:21:38 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:24:34 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.5 - - [29/Nov/2023:00:31:00 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:35:14 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:42:22 +0800] "GET /sync_cteam_itr HTTP/1.1" 504 3693 "-" "curl/7.29.0"
172.16.1.5 - - [29/Nov/2023:00:45:15 +0800] "GET /reset_cteam_status HTTP/1.1" 200 75 "-" "curl/7.29.0"
172.16.1.11 - - [29/Nov/2023:00:52:22 +0800] "GET /sync_cteam_itr HTTP/1.1" 200 50 "-" "curl/7.29.0"
172.16.1.5 - - [29/Nov/2023:00:54:03 +0800] "GET /sync_cteam_itr HTTP/1.1" 200 51 "-" "curl/7.29.0"
[root@bk-07 welink]#
[root@bk-07 welink]# tail /var/log/nginx/error.log
2023/11/29 00:06:57 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:10:28 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:13:34 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:16:08 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:18:34 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:21:38 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:24:34 [error] 11#11: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:31:00 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.5, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:35:14 [error] 11#11: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
2023/11/29 00:42:22 [error] 10#10: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.1.11, server: localhost, request: "GET /sync_cteam_itr HTTP/1.1", upstream: "http://127.0.0.1:5007/sync_cteam_itr", host: "172.16.1.11:83"
解决方法：
1、修改uwsgi的配置
在uwsgi的配置里添加3个配置项：
harakiri = 600
http-timeout = 600
http-timeout-keep-alive = 600
2、修改nginx的配置
在nginx的http配置块里添加3个配置项：
http {
...
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
...
}
3、重新服务
如果是docker部署的，则重启docker服务。
4、验证结果
[root@bk-07 ~]# curl http://172.16.1.11:83/sync_cteam_itr
任务执行完成，耗时: 77.0742256641388 秒
上一篇： [Linux#61][UDP] port | netstat | udp缓冲区 | stm32
下一篇： ubuntu基本配置
本文标签
【nginx报错】upstream timed out (110: Connection timed out) while reading response header from upstream
声明
本文内容仅代表作者观点，或转载于其他网站，本站不以此文作为商业用途
如有涉及侵权，请联系本站进行删除
转载本站原创文章，请注明来源及作者。