文章目录
  1. 1. 创建隔离独立的Python运行环境
  2. 2. 配置多进程访问Python App的环境
  3. 3. 配置守护进程
    1. 3.1. supervisor
    2. 3.2. 将supervisord设置为系统开机自动启动进程
    3. 3.3. 进程服务重启
    4. 3.4. 补充更新
      1. 3.4.1. 在 Ubuntu 发行版下

使用到的工具链:

  • virtualenv
  • gunicorn
  • supervisord

创建隔离独立的Python运行环境

  1. 使用virtualenv创建独立的应用环境
    virtualenv myProjectName
    如果想指定Python的版本
    virtualenv -p /path/to/python myProjectName

  2. 修改requirements.txt 文件,添加gunicorn和supervisor的安装
    例如:

    1
    2
    3
    4
    ......

    supervisor==3.3.3
    gunicorn==19.7.1
  3. 安装依赖包
    pip install -r requirements.txt

配置多进程访问Python App的环境

创建gunicorn配置文件:
vim gunicorn_config.py

文件模版:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
#
# Server socket
#
# bind - The socket to bind.
#
# A string of the form: 'HOST', 'HOST:PORT', 'unix:PATH'.
# An IP is a valid HOST.
#
# backlog - The number of pending connections. This refers
# to the number of clients that can be waiting to be
# served. Exceeding this number results in the client
# getting an error when attempting to connect. It should
# only affect servers under significant load.
#
# Must be a positive integer. Generally set in the 64-2048
# range.
#
bind = '127.0.0.1:8000'
backlog = 2048

#
# Worker processes
#
# workers - The number of worker processes that this server
# should keep alive for handling requests.
#
# A positive integer generally in the 2-4 x $(NUM_CORES)
# range. You'll want to vary this a bit to find the best
# for your particular application's work load.
#
# worker_class - The type of workers to use. The default
# sync class should handle most 'normal' types of work
# loads. You'll want to read
# http://docs.gunicorn.org/en/latest/design.html#choosing-a-worker-type
# for information on when you might want to choose one
# of the other worker classes.
#
# A string referring to a Python path to a subclass of
# gunicorn.workers.base.Worker. The default provided values
# can be seen at
# http://docs.gunicorn.org/en/latest/settings.html#worker-class
#
# worker_connections - For the eventlet and gevent worker classes
# this limits the maximum number of simultaneous clients that
# a single process can handle.
#
# A positive integer generally set to around 1000.
#
# timeout - If a worker does not notify the master process in this
# number of seconds it is killed and a new worker is spawned
# to replace it.
#
# Generally set to thirty seconds. Only set this noticeably
# higher if you're sure of the repercussions for sync workers.
# For the non sync workers it just means that the worker
# process is still communicating and is not tied to the length
# of time required to handle a single request.
#
# keepalive - The number of seconds to wait for the next request
# on a Keep-Alive HTTP connection.
#
# A positive integer. Generally set in the 1-5 seconds range.
#
workers = 1
# worker_class = 'sync'
# worker_connections = 1000
# timeout = 30
# keepalive = 2

#
# spew - Install a trace function that spews every line of Python
# that is executed when running the server. This is the
# nuclear option.
#
# True or False
#
spew = False

#
# Server mechanics
#
# daemon - Detach the main Gunicorn process from the controlling
# terminal with a standard fork/fork sequence.
#
# True or False
#
# pidfile - The path to a pid file to write
#
# A path string or None to not write a pid file.
#
# user - Switch worker processes to run as this user.
#
# A valid user id (as an integer) or the name of a user that
# can be retrieved with a call to pwd.getpwnam(value) or None
# to not change the worker process user.
#
# group - Switch worker process to run as this group.
#
# A valid group id (as an integer) or the name of a user that
# can be retrieved with a call to pwd.getgrnam(value) or None
# to change the worker processes group.
#
# umask - A mask for file permissions written by Gunicorn. Note that
# this affects unix socket permissions.
#
# A valid value for the os.umask(mode) call or a string
# compatible with int(value, 0) (0 means Python guesses
# the base, so values like "0", "0xFF", "0022" are valid
# for decimal, hex, and octal representations)
#
# tmp_upload_dir - A directory to store temporary request data when
# requests are read. This will most likely be disappearing soon.
#
# A path to a directory where the process owner can write. Or
# None to signal that Python should choose one on its own.
#
daemon = False
pidfile = None
umask = 0
user = None
group = None
tmp_upload_dir = None

#
# Logging
#
# logfile - The path to a log file to write to.
#
# A path string. "-" means log to stdout.
#
# loglevel - The granularity of log output
#
# A string of "debug", "info", "warning", "error", "critical"
#
errorlog = '/tmp/demo_error.log'
loglevel = 'info'
accesslog = '/tmp/demo_access.log'
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

#
# Process naming
#
# proc_name - A base to use with setproctitle to change the way
# that Gunicorn processes are reported in the system process
# table. This affects things like 'ps' and 'top'. If you're
# going to be running more than one instance of Gunicorn you'll
# probably want to set a name to tell them apart. This requires
# that you install the setproctitle module.
#
# A string or None to choose a default of something like 'gunicorn'.
#

proc_name = 'demoServer'


#
# Server hooks
#
# post_fork - Called just after a worker has been forked.
#
# A callable that takes a server and worker instance
# as arguments.
#
# pre_fork - Called just prior to forking the worker subprocess.
#
# A callable that accepts the same arguments as after_fork
#
# pre_exec - Called just prior to forking off a secondary
# master process during things like config reloading.
#
# A callable that takes a server instance as the sole argument.
#

def post_fork(server, worker):
server.log.info("Worker spawned (pid: %s)", worker.pid)


def pre_fork(server, worker):
pass


def pre_exec(server):
server.log.info("Forked child, re-executing.")


def when_ready(server):
server.log.info("Server is ready. Spawning workers")


def worker_int(worker):
worker.log.info("worker received INT or QUIT signal")

## get traceback info
import threading, sys, traceback
id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
code = []
for threadId, stack in sys._current_frames().items():
code.append("\n# Thread: %s(%d)" % (id2name.get(threadId,""),
threadId))
for filename, lineno, name, line in traceback.extract_stack(stack):
code.append('File: "%s", line %d, in %s' % (filename,
lineno, name))
if line:
code.append(" %s" % (line.strip()))
worker.log.debug("\n".join(code))


def worker_abort(worker):
worker.log.info("worker received SIGABRT signal")

如果到这一步不需要守护进程,就只需要在配置文件中将daemon=False 改为daemon=True
然后通过启动gunicorn的命令将应用启动
gunicorn myProjectMainModual:app -c /path/to/myProjectName/gunicorn_config.py
此时应用就已经启动起来了(如果不需要守护进程的情况下)

配置守护进程

在CentOS7环境下使用supervisor Supervisor: A Process Control System 做为gunicorn的守护进程, 以下是如何在CentOS7环境下安装和配置supervisor的过程。

supervisor

  • 安装supervisor

    1. 切换用户到root用户:sudo su -
    2. 使用pip install安装supervisor pip install supervisor
    3. 创建supervisord配置文件 echo_supervisord_conf > /etc/supervisord.conf
  • 配置supervisor

    1. 切换回需要运行supervisord的系统用户exit
    2. 编辑/etc/supervisord.conf配置文件,添加program 节点
      配置文件模版:
    1
    2
    3
    4
    5
    6
    7
    [program:demo]
    command=/usr/local/bin/gunicorn myProjectMainModual:app -c /path/to/myProjectName/gunicorn_config.py
    directory=/path/to/myProjectName
    user=<user_name>
    autostart=true
    autorestart=true
    redirect_stderr=true
  • 启动supervisord

supervisord -c /etc/supervisord.conf

  • 启动supervisorctl 控制配置
    事实上如果是第一次启动,上面启动supervisord的过程就已经将新加载的program节点加载进来了,后续如果想再添加和加载program节点可以通过supervisorctl进行。
    配置好之后通过supervisorctl命令将刚才配置好的需要被守护的程序加载到supervisord中:
    supervisorctl -c /etc/supervisord.conf
    以下以之上配置的demo程序为例
    在supervisorctl的命令行中supervisor>输入:
    supervisor> reread
    supervisor> add demo
    没有提示任何错误后,这个程序就完成了守护进程的配置工作。
    更多supervisorctl指令请参照官方文档supervisorctl Command-Line Options

将supervisord设置为系统开机自动启动进程

  • 创建systemd开机自动脚本

    1. 创建文件
      sudo vim /etc/systemd/system/supervisord.service
    2. 按照以下配置文件模版进行编辑
      Running supervisord automatically on startup
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    # supervisord service for systemd (CentOS 7.0+)
    # by ET-CS (https://github.com/ET-CS)
    # https://github.com/Supervisor/initscripts/blob/master/centos-systemd-etcs
    [Unit]
    Description=Supervisor daemon

    [Service]
    Type=forking
    ExecStart=/usr/bin/supervisord
    ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
    ExecReload=/usr/bin/supervisorctl $OPTIONS reload
    KillMode=process
    Restart=on-failure
    RestartSec=42s

    [Install]
    WantedBy=multi-user.target
  • 启动服务
    sudo systemctl start supervisord

  • 查看服务
    sudo systemctl status supervisord

  • 设置服务为系统开机自动启动
    sudo systemctl enable supervisord

进程服务重启

通过以上配置就形成了 systemd -> supervisord -> gunicorn -> Python Application(Flask)的守护调用关系。
由于supervisord在这里扮演的是守护进程角色,所以WSGI Python 应用进程的意外退出将有supervisord负责重启和守护;
但是由于supervisord本身也是一个系统进程,只是这个进程身份稍微比较特殊而已,也就是说这个进程也是存在意外退出的风险,遇到这样upstream服务不可用的状态可以用sudo systemctl status supervisord来查看服务状态,如果确定是服务进程意外退出可以使用sudo systemctl restart supervisord重启服务。

补充更新

在 Ubuntu 发行版下

在Ubuntu 发行版下,supervisor可以通过 apt install supervisor 的方式安装即可,supervisor 自动就设置为systemd的自启动服务。

文章目录
  1. 1. 创建隔离独立的Python运行环境
  2. 2. 配置多进程访问Python App的环境
  3. 3. 配置守护进程
    1. 3.1. supervisor
    2. 3.2. 将supervisord设置为系统开机自动启动进程
    3. 3.3. 进程服务重启
    4. 3.4. 补充更新
      1. 3.4.1. 在 Ubuntu 发行版下