部署一个标准Python（Flask）应用的备忘录

文章目录

1. 1. 创建隔离独立的Python运行环境
2. 2. 配置多进程访问Python App的环境
3. 3. 配置守护进程
   1. 3.1. supervisor
   2. 3.2. 将supervisord设置为系统开机自动启动进程
   3. 3.3. 进程服务重启
   4. 3.4. 补充更新
      1. 3.4.1. 在 Ubuntu 发行版下

使用到的工具链：

• virtualenv
• gunicorn
• supervisord

创建隔离独立的Python运行环境

1. 使用virtualenv创建独立的应用环境
   virtualenv myProjectName
   如果想指定Python的版本
   virtualenv -p /path/to/python myProjectName

2. 修改requirements.txt 文件，添加gunicorn和supervisor的安装
   例如:

   ......
   
   supervisor==3.3.3
   gunicorn==19.7.1

3. 安装依赖包
   pip install -r requirements.txt

配置多进程访问Python App的环境

创建gunicorn配置文件：
vim gunicorn_config.py

文件模版：

#
# Server socket
#
#   bind - The socket to bind.
#
#       A string of the form: 'HOST', 'HOST:PORT', 'unix:PATH'.
#       An IP is a valid HOST.
#
#   backlog - The number of pending connections. This refers
#       to the number of clients that can be waiting to be
#       served. Exceeding this number results in the client
#       getting an error when attempting to connect. It should
#       only affect servers under significant load.
#
#       Must be a positive integer. Generally set in the 64-2048
#       range.
#
bind = '127.0.0.1:8000'
backlog = 2048

#
# Worker processes
#
#   workers - The number of worker processes that this server
#       should keep alive for handling requests.
#
#       A positive integer generally in the 2-4 x $(NUM_CORES)
#       range. You'll want to vary this a bit to find the best
#       for your particular application's work load.
#
#   worker_class - The type of workers to use. The default
#       sync class should handle most 'normal' types of work
#       loads. You'll want to read
#       http://docs.gunicorn.org/en/latest/design.html#choosing-a-worker-type
#       for information on when you might want to choose one
#       of the other worker classes.
#
#       A string referring to a Python path to a subclass of
#       gunicorn.workers.base.Worker. The default provided values
#       can be seen at
#       http://docs.gunicorn.org/en/latest/settings.html#worker-class
#
#   worker_connections - For the eventlet and gevent worker classes
#       this limits the maximum number of simultaneous clients that
#       a single process can handle.
#
#       A positive integer generally set to around 1000.
#
#   timeout - If a worker does not notify the master process in this
#       number of seconds it is killed and a new worker is spawned
#       to replace it.
#
#       Generally set to thirty seconds. Only set this noticeably
#       higher if you're sure of the repercussions for sync workers.
#       For the non sync workers it just means that the worker
#       process is still communicating and is not tied to the length
#       of time required to handle a single request.
#
#   keepalive - The number of seconds to wait for the next request
#       on a Keep-Alive HTTP connection.
#
#       A positive integer. Generally set in the 1-5 seconds range.
#
workers = 1
# worker_class = 'sync'
# worker_connections = 1000
# timeout = 30
# keepalive = 2

#
#   spew - Install a trace function that spews every line of Python
#       that is executed when running the server. This is the
#       nuclear option.
#
#       True or False
#
spew = False

#
# Server mechanics
#
#   daemon - Detach the main Gunicorn process from the controlling
#       terminal with a standard fork/fork sequence.
#
#       True or False
#
#   pidfile - The path to a pid file to write
#
#       A path string or None to not write a pid file.
#
#   user - Switch worker processes to run as this user.
#
#       A valid user id (as an integer) or the name of a user that
#       can be retrieved with a call to pwd.getpwnam(value) or None
#       to not change the worker process user.
#
#   group - Switch worker process to run as this group.
#
#       A valid group id (as an integer) or the name of a user that
#       can be retrieved with a call to pwd.getgrnam(value) or None
#       to change the worker processes group.
#
#   umask - A mask for file permissions written by Gunicorn. Note that
#       this affects unix socket permissions.
#
#       A valid value for the os.umask(mode) call or a string
#       compatible with int(value, 0) (0 means Python guesses
#       the base, so values like "0", "0xFF", "0022" are valid
#       for decimal, hex, and octal representations)
#
#   tmp_upload_dir - A directory to store temporary request data when
#       requests are read. This will most likely be disappearing soon.
#
#       A path to a directory where the process owner can write. Or
#       None to signal that Python should choose one on its own.
#
daemon = False
pidfile = None
umask = 0
user = None
group = None
tmp_upload_dir = None

#
#   Logging
#
#   logfile - The path to a log file to write to.
#
#       A path string. "-" means log to stdout.
#
#   loglevel - The granularity of log output
#
#       A string of "debug", "info", "warning", "error", "critical"
#
errorlog = '/tmp/demo_error.log'
loglevel = 'info'
accesslog = '/tmp/demo_access.log'
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

#
# Process naming
#
#   proc_name - A base to use with setproctitle to change the way
#       that Gunicorn processes are reported in the system process
#       table. This affects things like 'ps' and 'top'. If you're
#       going to be running more than one instance of Gunicorn you'll
#       probably want to set a name to tell them apart. This requires
#       that you install the setproctitle module.
#
#       A string or None to choose a default of something like 'gunicorn'.
#

proc_name = 'demoServer'


#
# Server hooks
#
#   post_fork - Called just after a worker has been forked.
#
#       A callable that takes a server and worker instance
#       as arguments.
#
#   pre_fork - Called just prior to forking the worker subprocess.
#
#       A callable that accepts the same arguments as after_fork
#
#   pre_exec - Called just prior to forking off a secondary
#       master process during things like config reloading.
#
#       A callable that takes a server instance as the sole argument.
#

def post_fork(server, worker):
    server.log.info("Worker spawned (pid: %s)", worker.pid)


def pre_fork(server, worker):
    pass


def pre_exec(server):
    server.log.info("Forked child, re-executing.")


def when_ready(server):
    server.log.info("Server is ready. Spawning workers")


def worker_int(worker):
    worker.log.info("worker received INT or QUIT signal")

    ## get traceback info
    import threading, sys, traceback
    id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
    code = []
    for threadId, stack in sys._current_frames().items():
        code.append("\n# Thread: %s(%d)" % (id2name.get(threadId,""),
            threadId))
        for filename, lineno, name, line in traceback.extract_stack(stack):
            code.append('File: "%s", line %d, in %s' % (filename,
                lineno, name))
            if line:
                code.append("  %s" % (line.strip()))
    worker.log.debug("\n".join(code))


def worker_abort(worker):
    worker.log.info("worker received SIGABRT signal")

如果到这一步不需要守护进程，就只需要在配置文件中将daemon=False 改为daemon=True
然后通过启动gunicorn的命令将应用启动
gunicorn myProjectMainModual:app -c /path/to/myProjectName/gunicorn_config.py
此时应用就已经启动起来了（如果不需要守护进程的情况下）

配置守护进程

在CentOS7环境下使用supervisor Supervisor: A Process Control System 做为gunicorn的守护进程， 以下是如何在CentOS7环境下安装和配置supervisor的过程。

supervisor

• 安装supervisor

  1. 切换用户到root用户：sudo su -
  2. 使用pip install安装supervisor pip install supervisor
  3. 创建supervisord配置文件 echo_supervisord_conf > /etc/supervisord.conf

• 配置supervisor

  1. 切换回需要运行supervisord的系统用户exit
  2. 编辑/etc/supervisord.conf配置文件，添加program 节点
     配置文件模版：

  [program:demo]
  command=/usr/local/bin/gunicorn myProjectMainModual:app -c /path/to/myProjectName/gunicorn_config.py
  directory=/path/to/myProjectName
  user=<user_name>
  autostart=true
  autorestart=true
  redirect_stderr=true

• 启动supervisord

supervisord -c /etc/supervisord.conf

• 启动supervisorctl 控制配置
  事实上如果是第一次启动，上面启动supervisord的过程就已经将新加载的program节点加载进来了，后续如果想再添加和加载program节点可以通过supervisorctl进行。
  配置好之后通过supervisorctl命令将刚才配置好的需要被守护的程序加载到supervisord中:
  supervisorctl -c /etc/supervisord.conf
  以下以之上配置的demo程序为例
  在supervisorctl的命令行中supervisor>输入：
  supervisor> reread
supervisor> add demo
  没有提示任何错误后，这个程序就完成了守护进程的配置工作。
  更多supervisorctl指令请参照官方文档supervisorctl Command-Line Options

将supervisord设置为系统开机自动启动进程

• 创建systemd开机自动脚本

  1. 创建文件
     sudo vim /etc/systemd/system/supervisord.service
  2. 按照以下配置文件模版进行编辑
     Running supervisord automatically on startup

  # supervisord service for systemd (CentOS 7.0+)
  # by ET-CS (https://github.com/ET-CS)
  # https://github.com/Supervisor/initscripts/blob/master/centos-systemd-etcs
  [Unit]
  Description=Supervisor daemon
  
  [Service]
  Type=forking
  ExecStart=/usr/bin/supervisord
  ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
  ExecReload=/usr/bin/supervisorctl $OPTIONS reload
  KillMode=process
  Restart=on-failure
  RestartSec=42s
  
  [Install]
  WantedBy=multi-user.target

• 启动服务
  sudo systemctl start supervisord

• 查看服务
  sudo systemctl status supervisord

• 设置服务为系统开机自动启动
  sudo systemctl enable supervisord

进程服务重启

通过以上配置就形成了 systemd -> supervisord -> gunicorn -> Python Application(Flask)的守护调用关系。
由于supervisord在这里扮演的是守护进程角色，所以WSGI Python 应用进程的意外退出将有supervisord负责重启和守护；
但是由于supervisord本身也是一个系统进程，只是这个进程身份稍微比较特殊而已，也就是说这个进程也是存在意外退出的风险，遇到这样upstream服务不可用的状态可以用sudo systemctl status supervisord来查看服务状态，如果确定是服务进程意外退出可以使用sudo systemctl restart supervisord重启服务。

补充更新

在 Ubuntu 发行版下

在Ubuntu 发行版下，supervisor可以通过 apt install supervisor 的方式安装即可，supervisor 自动就设置为systemd的自启动服务。