基于flask+request小说爬取项目

longzx-9527, updated 🕥 2022-12-08 00:59:03

基于flask+requests个人博客系统

1.基本环境搭建

python 1.本人使用的系统是 Centos7 2.python环境 2.1 安装python3.6 2.2 安装pip工具 3.安装mysql数据库使用的是mysql 5.7 charset=utf8 4.建立相关数据库及表

2.安装教程(推荐安装环境：Centos7,python版本要超过3.4)

1.git clone https://github.com/longzx-9527/flask_spider.git
2.cd flask_spider
3.pyvenv  myvenv #虚拟环境
4.source venv/bin/activate #激活虚拟环境
5.pip install -r  requirement.txt # 安装依赖

以上，应该安装好了python依赖包。接下来是初始化数据：

1.首先你应该创建了一个blog数据库（utf-8格式）,然后修改config.py里面的user、passwd、db
2.初始化数据库：python manage.py db init
3.生成数据库语句：python manage.py db migrate
4.创建数据库：python manage.py upgrade

运行：./start.sh

2.个人博客首页

2.1 首页界面

2.2 可以发布一些自己写的文章

写文章

2.3 文章显示

文章显示

2.4 文章管理

文章显示

3.小说爬取展示

最终实现效果如下图：

首页显示

可以输入查询小说，如果小说不存在，就调用后台爬虫程序下载

章节列表

点开具体页面显示，小说章节列表，对于每个章节，如果本地没有就直接下载，可以点开具体章节开心的阅读，而没有广告，是的没有广告，纯净的

章节内容

Issues

Bump certifi from 2018.1.18 to 2022.12.7

opened on 2022-12-08 00:59:02 by dependabot[bot]

Bumps certifi from 2018.1.18 to 2022.12.7.

Commits

9e9e840 2022.12.07
b81bdb2 2022.09.24
939a28f 2022.09.14
aca828a 2022.06.15.2
de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...
b8eb5e9 2022.06.15.1
47fb7ab Fix deprecation warning on Python 3.11 (#199)
b0b48e0 fixes #198 -- update link in license
9d514b4 2022.06.15
4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/longzx-9527/flask_spider/network/alerts).

Bump mako from 1.0.7 to 1.2.2

opened on 2022-09-16 17:48:02 by dependabot[bot]

Bumps mako from 1.0.7 to 1.2.2.

Release notes

Sourced from mako's releases.

1.2.2

Released: Mon Aug 29 2022

bug

[bug] [lexer] Fixed issue in lexer where the regexp used to match tags would not correctly interpret quoted sections individually. While this parsing issue still produced the same expected tag structure later on, the mis-handling of quoted sections was also subject to a regexp crash if a tag had a large number of quotes within its quoted sections.

References: #366

1.2.1

Released: Thu Jun 30 2022

bug

[bug] [tests] Various fixes to the test suite in the area of exception message rendering to accommodate for variability in Python versions as well as Pygments.

References: #360

misc

[performance] Optimized some codepaths within the lexer/Python code generation process, improving performance for generation of templates prior to their being cached. Pull request courtesy Takuto Ikuta.

References: #361

1.2.0

Released: Thu Mar 10 2022

changed

[changed] [py3k] Corrected "universal wheel" directive in setup.cfg so that building a wheel does not target Python 2.

References: #351

[changed] [py3k] The bytestring_passthrough template argument is removed, as this flag only applied to Python 2.

... (truncated)

Commits

See full diff in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump lxml from 4.2.1 to 4.9.1

opened on 2022-07-06 19:47:42 by dependabot[bot]

Bumps lxml from 4.2.1 to 4.9.1.

Changelog

Sourced from lxml's changelog.

4.9.1 (2022-07-01)

Bugs fixed

A crash was resolved when using iterwalk() (or canonicalize()) after parsing certain incorrect input. Note that iterwalk() can crash on valid input parsed with the same parser after failing to parse the incorrect input.

4.9.0 (2022-06-01)

Bugs fixed

GH#341: The mixin inheritance order in lxml.html was corrected. Patch by xmo-odoo.

Other changes

Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.

Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35 (libxml2 2.9.12+ and libxslt 1.1.34 on Windows).

GH#343: Windows-AArch64 build support in Visual Studio. Patch by Steve Dower.

4.8.0 (2022-02-17)

Features added

GH#337: Path-like objects are now supported throughout the API instead of just strings. Patch by Henning Janssen.

The ElementMaker now supports QName values as tags, which always override the default namespace of the factory.

Bugs fixed

GH#338: In lxml.objectify, the XSI float annotation "nan" and "inf" were spelled in lower case, whereas XML Schema datatypes define them as "NaN" and "INF" respectively.

... (truncated)

Commits

d01872c Prevent parse failure in new test from leaking into later test runs.
d65e632 Prepare release of lxml 4.9.1.
86368e9 Fix a crash when incorrect parser input occurs together with usages of iterwa...
50c2764 Delete unused Travis CI config and reference in docs (GH-345)
8f0bf2d Try to speed up the musllinux AArch64 build by splitting the different CPytho...
b9f7074 Remove debug print from test.
b224e0f Try to install 'xz' in wheel builds, if available, since it's now needed to e...
897ebfa Update macOS deployment target version from 10.14 to 10.15 since 10.14 starts...
853c9e9 Prepare release of 4.9.0.
d3f77e6 Add a test for https://bugs.launchpad.net/lxml/+bug/1965070 leaving out the a...
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump ipython from 6.2.1 to 7.16.3

opened on 2022-01-21 19:11:14 by dependabot[bot]

Bumps ipython from 6.2.1 to 7.16.3.

Commits

d43c7c7 release 7.16.3
5fa1e40 Merge pull request from GHSA-pq7m-3gw7-gq5x
8df8971 back to dev
9f477b7 release 7.16.2
138f266 bring back release helper from master branch
5aa3634 Merge pull request #13341 from meeseeksmachine/auto-backport-of-pr-13335-on-7...
bcae8e0 Backport PR #13335: What's new 7.16.2
8fcdcd3 Pin Jedi to <0.17.2.
2486838 release 7.16.1
20bdc6f fix conda build
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump markdown2 from 2.3.5 to 2.4.0

opened on 2021-06-02 21:46:49 by dependabot[bot]

Bumps markdown2 from 2.3.5 to 2.4.0.

Changelog

Sourced from markdown2's changelog.

python-markdown2 2.4.0

[pull #377] Fixed bug breaking strings elements in metadata lists

[pull #380] When rendering fenced code blocks, also add the language-LANG class

[pull #387] Regex DoS fixes

python-markdown2 2.3.10

[pull #356] Don't merge sequential quotes into a single blockquote

[pull #357] use style=text-align for table alignment

[pull #360] introduce underline extra

[pull #368] Support for structured and nested values in metadata

[pull #371] add noopener to external links

python-markdown2 2.3.9

[pull #335] Added header support for wiki tables

[pull #336] Reset _toc when convert is run

[pull #353] XSS fix

[pull #350] XSS fix

python-markdown2 2.3.8

[pull #317] Temporary fix to issue #150

[pull #319] Stop XML escaping the body of a link

[pull #322] Don't auto link patterns surrounded by triple quotes

[pull #324] Add class configurability to the enclosing tag

[pull #328] Accept [X] as marked task

python-markdown2 2.3.7

[pull #306] Drop support for legacy Python versions

[pull #307] Fix syntax highlighting test cases that depend on Pygments output

[pull #308] Add support for Python 3.7

[pull #304] Add Wheel package support

[pull #312] Fix toc_depth initialization regression

[pull #315] XSS fix

python-markdown2 2.3.6

[pull #282] Add TOC depth option

[pull #283] Fix to add TOC html to output via CLI

[pull #284] Do not remove anchors in safe_mode

[pull #288] fixing cuddled-lists with a single list item

[pull #292] Fix Wrong rendering of last list element

... (truncated)

Commits

3149185 prepare for 2.4.0 release
887e958 Update CHANGES.md
7b65126 Merge pull request #387 from trentm/regex-dos
c4b4ccb Be forgiving
e1954d3 Pretty comment alignment
96dff22 Regex DOS fixes
330d34c Merge pull request #381 from timgates42/bugfix_typo_because
b98813f docs: fix simple typo, becase -> because
cf9c53d Merge pull request #380 from dkasak/fenced-code-block-additional-tag
b9e3ec7 Pygments test fix for new version
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump urllib3 from 1.22 to 1.26.5

opened on 2021-06-01 22:10:47 by dependabot[bot]

Bumps urllib3 from 1.22 to 1.26.5.

Release notes

Sourced from urllib3's releases.

1.26.5

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Fixed deprecation warnings emitted in Python 3.10.

Updated vendored six library to 1.16.0.

Improved performance of URL parser when splitting the authority component.

If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

1.26.4

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

1.26.3

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Fixed bytes and string comparison issue with headers (Pull #2141)

Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme (Pull #2107)

If you or your organization rely on urllib3 consider supporting us via GitHub Sponsors

1.26.2

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

1.26.1

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

1.26.0

:warning: IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning should opt-in explicitly by setting ssl_version=ssl.PROTOCOL_TLSv1_1 (Pull #2002) Starting in urllib3 v2.0: Connections that receive a DeprecationWarning will fail

Deprecated Retry options Retry.DEFAULT_METHOD_WHITELIST, Retry.DEFAULT_REDIRECT_HEADERS_BLACKLIST and Retry(method_whitelist=...) in favor of Retry.DEFAULT_ALLOWED_METHODS, Retry.DEFAULT_REMOVE_HEADERS_ON_REDIRECT, and Retry(allowed_methods=...) (Pull #2000) Starting in urllib3 v2.0: Deprecated options will be removed

... (truncated)

Changelog

Sourced from urllib3's changelog.

1.26.5 (2021-05-26)

Fixed deprecation warnings emitted in Python 3.10.

Updated vendored six library to 1.16.0.

Improved performance of URL parser when splitting the authority component.

1.26.4 (2021-03-15)

Changed behavior of the default SSLContext when connecting to HTTPS proxy during HTTPS requests. The default SSLContext now sets check_hostname=True.

1.26.3 (2021-01-26)

Fixed bytes and string comparison issue with headers (Pull #2141)

Changed ProxySchemeUnknown error message to be more actionable if the user supplies a proxy URL without a scheme. (Pull #2107)

1.26.2 (2020-11-12)

Fixed an issue where wrap_socket and CERT_REQUIRED wouldn't be imported properly on Python 2.7.8 and earlier (Pull #2052)

1.26.1 (2020-11-11)

Fixed an issue where two User-Agent headers would be sent if a User-Agent header key is passed as bytes (Pull #2047)

1.26.0 (2020-11-10)

NOTE: urllib3 v2.0 will drop support for Python 2. Read more in the v2.0 Roadmap <https://urllib3.readthedocs.io/en/latest/v2-roadmap.html>_.

Added support for HTTPS proxies contacting HTTPS servers (Pull #1923, Pull #1806)

Deprecated negotiating TLSv1 and TLSv1.1 by default. Users that still wish to use TLS earlier than 1.2 without a deprecation warning

... (truncated)

Commits

d161647 Release 1.26.5
2d4a3fe Improve performance of sub-authority splitting in URL
2698537 Update vendored six to 1.16.0
07bed79 Fix deprecation warnings for Python 3.10 ssl module
d725a9b Add Python 3.10 to GitHub Actions
339ad34 Use pytest==6.2.4 on Python 3.10+
f271c9c Apply latest Black formatting
1884878 [1.26] Properly proxy EOF on the SSLTransport test suite
a891304 Release 1.26.4
8d65ea1 Merge pull request from GHSA-5phf-pp7p-vc2r
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

longzx-9527

GitHub Repository