Every web site provides APIs.

gaojiuli, updated 🕥 2022-07-05 22:11:04

Toapi

Build Coverage Python Version License

Overview

Toapi give you the ability to make every web site provides APIs.

Features

  • Automatic converting HTML web site to API service.
  • Automatic caching every page of source site.
  • Automatic caching every request.
  • Support merging multiple web sites into one API service.

Get Started

Installation

text $ pip install toapi

Usage

create app.py and copy the code:

```python from flask import request from htmlparsing import Attr, Text from toapi import Api, Item

api = Api()

@api.site('https://news.ycombinator.com') @api.list('.athing') @api.route('/posts?page={page}', '/news?p={page}') @api.route('/posts', '/news?p=1') class Post(Item): url = Attr('.storylink', 'href') title = Text('.storylink')

@api.site('https://news.ycombinator.com') @api.route('/posts?page={page}', '/news?p={page}') @api.route('/posts', '/news?p=1') class Page(Item): next_page = Attr('.morelink', 'href')

def clean_next_page(self, value):
    return api.convert_string('/' + value, '/news?p={page}', request.host_url.strip('/') + '/posts?page={page}')

api.run(debug=True, host='0.0.0.0', port=5000) ```

run python app.py

then open your browser and visit http://127.0.0.1:5000/posts?page=1

you will get the result like:

json { "Page": { "next_page": "http://127.0.0.1:5000/posts?page=2" }, "Post": [ { "title": "Mathematicians Crack the Cursed Curve", "url": "https://www.quantamagazine.org/mathematicians-crack-the-cursed-curve-20171207/" }, { "title": "Stuffing a Tesla Drivetrain into a 1981 Honda Accord", "url": "https://jalopnik.com/this-glorious-madman-stuffed-a-p85-tesla-drivetrain-int-1823461909" } ] }

Contributing

Write code and test code and pull request.

Issues

Your Documentation is unable to access

opened on 2022-11-22 16:30:21 by ZKluv

well,may you see that your website is not accessed now! Maybe your project is not alive. It is so sad

Upgrade: Bump ujson from 4.0.2 to 5.4.0

opened on 2022-07-05 22:11:00 by dependabot[bot]

Bumps ujson from 4.0.2 to 5.4.0.

Release notes

Sourced from ujson's releases.

5.4.0

Added

Fixed

5.3.0

Added

Changed

Fixed

5.2.0

Added

Fixed

5.1.0

Changed

... (truncated)

Commits
  • 9c20de0 Merge pull request from GHSA-fm67-cv37-96ff
  • b21da40 Fix double free on string decoding if realloc fails
  • 67ec071 Merge pull request #555 from JustAnotherArchivist/fix-decode-surrogates-2
  • bc7bdff Replace wchar_t string decoding implementation with a uint32_t-based one
  • cc70119 Merge pull request #548 from JustAnotherArchivist/arbitrary-ints
  • 4b5cccc Merge pull request #553 from bwoodsend/pypy-ci
  • abe26fc Merge pull request #551 from bwoodsend/bye-bye-travis
  • 3efb5cc Delete old TravisCI workflow and references.
  • 404de1a xfail test_decode_surrogate_characters() on Windows PyPy.
  • f7e66dc Switch to musl docker base images.
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/gaojiuli/toapi/network/alerts).

Fix simple typo: programe -> program

opened on 2020-02-29 19:53:41 by timgates42

Closes #134

Fix simple typo: programe -> program

opened on 2020-02-29 19:53:29 by timgates42

There is a small typo in docs/topics/storage.md. Should read program rather than programe.

Fix missing install requirement

opened on 2019-12-30 05:59:12 by medecau

cssselect PYPI package was not part of the install_requirements list in setup.py

Problem: can't start the app

opened on 2019-09-19 02:20:51 by triangle959

When I ran this example, I reported the following error

2019/09/19 10:15:50 [Register] OK 2019/09/19 10:15:50 [Register] OK 2019/09/19 10:15:50 [Register] OK 2019/09/19 10:15:50 [Register] OK 2019/09/19 10:15:50 [Serving ] OK http://0.0.0.0:5001 2019/09/19 10:15:50 [Serving ] FAIL Windows error 1 2019/09/19 10:15:50 [Serving ] FAIL Traceback (most recent call last): File "D:\python\lib\site-packages\toapi\api.py", line 50, in run self.app.run(host, port, **options) File "D:\python\lib\site-packages\flask\app.py", line 938, in run cli.show_server_banner(self.env, self.debug, self.name, False) File "D:\python\lib\site-packages\flask\cli.py", line 629, in show_server_banner click.echo(message) File "D:\python\lib\site-packages\click\utils.py", line 260, in echo file.write(message) File "D:\python\lib\site-packages\click_winconsole.py", line 180, in write return self._text_stream.write(x) File "D:\python\lib\site-packages\click_winconsole.py", line 164, in write raise OSError(self._get_error_message(GetLastError())) OSError: Windows error 1

toapi, version 2.1.0 Flask 1.0.2 Python 3.6.0

Elliot Gao

Full Stack Developer.

GitHub Repository Homepage

html json api python web spider crawler flask toapi