Asyncio cache manager for redis, memcached and memory

aio-libs, updated 🕥 2023-03-30 19:07:28

aiocache

Asyncio cache supporting multiple backends (memory, redis and memcached).

.. image:: https://travis-ci.org/argaen/aiocache.svg?branch=master :target: https://travis-ci.org/argaen/aiocache

.. image:: https://codecov.io/gh/argaen/aiocache/branch/master/graph/badge.svg :target: https://codecov.io/gh/argaen/aiocache

.. image:: https://badge.fury.io/py/aiocache.svg :target: https://pypi.python.org/pypi/aiocache

.. image:: https://img.shields.io/pypi/pyversions/aiocache.svg :target: https://pypi.python.org/pypi/aiocache

.. image:: https://api.codacy.com/project/badge/Grade/96f772e38e63489ca884dbaf6e9fb7fd :target: https://www.codacy.com/app/argaen/aiocache

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg :target: https://github.com/ambv/black

This library aims for simplicity over specialization. All caches contain the same minimum interface which consists on the following functions:

  • add: Only adds key/value if key does not exist.
  • get: Retrieve value identified by key.
  • set: Sets key/value.
  • multi_get: Retrieves multiple key/values.
  • multi_set: Sets multiple key/values.
  • exists: Returns True if key exists False otherwise.
  • increment: Increment the value stored in the given key.
  • delete: Deletes key and returns number of deleted items.
  • clear: Clears the items stored.
  • raw: Executes the specified command using the underlying client.

.. role:: python(code) :language: python

.. contents::

.. section-numbering:

Installing

  • pip install aiocache
  • pip install aiocache[redis]
  • pip install aiocache[memcached]
  • pip install aiocache[redis,memcached]
  • pip install aiocache[msgpack]

Usage

Using a cache is as simple as

.. code-block:: python

>>> import asyncio
>>> from aiocache import Cache
>>> cache = Cache(Cache.MEMORY) # Here you can also use Cache.REDIS and Cache.MEMCACHED, default is Cache.MEMORY
>>> with asyncio.Runner() as runner:
>>>     runner.run(cache.set('key', 'value'))
True
>>>     runner.run(cache.get('key'))
'value'

Or as a decorator

.. code-block:: python

import asyncio

from collections import namedtuple

from aiocache import cached, Cache
from aiocache.serializers import PickleSerializer
# With this we can store python objects in backends like Redis!

Result = namedtuple('Result', "content, status")


@cached(
    ttl=10, cache=Cache.REDIS, key="key", serializer=PickleSerializer(), port=6379, namespace="main")
async def cached_call():
    print("Sleeping for three seconds zzzz.....")
    await asyncio.sleep(3)
    return Result("content", 200)


async def run():
    await cached_call()
    await cached_call()
    await cached_call()
    cache = Cache(Cache.REDIS, endpoint="127.0.0.1", port=6379, namespace="main")
    await cache.delete("key")

if __name__ == "__main__":
    asyncio.run(run())

The recommended approach to instantiate a new cache is using the Cache constructor. However you can also instantiate directly using aiocache.RedisCache, aiocache.SimpleMemoryCache or aiocache.MemcachedCache.

You can also setup cache aliases so its easy to reuse configurations

.. code-block:: python

import asyncio

from aiocache import caches

# You can use either classes or strings for referencing classes caches.set_config({ 'default': { 'cache': "aiocache.SimpleMemoryCache", 'serializer': { 'class': "aiocache.serializers.StringSerializer" } }, 'redis_alt': { 'cache': "aiocache.RedisCache", 'endpoint': "127.0.0.1", 'port': 6379, 'timeout': 1, 'serializer': { 'class': "aiocache.serializers.PickleSerializer" }, 'plugins': [ {'class': "aiocache.plugins.HitMissRatioPlugin"}, {'class': "aiocache.plugins.TimingPlugin"} ] } })

async def default_cache(): cache = caches.get('default') # This always returns the SAME instance await cache.set("key", "value") assert await cache.get("key") == "value"

async def alt_cache(): cache = caches.create('redis_alt') # This creates a NEW instance on every call await cache.set("key", "value") assert await cache.get("key") == "value"

async def test_alias(): await default_cache() await alt_cache()

  await caches.get("redis_alt").delete("key")

if name == "main": asyncio.run(test_alias())

How does it work

Aiocache provides 3 main entities:

  • backends: Allow you specify which backend you want to use for your cache. Currently supporting: SimpleMemoryCache, RedisCache using redis_ and MemCache using aiomcache_.
  • serializers: Serialize and deserialize the data between your code and the backends. This allows you to save any Python object into your cache. Currently supporting: StringSerializer, PickleSerializer, JsonSerializer, and MsgPackSerializer. But you can also build custom ones.
  • plugins: Implement a hooks system that allows to execute extra behavior before and after of each command.

If you are missing an implementation of backend, serializer or plugin you think it could be interesting for the package, do not hesitate to open a new issue.

.. image:: docs/images/architecture.png :align: center

Those 3 entities combine during some of the cache operations to apply the desired command (backend), data transformation (serializer) and pre/post hooks (plugins). To have a better vision of what happens, here you can check how set function works in aiocache:

.. image:: docs/images/set_operation_flow.png :align: center

Amazing examples

In examples folder <https://github.com/argaen/aiocache/tree/master/examples>_ you can check different use cases:

  • Sanic, Aiohttp and Tornado <https://github.com/argaen/aiocache/tree/master/examples/frameworks>_
  • Python object in Redis <https://github.com/argaen/aiocache/blob/master/examples/python_object.py>_
  • Custom serializer for compressing data <https://github.com/argaen/aiocache/blob/master/examples/serializer_class.py>_
  • TimingPlugin and HitMissRatioPlugin demos <https://github.com/argaen/aiocache/blob/master/examples/plugins.py>_
  • Using marshmallow as a serializer <https://github.com/argaen/aiocache/blob/master/examples/marshmallow_serializer_class.py>_
  • Using cached decorator <https://github.com/argaen/aiocache/blob/master/examples/cached_decorator.py>_.
  • Using multi_cached decorator <https://github.com/argaen/aiocache/blob/master/examples/multicached_decorator.py>_.

Documentation

  • Usage <http://aiocache.readthedocs.io/en/latest>_
  • Caches <http://aiocache.readthedocs.io/en/latest/caches.html>_
  • Serializers <http://aiocache.readthedocs.io/en/latest/serializers.html>_
  • Plugins <http://aiocache.readthedocs.io/en/latest/plugins.html>_
  • Configuration <http://aiocache.readthedocs.io/en/latest/configuration.html>_
  • Decorators <http://aiocache.readthedocs.io/en/latest/decorators.html>_
  • Testing <http://aiocache.readthedocs.io/en/latest/testing.html>_
  • Examples <https://github.com/argaen/aiocache/tree/master/examples>_

.. _redis: https://github.com/redis/redis-py .. _aiomcache: https://github.com/aio-libs/aiomcache

Issues

Bump types-redis from 4.5.3.1 to 4.5.4.1

opened on 2023-03-30 19:07:27 by dependabot[bot]

Bumps types-redis from 4.5.3.1 to 4.5.4.1.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Library is unusable with redis older than 4.2.0 installed, even if redis isn't used

opened on 2023-03-16 09:01:23 by gcbirzan-plutoflume

The Python redis library didn't have async support before 4.2.0, but the check for whether redis is installed doesn't take that into account: https://github.com/aio-libs/aiocache/blob/19e04665dfd1fad00c2d3ab9061a81db9acd368e/aiocache/init.py#L14

If redis is importable, the code then goes on to import redis.asyncio, which for older versions doesn't exist: https://github.com/aio-libs/aiocache/blob/19e04665dfd1fad00c2d3ab9061a81db9acd368e/aiocache/backends/redis.py#L5

While, technically, this isn't something that one should do, we hit this problem because of our complicated dependencies, where an internal library we use in our async code also depends on an older redis, even though that part is not used by our async code.

I think rather than hardcoding a version check, an import redis.asyncio in the try/except block.

Clear cache if it exists upon setting alias

opened on 2023-03-16 03:15:07 by cancan101

What do these changes do?

For consistency with set_config

Are there changes in behavior for the user?

Related issue number

Checklist

  • [ ] I think the code is well written
  • [ ] Unit tests for the changes exist
  • [ ] Documentation reflects the changes
  • [ ] Add a new news fragment into the CHANGES folder
  • name it <issue_id>.<type> (e.g. 588.bugfix)
  • if you don't have an issue_id change it to the pr id after creating the PR
  • ensure type is one of the following:
    • .feature: Signifying a new feature.
    • .bugfix: Signifying a bug fix.
    • .doc: Signifying a documentation improvement.
    • .removal: Signifying a deprecation or removal of public API.
    • .misc: A ticket has been closed, but it is not of interest to users.
  • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

add ssl support

opened on 2023-03-15 05:39:44 by cancan101

What do these changes do?

Are there changes in behavior for the user?

Related issue number

Checklist

  • [ ] I think the code is well written
  • [ ] Unit tests for the changes exist
  • [ ] Documentation reflects the changes
  • [ ] Add a new news fragment into the CHANGES folder
  • name it <issue_id>.<type> (e.g. 588.bugfix)
  • if you don't have an issue_id change it to the pr id after creating the PR
  • ensure type is one of the following:
    • .feature: Signifying a new feature.
    • .bugfix: Signifying a bug fix.
    • .doc: Signifying a documentation improvement.
    • .removal: Signifying a deprecation or removal of public API.
    • .misc: A ticket has been closed, but it is not of interest to users.
  • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

Merge cache implementation classes for each backend

opened on 2023-03-04 20:35:26 by pshafer-als

Background

Each backend cache implementation is currently split into two classes, with the "interface" class derived from the "backend" class.

  • SimpleMemoryCache subclasses SimpleMemoryBackend, which subclasses BaseCache[str].
  • MemcachedCache subclasses MemcachedBackend, which subclasses BaseCache[bytes].
  • RedisCache subclasses RedisBackend, which subclasses BaseCache[str].

Suggestion

Merge each pair into a single subclass of BaseClass. In other words, the ...Backend classes do not appear to provide any benefit or value by being distinct classes.

  • SimpleMemoryCache is highly bound to SimpleMemoryBackend, even in its name. It seems unlikely that anyone would use SimpleMemoryBackend directly, rather than using SimpleMemoryCache.
  • Also SimpleMemoryBackend does not appear to provide value as a base class for user customization. If a specialized memory cache is needed, one would probably derive from SimpleMemoryCache and override only what is needed, or derive from BaseCache if more extensive changes are required.

Question

Are any aiocache users relying on SimpleMemoryBackend, MemcachedBackend, RedisBackend classes, or could their functionality be merged into the corresponding ...Cache implementations?

Remove factory

opened on 2023-02-27 19:31:55 by Dreamsorcerer

factoyr.py can probably be removed. It's overly complicated and causes typing issues while seeming to provide no benefit at all.

I think it's probably preferable to simply import the caches the user wants explicitly: from aiocache.backends.redis import RedisCache

Releases

v0.12.0 2023-01-13 20:41:32

  • Added async with support to BaseCache.
  • Added initial typing support.
  • Migrated to redis library (aioredis is no longer supported).
  • SimpleMemoryBackend now has a cache per instance, rather than a global cache.
  • Improved support for build_key(key, namespace) #569 -- Padraic Shafer
  • Removed deprecated loop parameters.
  • Removed deprecated cache parameter from create().
  • Added support for keyword arguments in TimingPlugin methods.
  • Fixed inconsistent enum keys between different Python versions. -- Padraic Shafer
  • Fixed .clear() breaking when no keys are present.
  • Fixed from aiocache import *.
  • Fixed .delete() when values are falsy.

0.11.1 2019-07-31 04:54:11

  • Don't hardcode import redis and memcached in factory #461 - Manuel Miranda

0.11.0 2019-07-31 02:37:08

  • Support str for timeout and ttl #454 - Manuel Miranda
  • Add aiocache_wait_for_write decorator param #448 - Manuel Miranda
  • Extend and improve usage of Cache class #446 - Manuel Miranda
  • Add caches.add functionality #440 - Manuel Miranda
  • Use raw msgpack attribute for loads #439 - Manuel Miranda
  • Add docs regarding plugin timeouts and multicached #438 - Manuel Miranda
  • Fix typehints in lock.py #434 - Aviv
  • Use pytest_configure instead of pytest_namespace #436 - Manuel Miranda
  • Add Cache class factory #430 - Manuel Miranda

0.10.1 2018-11-15 21:45:01

  • Cancel the previous ttl timer if exists when setting a new value in the in-memory cache #424 - Minh Tu Le

  • Add python 3.7 to CI, now its supported! #420 - Manuel Miranda

  • Add function as parameter for key_builder #417 - Manuel Miranda

  • Always use name when getting logger #412 - Mansur Mamkin

  • Format code with black #410 - Manuel Miranda

0.10.0 2018-06-17 21:09:29

  • Cache can be disabled in decorated functions using cache_read and cache_write #404 - Josep Cugat

  • Cache constructor can receive now default ttl #405 - Josep Cugat

0.9.1 2018-04-26 22:37:58

  • Single deploy step #395 - Manuel Miranda

  • Catch ImportError when importing optional msgpack #398 - Paweł Kowalski

  • Lazy load redis asyncio.Lock #397 - Jordi Soucheiron

aio-libs

The set of asyncio-based libraries built with high quality

GitHub Repository Homepage

python-3 asyncio redis memcached cache cachemanager