Mercurial > libervia-backend

annotate libervia/backend/tools/common/async_utils.py @ 4095:684ba556a617

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
core (memory/sqla_mapping): fix legacy pickled values: folloing packages refactoring, legacy pickled values could not be unpickled (due to use of old classes). This temporary workaround fix it, but the right thing to do will be to move from pickle to JSON at some point.
author Goffi <goffi@goffi.org>
date Mon, 12 Jun 2023 14:57:27 +0200
parents 4b842c1fb686
children 0d7bb4df2343
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
2
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
3
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Libervia: an XMPP client
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
6
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
10 # (at your option) any later version.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
11
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
16
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
19
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
20 """tools to launch process in a async way (using Twisted)"""
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
21
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from collections import OrderedDict
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from typing import Optional, Callable, Awaitable
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 3936
diff changeset
24 from libervia.backend.core.log import getLogger
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
25
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
26
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
27 log = getLogger(__name__)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
28
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
29
3936
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
30 def async_lru(maxsize: Optional[int] = 50) -> Callable:
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
31 """Decorator to cache async function results using LRU algorithm
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
32
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
33 @param maxsize: maximum number of items to keep in cache.
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
34 None to have no limit
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
35
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
36 """
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
37 def decorator(func: Callable) -> Callable:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
38 cache = OrderedDict()
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
39 async def wrapper(*args) -> Awaitable:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
40 if args in cache:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
41 log.debug(f"using result in cache for {args}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
42 cache.move_to_end(args)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
43 result = cache[args]
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
44 return result
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
45 log.debug(f"caching result for {args}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
46 result = await func(*args)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
47 cache[args] = result
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
48 if maxsize is not None and len(cache) > maxsize:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
49 value = cache.popitem(False)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
50 log.debug(f"Removing LRU value: {value}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
51 return result
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
52 return wrapper
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
53 return decorator