Mercurial > libervia-backend
annotate libervia/backend/tools/common/async_utils.py @ 4306:94e0968987cd
plugin XEP-0033: code modernisation, improve delivery, data validation:
- Code has been rewritten using Pydantic models and `async` coroutines for data validation
and cleaner element parsing/generation.
- Delivery has been completely rewritten. It now works even if server doesn't support
multicast, and send to local multicast service first. Delivering to local multicast
service first is due to bad support of XEP-0033 in server (notably Prosody which has an
incomplete implementation), and the current impossibility to detect if a sub-domain
service handles fully multicast or only for local domains. This is a workaround to have
a good balance between backward compatilibity and use of bandwith, and to make it work
with the incoming email gateway implementation (the gateway will only deliver to
entities of its own domain).
- disco feature checking now uses `async` corountines. `host` implementation still use
Deferred return values for compatibility with legacy code.
rel 450
author | Goffi <goffi@goffi.org> |
---|---|
date | Thu, 26 Sep 2024 16:12:01 +0200 |
parents | 0d7bb4df2343 |
children |
rev | line source |
---|---|
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
1 #!/usr/bin/env python3 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
2 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
3 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
4 # Libervia: an XMPP client |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org) |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
6 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
7 # This program is free software: you can redistribute it and/or modify |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
8 # it under the terms of the GNU Affero General Public License as published by |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
9 # the Free Software Foundation, either version 3 of the License, or |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
10 # (at your option) any later version. |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
11 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
12 # This program is distributed in the hope that it will be useful, |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
15 # GNU Affero General Public License for more details. |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
16 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
17 # You should have received a copy of the GNU Affero General Public License |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
18 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
19 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
20 """tools to launch process in a async way (using Twisted)""" |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
21 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
22 from collections import OrderedDict |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
23 from typing import Optional, Callable, Awaitable |
4071
4b842c1fb686
refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents:
3936
diff
changeset
|
24 from libervia.backend.core.log import getLogger |
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
25 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
26 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
27 log = getLogger(__name__) |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
28 |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
29 |
3936
ebe45ea2df3b
tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents:
3725
diff
changeset
|
30 def async_lru(maxsize: Optional[int] = 50) -> Callable: |
ebe45ea2df3b
tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents:
3725
diff
changeset
|
31 """Decorator to cache async function results using LRU algorithm |
ebe45ea2df3b
tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents:
3725
diff
changeset
|
32 |
4270
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
33 @param maxsize: maximum number of items to keep in cache. |
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
34 None to have no limit |
3936
ebe45ea2df3b
tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents:
3725
diff
changeset
|
35 |
ebe45ea2df3b
tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents:
3725
diff
changeset
|
36 """ |
4270
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
37 |
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
38 def decorator(func: Callable) -> Callable: |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
39 cache = OrderedDict() |
4270
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
40 |
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
41 async def wrapper(*args) -> Awaitable: |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
42 if args in cache: |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
43 log.debug(f"using result in cache for {args}") |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
44 cache.move_to_end(args) |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
45 result = cache[args] |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
46 return result |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
47 log.debug(f"caching result for {args}") |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
48 result = await func(*args) |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
49 cache[args] = result |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
50 if maxsize is not None and len(cache) > maxsize: |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
51 value = cache.popitem(False) |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
52 log.debug(f"Removing LRU value: {value}") |
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
53 return result |
4270
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
54 |
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
55 return wrapper |
4270
0d7bb4df2343
Reformatted code base using black.
Goffi <goffi@goffi.org>
parents:
4071
diff
changeset
|
56 |
3725
9b45f0f168cf
tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
57 return decorator |