annotate libervia/backend/tools/common/async_utils.py @ 4306:94e0968987cd

plugin XEP-0033: code modernisation, improve delivery, data validation: - Code has been rewritten using Pydantic models and `async` coroutines for data validation and cleaner element parsing/generation. - Delivery has been completely rewritten. It now works even if server doesn't support multicast, and send to local multicast service first. Delivering to local multicast service first is due to bad support of XEP-0033 in server (notably Prosody which has an incomplete implementation), and the current impossibility to detect if a sub-domain service handles fully multicast or only for local domains. This is a workaround to have a good balance between backward compatilibity and use of bandwith, and to make it work with the incoming email gateway implementation (the gateway will only deliver to entities of its own domain). - disco feature checking now uses `async` corountines. `host` implementation still use Deferred return values for compatibility with legacy code. rel 450
author Goffi <goffi@goffi.org>
date Thu, 26 Sep 2024 16:12:01 +0200
parents 0d7bb4df2343
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
2
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
3
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Libervia: an XMPP client
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
6
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
10 # (at your option) any later version.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
11
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
16
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
19
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
20 """tools to launch process in a async way (using Twisted)"""
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
21
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from collections import OrderedDict
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from typing import Optional, Callable, Awaitable
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 3936
diff changeset
24 from libervia.backend.core.log import getLogger
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
25
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
26
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
27 log = getLogger(__name__)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
28
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
29
3936
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
30 def async_lru(maxsize: Optional[int] = 50) -> Callable:
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
31 """Decorator to cache async function results using LRU algorithm
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
32
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
33 @param maxsize: maximum number of items to keep in cache.
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
34 None to have no limit
3936
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
35
ebe45ea2df3b tools (common/async_utils): set default `maxsize` to 50 in `async_lru`
Goffi <goffi@goffi.org>
parents: 3725
diff changeset
36 """
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
37
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
38 def decorator(func: Callable) -> Callable:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
39 cache = OrderedDict()
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
40
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
41 async def wrapper(*args) -> Awaitable:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
42 if args in cache:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
43 log.debug(f"using result in cache for {args}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
44 cache.move_to_end(args)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
45 result = cache[args]
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
46 return result
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
47 log.debug(f"caching result for {args}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
48 result = await func(*args)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
49 cache[args] = result
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
50 if maxsize is not None and len(cache) > maxsize:
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
51 value = cache.popitem(False)
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
52 log.debug(f"Removing LRU value: {value}")
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
53 return result
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
54
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
55 return wrapper
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
56
3725
9b45f0f168cf tools (common): new `async_utils` module with an async version of `lru_cache`
Goffi <goffi@goffi.org>
parents:
diff changeset
57 return decorator