annotate sat/plugins/plugin_comp_ap_gateway/__init__.py @ 3769:a75874df92b8

tests (e2e/cli/cache_search): make the test more resilient: the caching is triggered by `li.blog.get`, but it is done in parallel without blocking the command, thus there is a possibility that the data is not yet in cache when the following test is done. A loop with delay is done, to be sure that cache is filled before doing the assertion.
author Goffi <goffi@goffi.org>
date Fri, 13 May 2022 19:27:21 +0200
parents 125c7043b277
children efc34a89e70b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia ActivityPub Gateway
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 import base64
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
20 import calendar
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 import hashlib
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
22 import json
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from pathlib import Path
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
24 from pprint import pformat
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
25 import re
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
26 from typing import Any, Dict, List, Optional, Tuple, Union, Callable, Awaitable, overload
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
27 from urllib import parse
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
28
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
29 from cryptography.exceptions import InvalidSignature
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 from cryptography.hazmat.primitives import serialization
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 from cryptography.hazmat.primitives import hashes
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
32 from cryptography.hazmat.primitives.asymmetric import rsa
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 from cryptography.hazmat.primitives.asymmetric import padding
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
34 import dateutil
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
35 import shortuuid
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
36 from sqlalchemy.exc import IntegrityError
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 import treq
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 from treq.response import _Response as TReqResponse
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
39 from twisted.internet import defer, reactor, threads
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
40 from twisted.web import http
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
41 from twisted.words.protocols.jabber import error, jid
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
42 from twisted.words.xish import domish
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
43 from wokkel import rsm
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
44
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
45 from sat.core import exceptions
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
46 from sat.core.constants import Const as C
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
47 from sat.core.core_types import SatXMPPEntity
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
48 from sat.core.i18n import _
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 from sat.core.log import getLogger
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
50 from sat.memory.sqla_mapping import SubscriptionState
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 from sat.tools import utils
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
52 from sat.tools.common import data_format, tls, uri
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
53 from sat.tools.common.async_utils import async_lru
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
54
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
55 from .constants import (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
56 ACTIVITY_OBJECT_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
57 ACTIVITY_TARGET_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
58 ACTIVITY_TYPES,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
59 ACTIVITY_TYPES_LOWER,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
60 AP_MB_MAP,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
61 COMMENTS_MAX_PARENTS,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
62 CONF_SECTION,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
63 IMPORT_NAME,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
64 LRU_MAX_SIZE,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
65 MEDIA_TYPE_AP,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
66 TYPE_ACTOR,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
67 TYPE_ITEM,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
68 )
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
69 from .http_server import HTTPServer
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
70 from .pubsub_service import APPubsubService
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
71
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
72
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 log = getLogger(__name__)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
74
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
75 IMPORT_NAME = "ap-gateway"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
76
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
77 PLUGIN_INFO = {
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
78 C.PI_NAME: "ActivityPub Gateway component",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
79 C.PI_IMPORT_NAME: IMPORT_NAME,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
80 C.PI_MODES: [C.PLUG_MODE_COMPONENT],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
81 C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
82 C.PI_PROTOCOLS: [],
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
83 C.PI_DEPENDENCIES: ["XEP-0106", "XEP-0277", "XEP-0060", "XEP-0465", "PUBSUB_CACHE"],
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
84 C.PI_RECOMMENDATIONS: [],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
85 C.PI_MAIN: "APGateway",
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
86 C.PI_HANDLER: C.BOOL_TRUE,
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 C.PI_DESCRIPTION: _(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
88 "Gateway for bidirectional communication between XMPP and ActivityPub."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
89 ),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
90 }
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
91
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
92 HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
93 RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
94 RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
95 RE_ALLOWED_UNQUOTED = re.compile(r"^[a-zA-Z0-9_-]+$")
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
96
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
97
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
98 class APGateway:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
99
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
100 def __init__(self, host):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
101 self.host = host
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
102 self.initialised = False
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
103 self.client = None
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
104 self._m = host.plugins["XEP-0277"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
105 self._p = host.plugins["XEP-0060"]
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
106 self._e = host.plugins["XEP-0106"]
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
107 self._pps = host.plugins["XEP-0465"]
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
108 self._c = host.plugins["PUBSUB_CACHE"]
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
109 self._p.addManagedNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
110 "", items_cb=self._itemsReceived
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
111 )
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
112 self.pubsub_service = APPubsubService(self)
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
113
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
114 host.bridge.addMethod(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 "APSend",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
116 ".plugin",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 in_sign="sss",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
118 out_sign="",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 method=self._publishMessage,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
120 async_=True,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
121 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
122
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
123 def getHandler(self, __):
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
124 return self.pubsub_service
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
125
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 async def init(self, client):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
127 if self.initialised:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
128 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
129
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 self.initialised = True
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
131 log.info(_("ActivityPub Gateway initialization"))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
132
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
133 # RSA keys
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
134 stored_data = await self.host.memory.storage.getPrivates(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
135 IMPORT_NAME, ["rsa_key"], profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
136 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
137 private_key_pem = stored_data.get("rsa_key")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
138 if private_key_pem is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
139 self.private_key = await threads.deferToThread(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
140 rsa.generate_private_key,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
141 public_exponent=65537,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
142 key_size=4096,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
143 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
144 private_key_pem = self.private_key.private_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
145 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
146 format=serialization.PrivateFormat.PKCS8,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
147 encryption_algorithm=serialization.NoEncryption()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
148 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
149 await self.host.memory.storage.setPrivateValue(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
150 IMPORT_NAME, "rsa_key", private_key_pem, profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
151 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
152 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
153 self.private_key = serialization.load_pem_private_key(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
154 private_key_pem.encode(),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
155 password=None,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
156 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
157 self.public_key = self.private_key.public_key()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
158 self.public_key_pem = self.public_key.public_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
159 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
160 format=serialization.PublicFormat.SubjectPublicKeyInfo
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
161 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
162
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
163 # params
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
164 # URL and port
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
165 self.public_url = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
166 CONF_SECTION, "public_url"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
167 ) or self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
168 CONF_SECTION, "xmpp_domain"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
169 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
170 if self.public_url is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
171 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
172 '"public_url" not set in configuration, this is mandatory to have'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
173 "ActivityPub Gateway running. Please set this option it to public facing "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
174 f"url in {CONF_SECTION!r} configuration section."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
175 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
176 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
177 if parse.urlparse(self.public_url).scheme:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
178 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
179 "Scheme must not be specified in \"public_url\", please remove it from "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
180 "\"public_url\" configuration option. ActivityPub Gateway won't be run."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
181 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
182 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
183 self.http_port = int(self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
184 CONF_SECTION, 'http_port', 8123))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
185 connection_type = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
186 CONF_SECTION, 'http_connection_type', 'https')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
187 if connection_type not in ('http', 'https'):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
188 raise exceptions.ConfigError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
189 'bad ap-gateay http_connection_type, you must use one of "http" or '
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
190 '"https"'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
191 )
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
192 self.max_items = int(self.host.memory.getConfig(
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
193 CONF_SECTION, 'new_node_max_items', 50
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
194
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
195 ))
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
196 self.comments_max_depth = int(self.host.memory.getConfig(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
197 CONF_SECTION, 'comments_max_depth', 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
198 ))
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
199 self.ap_path = self.host.memory.getConfig(CONF_SECTION, 'ap_path', '_ap')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
200 self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
201 # True (default) if we provide gateway only to entities/services from our server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
202 self.local_only = C.bool(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
203 self.host.memory.getConfig(CONF_SECTION, 'local_only', C.BOOL_TRUE)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
204 )
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
205
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
206 # HTTP server launch
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
207 self.server = HTTPServer(self)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
208 if connection_type == 'http':
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
209 reactor.listenTCP(self.http_port, self.server)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
210 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
211 options = tls.getOptionsFromConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
212 self.host.memory.config, CONF_SECTION)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
213 tls.TLSOptionsCheck(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
214 context_factory = tls.getTLSContextFactory(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
215 reactor.listenSSL(self.http_port, self.server, context_factory)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
216
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
217 async def profileConnecting(self, client):
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
218 self.client = client
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
219 await self.init(client)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
220
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
221 async def _itemsReceived(self, client, itemsEvent):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
222 """Callback called when pubsub items are received
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
223
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
224 if the items are adressed to a JID corresponding to an AP actor, they are
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
225 converted to AP items and sent to the corresponding AP server.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
226
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
227 If comments nodes are linked, they are automatically subscribed to get new items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
228 from there too.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
229 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
230 if client != self.client:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
231 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
232 # we need recipient as JID and not gateway own JID to be able to use methods such
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
233 # as "subscribe"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
234 client = self.client.getVirtualClient(itemsEvent.sender)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
235 recipient = itemsEvent.recipient
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
236 if not recipient.user:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
237 log.debug("ignoring items event without local part specified")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
238 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
239
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
240 ap_account = self._e.unescape(recipient.user)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
241 await self.convertAndPostItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
242 client, ap_account, recipient, itemsEvent.nodeIdentifier, itemsEvent.items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
243 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
244
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
245 async def getVirtualClient(self, actor_id: str) -> SatXMPPEntity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
246 """Get client for this component with a specified jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
247
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
248 This is needed to perform operations with the virtual JID corresponding to the AP
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
249 actor instead of the JID of the gateway itself.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
250 @param actor_id: ID of the actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
251 @return: virtual client
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
252 """
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
253 local_jid = await self.getJIDFromId(actor_id)
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
254 return self.client.getVirtualClient(local_jid)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
255
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
256 def isActivity(self, data: dict) -> bool:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
257 """Return True if the data has an activity type"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
258 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
259 return (data.get("type") or "").lower() in ACTIVITY_TYPES_LOWER
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
260 except (KeyError, TypeError):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
261 return False
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
262
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
263 async def apGet(self, url: str) -> dict:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
264 """Retrieve AP JSON from given URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
265
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
266 @raise error.StanzaError: "service-unavailable" is sent when something went wrong
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
267 with AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
268 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
269 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
270 return await treq.json_content(await treq.get(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
271 url,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
272 headers = {
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
273 "Accept": [MEDIA_TYPE_AP],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
274 "Content-Type": [MEDIA_TYPE_AP],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
275 }
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
276 ))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
277 except Exception as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
278 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
279 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
280 text=f"Can't get AP data at {url}: {e}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
281 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
282
3742
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
283 @overload
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
284 async def apGetObject(self, data: dict, key: str) -> Optional[dict]:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
285 ...
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
286
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
287 @overload
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
288 async def apGetObject(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
289 self, data: Union[str, dict], key: None = None
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
290 ) -> dict:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
291 ...
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
292
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
293 async def apGetObject(self, data, key = None):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
294 """Retrieve an AP object, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
295
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
296 This method is to be used with attributes marked as "Functional" in
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
297 https://www.w3.org/TR/activitystreams-vocabulary
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
298 @param data: AP object where an other object is looked for, or the object itself
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
299 @param key: name of the object to look for, or None if data is the object directly
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
300 @return: found object if any
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
301 """
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
302 if key is not None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
303 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
304 else:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
305 value = data
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
306 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
307 if key is None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
308 raise ValueError("None can't be used with apGetObject is key is None")
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
309 return None
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
310 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
311 return value
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
312 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
313 return await self.apGet(value)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
314 else:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
315 raise NotImplementedError(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
316 "was expecting a string or a dict, got {type(value)}: {value!r}}"
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
317 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
318
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
319 async def apGetList(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
320 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
321 data: dict,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
322 key: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
323 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
324 ) -> Optional[List[Dict[str, Any]]]:
3742
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
325 """Retrieve a list of objects from AP data, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
326
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
327 This method is to be used with non functional vocabularies. Use ``apGetObject``
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
328 otherwise.
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
329 If the value is a dictionary, it will be wrapped in a list
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
330 @param data: AP object where a list of objects is looked for
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
331 @param key: key of the list to look for
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
332 @param only_ids: if Trye, only items IDs are retrieved
3742
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
333 @return: list of objects, or None if the key is not present
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
334 """
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
335 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
336 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
337 return None
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
338 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
339 value = await self.apGet(value)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
340 if isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
341 return [value]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
342 if not isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
343 raise ValueError(f"A list was expected, got {type(value)}: {value!r}")
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
344 if only_ids:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
345 return [
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
346 {"id": v["id"]} if isinstance(v, dict) else {"id": v}
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
347 for v in value
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
348 ]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
349 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
350 return [await self.apGetObject(i) for i in value]
3742
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
351
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
352 async def apGetActors(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
353 self,
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
354 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
355 key: str,
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
356 as_account: bool = True
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
357 ) -> List[str]:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
358 """Retrieve AP actors from data
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
359
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
360 @param data: AP object containing a field with actors
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
361 @param key: field to use to retrieve actors
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
362 @param as_account: if True returns account handles, otherwise will return actor
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
363 IDs
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
364 @raise exceptions.DataError: there is not actor data or it is invalid
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
365 """
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
366 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
367 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
368 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
369 f"no actor associated to object {data.get('id')!r}"
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
370 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
371 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
372 actor_id = value.get("id")
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
373 if actor_id is None:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
374 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
375 f"invalid actor associated to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
376 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
377 value = [actor_id]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
378 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
379 value = [value]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
380 elif isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
381 try:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
382 value = [a if isinstance(a, str) else a["id"] for a in value]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
383 except (TypeError, KeyError):
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
384 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
385 f"invalid actors list to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
386 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
387 if not value:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
388 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
389 f"list of actors is empty"
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
390 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
391 if as_account:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
392 return [await self.getAPAccountFromId(actor_id) for actor_id in value]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
393 else:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
394 return value
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
395
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
396 async def apGetSenderActor(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
397 self,
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
398 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
399 ) -> str:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
400 """Retrieve actor who sent data
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
401
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
402 This is done by checking "attributedTo" field first, then "actor" field.
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
403 Only the first found actor is taken into accoun
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
404 @param data: AP object
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
405 @return: actor id of the sender
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
406 @raise exceptions.NotFound: no actor has been found in data
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
407 """
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
408 try:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
409 actors = await self.apGetActors(data, "attributedTo", as_account=False)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
410 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
411 actors = None
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
412 if not actors:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
413 try:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
414 actors = await self.apGetActors(data, "actor", as_account=False)
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
415 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
416 raise exceptions.NotFound(
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
417 'actor not specified in "attributedTo" or "actor"'
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
418 )
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
419 try:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
420 return actors[0]
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
421 except IndexError:
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
422 raise exceptions.NotFound("list of actors is empty")
bf0505d41c09 comp AP: helper methods to get AP objects:
Goffi <goffi@goffi.org>
parents: 3729
diff changeset
423
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
424 def mustEncode(self, text: str) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
425 """Indicate if a text must be period encoded"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
426 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
427 not RE_ALLOWED_UNQUOTED.match(text)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
428 or text.startswith("___")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
429 or "---" in text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
430 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
431
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
432 def periodEncode(self, text: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
433 """Period encode a text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
434
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
435 see [getJIDAndNode] for reasons of period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
436 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
437 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
438 parse.quote(text, safe="")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
439 .replace("---", "%2d%2d%2d")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
440 .replace("___", "%5f%5f%5f")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
441 .replace(".", "%2e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
442 .replace("~", "%7e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
443 .replace("%", ".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
444 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
445
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
446 async def getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
447 self,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
448 jid_: jid.JID,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
449 node: Optional[str]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
450 ) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
451 """Construct AP account from JID and node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
452
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
453 The account construction will use escaping when necessary
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
454 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
455 if not node or node == self._m.namespace:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
456 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
457
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
458 if node and not jid_.user and not self.mustEncode(node):
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
459 is_pubsub = await self.isPubsub(jid_)
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
460 # when we have a pubsub service, the user part can be used to set the node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
461 # this produces more user-friendly AP accounts
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
462 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
463 jid_.user = node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
464 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
465
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
466 is_local = self.isLocal(jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
467 user = jid_.user if is_local else jid_.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
468 if user is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
469 user = ""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
470 account_elts = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
471 if node and self.mustEncode(node) or self.mustEncode(user):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
472 account_elts = ["___"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
473 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
474 node = self.periodEncode(node)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
475 user = self.periodEncode(user)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
476
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
477 if not user:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
478 raise exceptions.InternalError("there should be a user part")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
479
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
480 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
481 account_elts.extend((node, "---"))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
482
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
483 account_elts.extend((
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
484 user, "@", jid_.host if is_local else self.client.jid.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
485 ))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
486 return "".join(account_elts)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
487
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
488 def isLocal(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
489 """Returns True if jid_ use a domain or subdomain of gateway's host"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
490 local_host = self.client.host.split(".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
491 assert local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
492 return jid_.host.split(".")[-len(local_host):] == local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
493
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
494 async def isPubsub(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
495 """Indicate if a JID is a Pubsub service"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
496 host_disco = await self.host.getDiscoInfos(self.client, jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
497 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
498 ("pubsub", "service") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
499 and not ("pubsub", "pep") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
500 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
501
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
502 async def getJIDAndNode(self, ap_account: str) -> Tuple[jid.JID, Optional[str]]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
503 """Decode raw AP account handle to get XMPP JID and Pubsub Node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
504
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
505 Username are case insensitive.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
506
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
507 By default, the username correspond to local username (i.e. username from
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
508 component's server).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
509
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
510 If local name's domain is a pubsub service (and not PEP), the username is taken as
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
511 a pubsub node.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
512
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
513 If ``---`` is present in username, the part before is used as pubsub node, and the
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
514 rest as a JID user part.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
515
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
516 If username starts with ``___``, characters are encoded using period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
517 (i.e. percent encoding where a ``.`` is used instead of ``%``).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
518
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
519 This horror is necessary due to limitation in some AP implementation (notably
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
520 Mastodon), cf. https://github.com/mastodon/mastodon/issues/17222
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
521
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
522 examples:
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
523
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
524 ``toto@example.org`` => JID = toto@example.org, node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
525
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
526 ``___toto.40example.net@example.org`` => JID = toto@example.net (this one is a
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
527 non-local JID, and will work only if setings ``local_only`` is False), node = None
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
528
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
529 ``toto@pubsub.example.org`` (with pubsub.example.org being a pubsub service) =>
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
530 JID = pubsub.example.org, node = toto
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
531
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
532 ``tata---toto@example.org`` => JID = toto@example.org, node = tata
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
533
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
534 ``___urn.3axmpp.3amicroblog.3a0@pubsub.example.org`` (with pubsub.example.org
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
535 being a pubsub service) ==> JID = pubsub.example.org, node = urn:xmpp:microblog:0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
536
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
537 @param ap_account: ActivityPub account handle (``username@domain.tld``)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
538 @return: service JID and pubsub node
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
539 if pubsub node is None, default microblog pubsub node (and possibly other
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
540 nodes that plugins may hanlde) will be used
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
541 @raise ValueError: invalid account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
542 @raise PermissionError: non local jid is used when gateway doesn't allow them
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
543 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
544 if ap_account.count("@") != 1:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
545 raise ValueError("Invalid AP account")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
546 if ap_account.startswith("___"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
547 encoded = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
548 ap_account = ap_account[3:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
549 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
550 encoded = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
551
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
552 username, domain = ap_account.split("@")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
553
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
554 if "---" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
555 node, username = username.rsplit("---", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
556 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
557 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
558
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
559 if encoded:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
560 username = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
561 RE_PERIOD_ENC.sub(r"%\g<hex>", username),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
562 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
563 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
564 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
565 node = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
566 RE_PERIOD_ENC.sub(r"%\g<hex>", node),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
567 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
568 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
569
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
570 if "@" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
571 username, domain = username.rsplit("@", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
572
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
573 if not node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
574 # we need to check host disco, because disco request to user may be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
575 # blocked for privacy reason (see
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
576 # https://xmpp.org/extensions/xep-0030.html#security)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
577 is_pubsub = await self.isPubsub(jid.JID(domain))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
578
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
579 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
580 # if the host is a pubsub service and not a PEP, we consider that username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
581 # is in fact the node name
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
582 node = username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
583 username = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
584
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
585 jid_s = f"{username}@{domain}" if username else domain
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
586 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
587 jid_ = jid.JID(jid_s)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
588 except RuntimeError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
589 raise ValueError(f"Invalid jid: {jid_s!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
590
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
591 if self.local_only and not self.isLocal(jid_):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
592 raise exceptions.PermissionError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
593 "This gateway is configured to map only local entities and services"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
594 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
595
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
596 return jid_, node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
597
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
598 def getLocalJIDFromAccount(self, account: str) -> jid.JID:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
599 """Compute JID linking to an AP account
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
600
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
601 The local jid is computer by escaping AP actor handle and using it as local part
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
602 of JID, where domain part is this gateway own JID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
603 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
604 return jid.JID(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
605 None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
606 (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
607 self._e.escape(account),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
608 self.client.jid.host,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
609 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
610 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
611 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
612
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
613 async def getJIDFromId(self, actor_id: str) -> jid.JID:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
614 """Compute JID linking to an AP Actor ID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
615
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
616 The local jid is computer by escaping AP actor handle and using it as local part
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
617 of JID, where domain part is this gateway own JID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
618 If the actor_id comes from local server (checked with self.public_url), it means
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
619 that we have an XMPP entity, and the original JID is returned
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
620 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
621 if self.isLocalURL(actor_id):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
622 request_type, extra_args = self.parseAPURL(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
623 if request_type != TYPE_ACTOR or len(extra_args) != 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
624 raise ValueError(f"invalid actor id: {actor_id!r}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
625 actor_jid, __ = await self.getJIDAndNode(extra_args[0])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
626 return actor_jid
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
627
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
628 account = await self.getAPAccountFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
629 return self.getLocalJIDFromAccount(account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
630
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
631 def parseAPURL(self, url: str) -> Tuple[str, List[str]]:
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
632 """Parse an URL leading to an AP endpoint
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
633
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
634 @param url: URL to parse (schema is not mandatory)
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
635 @return: endpoint type and extra arguments
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
636 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
637 path = parse.urlparse(url).path.lstrip("/")
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
638 type_, *extra_args = path[len(self.ap_path):].lstrip("/").split("/", 1)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
639 return type_, [parse.unquote(a) for a in extra_args]
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
640
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
641 def buildAPURL(self, type_:str , *args: str) -> str:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
642 """Build an AP endpoint URL
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
643
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
644 @param type_: type of AP endpoing
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
645 @param arg: endpoint dependant arguments
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
646 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
647 return parse.urljoin(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
648 self.base_ap_url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
649 str(Path(type_).joinpath(*(parse.quote_plus(a) for a in args)))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
650 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
651
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
652 def isLocalURL(self, url: str) -> bool:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
653 """Tells if an URL link to this component
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
654
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
655 ``public_url`` and ``ap_path`` are used to check the URL
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
656 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
657 return url.startswith(self.base_ap_url)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
658
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
659 def buildSignatureHeader(self, values: Dict[str, str]) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
660 """Build key="<value>" signature header from signature data"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
661 fields = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
662 for key, value in values.items():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
663 if key not in ("(created)", "(expired)"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
664 if '"' in value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
665 raise NotImplementedError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
666 "string escaping is not implemented, double-quote can't be used "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
667 f"in {value!r}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
668 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
669 value = f'"{value}"'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
670 fields.append(f"{key}={value}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
671
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
672 return ",".join(fields)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
673
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
674 def getDigest(self, body: bytes, algo="SHA-256") -> Tuple[str, str]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
675 """Get digest data to use in header and signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
676
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
677 @param body: body of the request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
678 @return: hash name and digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
679 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
680 if algo != "SHA-256":
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
681 raise NotImplementedError("only SHA-256 is implemented for now")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
682 return algo, base64.b64encode(hashlib.sha256(body).digest()).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
683
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
684 @async_lru(maxsize=LRU_MAX_SIZE)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
685 async def getActorPubKeyData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
686 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
687 actor_id: str
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
688 ) -> Tuple[str, str, rsa.RSAPublicKey]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
689 """Retrieve Public Key data from actor ID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
690
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
691 @param actor_id: actor ID (url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
692 @return: key_id, owner and public_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
693 @raise KeyError: publicKey is missing from actor data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
694 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
695 actor_data = await self.apGet(actor_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
696 pub_key_data = actor_data["publicKey"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
697 key_id = pub_key_data["id"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
698 owner = pub_key_data["owner"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
699 pub_key_pem = pub_key_data["publicKeyPem"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
700 pub_key = serialization.load_pem_public_key(pub_key_pem.encode())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
701 return key_id, owner, pub_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
702
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
703 def createActivity(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
704 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
705 activity: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
706 actor_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
707 object_: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
708 target: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
709 **kwargs,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
710 ) -> Dict[str, Any]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
711 """Generate base data for an activity
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
712
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
713 @param activity: one of ACTIVITY_TYPES
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
714 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
715 if activity not in ACTIVITY_TYPES:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
716 raise exceptions.InternalError(f"invalid activity: {activity!r}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
717 if object_ is None and activity in ACTIVITY_OBJECT_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
718 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
719 f'"object_" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
720 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
721 if target is None and activity in ACTIVITY_TARGET_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
722 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
723 f'"target" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
724 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
725 activity_id = f"{actor_id}#{activity.lower()}_{shortuuid.uuid()}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
726 data: Dict[str, Any] = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
727 "@context": "https://www.w3.org/ns/activitystreams",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
728 "actor": actor_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
729 "id": activity_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
730 "type": activity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
731 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
732 data.update(kwargs)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
733 if object_ is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
734 data["object"] = object_
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
735 if target is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
736 data["target"] = target
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
737
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
738 return data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
739
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
740 def getKeyId(self, actor_id: str) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
741 """Get local key ID from actor ID"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
742 return f"{actor_id}#main-key"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
743
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
744 async def checkSignature(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
745 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
746 signature: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
747 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
748 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
749 ) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
750 """Verify that signature matches given headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
751
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
752 see https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-06#section-3.1.2
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
753
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
754 @param signature: Base64 encoded signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
755 @param key_id: ID of the key used to sign the data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
756 @param headers: headers and their values, including pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
757 @return: id of the signing actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
758
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
759 @raise InvalidSignature: signature doesn't match headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
760 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
761 to_sign = "\n".join(f"{k.lower()}: {v}" for k,v in headers.items())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
762 if key_id.startswith("acct:"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
763 actor = key_id[5:]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
764 actor_id = await self.getAPActorIdFromAccount(actor)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
765 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
766 actor_id = key_id.split("#", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
767
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
768 pub_key_id, pub_key_owner, pub_key = await self.getActorPubKeyData(actor_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
769 if pub_key_id != key_id or pub_key_owner != actor_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
770 raise exceptions.EncryptionError("Public Key mismatch")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
771
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
772 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
773 pub_key.verify(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
774 base64.b64decode(signature),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
775 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
776 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
777 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
778 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
779 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
780 except InvalidSignature:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
781 raise exceptions.EncryptionError("Invalid signature (using PKC0S1 v1.5 and SHA-256)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
782
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
783 return actor_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
784
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
785 def getSignatureData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
786 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
787 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
788 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
789 ) -> Tuple[Dict[str, str], Dict[str, str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
790 """Generate and return signature and corresponding headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
791
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
792 @param parsed_url: URL where the request is sent/has been received
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
793 @param key_id: ID of the key (URL linking to the data with public key)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
794 @param date: HTTP datetime string of signature generation
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
795 @param body: body of the HTTP request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
796 @param headers: headers to sign and their value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
797 default value will be used if not specified
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
798
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
799 @return: headers and signature data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
800 ``headers`` is an updated copy of ``headers`` arguments, with pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
801 removed, and ``Signature`` added.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
802 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
803 to_sign = "\n".join(f"{k.lower()}: {v}" for k,v in headers.items())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
804 signature = base64.b64encode(self.private_key.sign(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
805 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
806 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
807 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
808 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
809 )).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
810 sign_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
811 "keyId": key_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
812 "Algorithm": "rsa-sha256",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
813 "headers": " ".join(headers.keys()),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
814 "signature": signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
815 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
816 new_headers = {k: v for k,v in headers.items() if not k.startswith("(")}
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
817 new_headers["Signature"] = self.buildSignatureHeader(sign_data)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
818 return new_headers, sign_data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
819
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
820 async def convertAndPostItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
821 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
822 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
823 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
824 service: jid.JID,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
825 node: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
826 items: List[domish.Element],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
827 subscribe_comments_nodes: bool = False,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
828 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
829 """Convert XMPP items to AP items and post them to actor inbox
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
830
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
831 @param ap_account: account of ActivityPub actor
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
832 @param service: JID of the virtual pubsub service corresponding to the AP actor
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
833 @param node: virtual node corresponding to the AP actor and items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
834 @param subscribe_comments_nodes: if True, comment nodes present in given items,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
835 they will be automatically subscribed
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
836 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
837 actor_id = await self.getAPActorIdFromAccount(ap_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
838 inbox = await self.getAPInboxFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
839 for item in items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
840 mb_data = await self._m.item2mbdata(client, item, service, node)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
841 if subscribe_comments_nodes:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
842 # we subscribe automatically to comment nodes if any
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
843 for comment_data in mb_data.get("comments", []):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
844 comment_service = jid.JID(comment_data["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
845 comment_node = comment_data["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
846 await self._p.subscribe(client, comment_service, comment_node)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
847 ap_item = await self.mbdata2APitem(client, mb_data)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
848 url_actor = ap_item["object"]["attributedTo"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
849 resp = await self.signAndPost(inbox, url_actor, ap_item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
850 if resp.code >= 300:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
851 text = await resp.text()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
852 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
853 f"unexpected return code while sending AP item: {resp.code}\n{text}\n"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
854 f"{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
855 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
856
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
857 async def signAndPost(self, url: str, actor_id: str, doc: dict) -> TReqResponse:
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
858 """Sign a documentent and post it to AP server
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
859
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
860 @param url: AP server endpoint
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
861 @param actor_id: originating actor ID (URL)
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
862 @param doc: document to send
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
863 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
864 p_url = parse.urlparse(url)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
865 body = json.dumps(doc).encode()
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
866 digest_algo, digest_hash = self.getDigest(body)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
867 digest = f"{digest_algo}={digest_hash}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
868
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
869 headers = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
870 "(request-target)": f"post {p_url.path}",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
871 "Host": p_url.hostname,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
872 "Date": http.datetimeToString().decode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
873 "Digest": digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
874 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
875 headers, __ = self.getSignatureData(self.getKeyId(actor_id), headers)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
876
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
877 headers["Content-Type"] = (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
878 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"'
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
879 )
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
880 resp = await treq.post(
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
881 url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
882 body,
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
883 headers=headers,
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
884 )
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
885 if resp.code >= 400:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
886 text = await resp.text()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
887 log.warning(f"POST request to {url} failed: {text}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
888 return resp
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
889
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
890 def _publishMessage(self, mess_data_s: str, service_s: str, profile: str):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
891 mess_data: dict = data_format.deserialise(mess_data_s) # type: ignore
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
892 service = jid.JID(service_s)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
893 client = self.host.getClient(profile)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
894 return defer.ensureDeferred(self.publishMessage(client, mess_data, service))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
895
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
896 @async_lru(maxsize=LRU_MAX_SIZE)
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
897 async def getAPActorIdFromAccount(self, account: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
898 """Retrieve account ID from it's handle using WebFinger
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
899
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
900 @param account: AP handle (user@domain.tld)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
901 @return: Actor ID (which is an URL)
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
902 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
903 if account.count("@") != 1 or "/" in account:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
904 raise ValueError("Invalid account: {account!r}")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
905 host = account.split("@")[1]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
906 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
907 finger_data = await treq.json_content(await treq.get(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
908 f"https://{host}/.well-known/webfinger?"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
909 f"resource=acct:{parse.quote_plus(account)}",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
910 ))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
911 except Exception as e:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
912 raise exceptions.DataError(f"Can't get webfinger data: {e}")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
913 for link in finger_data.get("links", []):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
914 if (
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
915 link.get("type") == "application/activity+json"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
916 and link.get("rel") == "self"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
917 ):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
918 href = link.get("href", "").strip()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
919 if not href:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
920 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
921 f"Invalid webfinger data for {account:r}: missing href"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
922 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
923 break
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
924 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
925 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
926 f"No ActivityPub link found for {account!r}"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
927 )
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
928 return href
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
929
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
930 async def getAPActorDataFromAccount(self, account: str) -> dict:
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
931 """Retrieve ActivityPub Actor data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
932
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
933 @param account: ActivityPub Actor identifier
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
934 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
935 href = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
936 return await self.apGet(href)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
937
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
938 @async_lru(maxsize=LRU_MAX_SIZE)
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
939 async def getAPInboxFromId(self, actor_id: str) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
940 """Retrieve inbox of an actor_id"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
941 data = await self.apGet(actor_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
942 return data["inbox"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
943
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
944 @async_lru(maxsize=LRU_MAX_SIZE)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
945 async def getAPAccountFromId(self, actor_id: str) -> str:
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
946 """Retrieve AP account from the ID URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
947
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
948 @param actor_id: AP ID of the actor (URL to the actor data)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
949 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
950 url_parsed = parse.urlparse(actor_id)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
951 actor_data = await self.apGet(actor_id)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
952 username = actor_data.get("preferredUsername")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
953 if not username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
954 raise exceptions.DataError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
955 'No "preferredUsername" field found, can\'t retrieve actor account'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
956 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
957 account = f"{username}@{url_parsed.hostname}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
958 # we try to retrieve the actor ID from the account to check it
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
959 found_id = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
960 if found_id != actor_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
961 # cf. https://socialhub.activitypub.rocks/t/how-to-retrieve-user-server-tld-handle-from-actors-url/2196
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
962 msg = (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
963 f"Account ID found on WebFinger {found_id!r} doesn't match our actor ID "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
964 f"({actor_id!r}). This AP instance doesn't seems to use "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
965 '"preferredUsername" as we expect.'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
966 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
967 log.warning(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
968 raise exceptions.DataError(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
969 return account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
970
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
971 async def getAPItems(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
972 self,
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
973 collection: dict,
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
974 max_items: Optional[int] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
975 chronological_pagination: bool = True,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
976 after_id: Optional[str] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
977 start_index: Optional[int] = None,
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
978 parser: Optional[Callable[[dict], Awaitable[domish.Element]]] = None,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
979 only_ids: bool = False,
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
980 ) -> Tuple[List[domish.Element], rsm.RSMResponse]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
981 """Retrieve AP items and convert them to XMPP items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
982
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
983 @param account: AP account handle to get items from
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
984 @param max_items: maximum number of items to retrieve
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
985 retrieve all items by default
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
986 @param chronological_pagination: get pages in chronological order
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
987 AP use reversed chronological order for pagination, "first" page returns more
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
988 recent items. If "chronological_pagination" is True, "last" AP page will be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
989 retrieved first.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
990 @param after_id: if set, retrieve items starting from given ID
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
991 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
992 if ``after_id`` is not already in cache, we have to retrieve every page until
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
993 we find it.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
994 In most common cases, ``after_id`` should be in cache though (client usually
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
995 use known ID when in-order pagination is used).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
996 @param start_index: start retrieving items from the one with given index
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
997 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
998 all pages before the requested index will be retrieved to count items.
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
999 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1000 if None, use default generic parser
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1001 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1002 Retrieving only item IDs avoid HTTP requests to retrieve items, it may be
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1003 sufficient in some use cases (e.g. when retrieving following/followers
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1004 collections)
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1005 @return: XMPP Pubsub items and corresponding RSM Response
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1006 Items are always returned in chronological order in the result
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1007 """
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1008 if parser is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1009 parser = self.apItem2Elt
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1010
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1011 rsm_resp: Dict[str, Union[bool, int]] = {}
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1012 try:
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1013 count = collection["totalItems"]
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1014 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1015 log.warning(
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1016 f'"totalItems" not found in collection {collection.get("id")}, '
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1017 "defaulting to 20"
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1018 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1019 count = 20
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1020 else:
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1021 log.info(f"{collection.get('id')} has {count} item(s)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1022
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1023 rsm_resp["count"] = count
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1024
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1025 if start_index is not None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1026 assert chronological_pagination and after_id is None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1027 if start_index >= count:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1028 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1029 elif start_index == 0:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1030 # this is the default behaviour
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1031 pass
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1032 elif start_index > 5000:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1033 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1034 "feature-not-implemented",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1035 text="Maximum limit for previous_index has been reached, this limit"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1036 "is set to avoid DoS"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1037 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1038 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1039 # we'll convert "start_index" to "after_id", thus we need the item just
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1040 # before "start_index"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1041 previous_index = start_index - 1
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1042 retrieved_items = 0
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1043 current_page = collection["last"]
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1044 while retrieved_items < count:
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1045 page_data, items = await self.parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1046 current_page, parser, only_ids
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1047 )
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1048 if not items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1049 log.warning(f"found an empty AP page at {current_page}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1050 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1051 page_start_idx = retrieved_items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1052 retrieved_items += len(items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1053 if previous_index <= retrieved_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1054 after_id = items[previous_index - page_start_idx]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1055 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1056 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1057 current_page = page_data["prev"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1058 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1059 log.warning(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1060 f"missing previous page link at {current_page}: {page_data!r}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1061 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1062 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1063 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1064 "Error while retrieving previous page from AP service at "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1065 f"{current_page}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1066 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1067
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1068 init_page = "last" if chronological_pagination else "first"
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1069 page = collection.get(init_page)
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1070 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1071 raise exceptions.DataError(
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1072 f"Initial page {init_page!r} not found for collection "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1073 f"{collection.get('id')})"
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1074 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1075 items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1076 page_items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1077 retrieved_items = 0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1078 found_after_id = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1079
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1080 while retrieved_items < count:
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1081 __, page_items = await self.parseAPPage(page, parser, only_ids)
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1082 if not page_items:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1083 break
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1084 retrieved_items += len(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1085 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1086 # if we have an after_id, we ignore all items until the requested one is
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1087 # found
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
1088 try:
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
1089 limit_idx = [i["id"] for i in page_items].index(after_id)
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
1090 except ValueError:
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1091 # if "after_id" is not found, we don't add any item from this page
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1092 page_id = page.get("id") if isinstance(page, dict) else page
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1093 log.debug(f"{after_id!r} not found at {page_id}, skipping")
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1094 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1095 found_after_id = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1096 if chronological_pagination:
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
1097 start_index = retrieved_items - len(page_items) + limit_idx + 1
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1098 page_items = page_items[limit_idx+1:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1099 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1100 start_index = count - (retrieved_items - len(page_items) +
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1101 limit_idx + 1)
3729
86eea17cafa7 component AP gateway: split plugin in several files:
Goffi <goffi@goffi.org>
parents: 3728
diff changeset
1102 page_items = page_items[:limit_idx]
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1103 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1104 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1105 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1106 if max_items is not None and len(items) >= max_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1107 if chronological_pagination:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1108 items = items[:max_items]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1109 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1110 items = items[-max_items:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1111 break
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1112 page = collection.get("prev" if chronological_pagination else "next")
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1113 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1114 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1115
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1116 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1117 raise error.StanzaError("item-not-found")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1118
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1119 if items:
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1120 if after_id is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1121 rsm_resp["index"] = 0 if chronological_pagination else count - len(items)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1122 if start_index is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1123 rsm_resp["index"] = start_index
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1124 elif after_id is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1125 log.warning("Can't determine index of first element")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1126 elif chronological_pagination:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1127 rsm_resp["index"] = 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1128 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1129 rsm_resp["index"] = count - len(items)
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1130 rsm_resp.update({
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1131 "first": items[0]["id"],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1132 "last": items[-1]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1133 })
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1134
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1135 return items, rsm.RSMResponse(**rsm_resp)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1136
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1137 async def apItem2MbDataAndElt(self, ap_item: dict) -> Tuple[dict, domish.Element]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1138 """Convert AP item to parsed microblog data and corresponding item element"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1139 mb_data = await self.apItem2MBdata(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1140 item_elt = await self._m.data2entry(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1141 self.client, mb_data, mb_data["id"], None, self._m.namespace
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1142 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1143 item_elt["publisher"] = mb_data["author_jid"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1144 return mb_data, item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1145
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1146 async def apItem2Elt(self, ap_item: dict) -> domish.Element:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1147 """Convert AP item to XMPP item element"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1148 __, item_elt = await self.apItem2MbDataAndElt(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1149 return item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1150
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1151 async def parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1152 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1153 page: Union[str, dict],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1154 parser: Callable[[dict], Awaitable[domish.Element]],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1155 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1156 ) -> Tuple[dict, List[domish.Element]]:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1157 """Convert AP objects from an AP page to XMPP items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1158
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1159 @param page: Can be either url linking and AP page, or the page data directly
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1160 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1161 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1162 @return: page data, pubsub items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1163 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1164 page_data = await self.apGetObject(page)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1165 if page_data is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1166 log.warning('No data found in collection')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1167 return {}, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1168 ap_items = await self.apGetList(page_data, "orderedItems", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1169 if ap_items is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1170 ap_items = await self.apGetList(page_data, "items", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1171 if not ap_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1172 log.warning(f'No item field found in collection: {page_data!r}')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1173 return page_data, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1174 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1175 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1176 "Items are not ordered, this is not spec compliant"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1177 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1178 items = []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1179 # AP Collections are in antichronological order, but we expect chronological in
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1180 # Pubsub, thus we reverse it
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1181 for ap_item in reversed(ap_items):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1182 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1183 items.append(await parser(ap_item))
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1184 except (exceptions.DataError, NotImplementedError, error.StanzaError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1185 continue
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1186
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1187 return page_data, items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1188
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1189 async def getCommentsNodes(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1190 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1191 item_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1192 parent_id: Optional[str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1193 ) -> Tuple[Optional[str], Optional[str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1194 """Get node where this item is and node to use for comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1195
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1196 if config option "comments_max_depth" is set, a common node will be used below the
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1197 given depth
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1198 @param item_id: ID of the reference item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1199 @param parent_id: ID of the parent item if any (the ID set in "inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1200 @return: a tuple with parent_node_id, comments_node_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1201 - parent_node_id is the ID of the node where reference item must be. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1202 returned when the root node (i.e. not a comments node) must be used.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1203 - comments_node_id: is the ID of the node to use for comments. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1204 returned when no comment node must be used (happens when we have reached
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1205 "comments_max_depth")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1206 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1207 if parent_id is None or not self.comments_max_depth:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1208 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1209 self._m.getCommentsNode(parent_id) if parent_id is not None else None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1210 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1211 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1212 parent_url = parent_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1213 parents = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1214 for __ in range(COMMENTS_MAX_PARENTS):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1215 parent_item = await self.apGet(parent_url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1216 parents.insert(0, parent_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1217 parent_url = parent_item.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1218 if parent_url is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1219 break
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1220 parent_limit = self.comments_max_depth-1
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1221 if len(parents) <= parent_limit:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1222 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1223 self._m.getCommentsNode(parents[-1]["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1224 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1225 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1226 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1227 last_level_item = parents[parent_limit]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1228 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1229 self._m.getCommentsNode(last_level_item["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1230 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1231 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1232
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1233 async def apItem2MBdata(self, ap_item: dict) -> dict:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1234 """Convert AP activity or object to microblog data
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1235
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1236 @return: AP Item's Object and microblog data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1237 @raise exceptions.DataError: something is invalid in the AP item
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1238 @raise NotImplemented: some AP data is not handled yet
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1239 @raise error.StanzaError: error while contacting the AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1240 """
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1241 is_activity = self.isActivity(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1242 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1243 ap_object = await self.apGetObject(ap_item, "object")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1244 if not ap_object:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1245 log.warning(f'No "object" found in AP item {ap_item!r}')
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1246 raise exceptions.DataError
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1247 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1248 ap_object = ap_item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1249 item_id = ap_object.get("id")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1250 if not item_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1251 log.warning(f'No "id" found in AP item: {ap_object!r}')
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1252 raise exceptions.DataError
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1253 mb_data = {"id": item_id}
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1254 for ap_key, mb_key in AP_MB_MAP.items():
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1255 data = ap_object.get(ap_key)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1256 if data is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1257 continue
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1258 mb_data[mb_key] = data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1259
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1260 # content
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1261 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1262 language, content_xhtml = ap_object["contentMap"].popitem()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1263 except (KeyError, AttributeError):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1264 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1265 mb_data["content_xhtml"] = mb_data["content"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1266 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1267 log.warning(f"no content found:\n{ap_object!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1268 raise exceptions.DataError
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1269 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1270 mb_data["language"] = language
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1271 mb_data["content_xhtml"] = content_xhtml
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1272
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1273 # author
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1274 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1275 authors = await self.apGetActors(ap_item, "actor")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1276 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1277 authors = await self.apGetActors(ap_object, "attributedTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1278 if len(authors) > 1:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1279 # we only keep first item as author
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1280 # TODO: handle multiple actors
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1281 log.warning("multiple actors are not managed")
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1282
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1283 account = authors[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1284 author_jid = self.getLocalJIDFromAccount(account).full()
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1285
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1286 mb_data["author"] = account.split("@", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1287 mb_data["author_jid"] = author_jid
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1288
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1289 # published/updated
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1290 for field in ("published", "updated"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1291 value = ap_object.get(field)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1292 if not value and field == "updated":
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1293 value = ap_object.get("published")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1294 if value:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1295 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1296 mb_data[field] = calendar.timegm(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1297 dateutil.parser.parse(str(value)).utctimetuple()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1298 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1299 except dateutil.parser.ParserError as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1300 log.warning(f"Can't parse {field!r} field: {e}")
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1301
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1302 # comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1303 in_reply_to = ap_object.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1304 __, comments_node = await self.getCommentsNodes(item_id, in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1305 if comments_node is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1306 comments_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1307 "service": author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1308 "node": comments_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1309 "uri": uri.buildXMPPUri(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1310 "pubsub",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1311 path=author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1312 node=comments_node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1313 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1314 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1315 mb_data["comments"] = [comments_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1316
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1317 return mb_data
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1318
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1319 async def getReplyToIdFromXMPPNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1320 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1321 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1322 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1323 parent_item: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1324 mb_data: dict
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1325 ) -> str:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1326 """Get URL to use for ``inReplyTo`` field in AP item.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1327
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1328 There is currently no way to know the parent service of a comment with XEP-0277.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1329 To work around that, we try to check if we have this item in the cache (we
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1330 should). If there is more that one item with this ID, we first try to find one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1331 with this author_jid. If nothing is found, we use ap_account to build `inReplyTo`.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1332
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1333 @param ap_account: AP account corresponding to the publication author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1334 @param parent_item: ID of the node where the publication this item is replying to
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1335 has been posted
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1336 @param mb_data: microblog data of the publication
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1337 @return: URL to use in ``inReplyTo`` field
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1338 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1339 # FIXME: propose a protoXEP to properly get parent item, node and service
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1340
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1341 found_items = await self.host.memory.storage.searchPubsubItems({
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1342 "profiles": [client.profile],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1343 "names": [parent_item]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1344 })
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1345 if not found_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1346 log.warning(f"parent item {parent_item!r} not found in cache")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1347 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1348 elif len(found_items) == 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1349 cached_node = found_items[0].node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1350 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1351 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1352 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1353 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1354 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1355 # we found several cached item with given ID, we check if there is one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1356 # corresponding to this author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1357 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1358 author = jid.JID(mb_data["author_jid"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1359 cached_item = next(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1360 i for i in found_items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1361 if jid.JID(i.data["publisher"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1362 == author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1363 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1364 except StopIteration:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1365 # no item corresponding to this author, we use ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1366 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1367 "Can't find a single cached item for parent item "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1368 f"{parent_item!r}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1369 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1370 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1371 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1372 cached_node = cached_item.node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1373 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1374 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1375 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1376 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1377
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1378 return self.buildAPURL(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1379 TYPE_ITEM, parent_ap_account, parent_item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1380 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1381
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1382 async def mbdata2APitem(self, client: SatXMPPEntity, mb_data: dict) -> dict:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1383 """Convert Libervia Microblog Data to ActivityPub item"""
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1384 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1385 node = mb_data["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1386 service = jid.JID(mb_data["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1387 except KeyError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1388 # node and service must always be specified when this method is used
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1389 raise exceptions.InternalError(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1390 "node or service is missing in mb_data"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1391 )
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1392 if not mb_data.get("id"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1393 mb_data["id"] = shortuuid.uuid()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1394 if not mb_data.get("author_jid"):
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1395 mb_data["author_jid"] = client.jid.full()
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1396 ap_account = await self.getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1397 jid.JID(mb_data["author_jid"]),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1398 None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1399 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1400 url_actor = self.buildAPURL(TYPE_ACTOR, ap_account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1401 url_item = self.buildAPURL(TYPE_ITEM, ap_account, mb_data["id"])
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1402 ap_object = {
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1403 "id": url_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1404 "type": "Note",
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1405 "published": utils.xmpp_date(mb_data["published"]),
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1406 "attributedTo": url_actor,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1407 "content": mb_data.get("content_xhtml") or mb_data["content"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1408 "to": ["https://www.w3.org/ns/activitystreams#Public"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1409 }
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1410
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1411 target_ap_account = self._e.unescape(service.user)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1412 actor_data = await self.getAPActorDataFromAccount(target_ap_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1413 followers = actor_data.get("followers")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1414 if followers:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1415 ap_object["cc"] = [followers]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1416
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1417 ap_item = {
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1418 "@context": "https://www.w3.org/ns/activitystreams",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1419 "id": url_item,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1420 "type": "Create",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1421 "actor": url_actor,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1422
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1423 "object": ap_object
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1424 }
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1425 language = mb_data.get("language")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1426 if language:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1427 ap_object["contentMap"] = {language: ap_object["content"]}
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1428 if self._m.isCommentNode(node):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1429 parent_item = self._m.getParentItem(node)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1430 if service.host == self.client.jid.userhost():
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1431 # the publication is on a virtual node (i.e. an XMPP node managed by
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1432 # this gateway and linking to an ActivityPub actor)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1433 ap_object["inReplyTo"] = parent_item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1434 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1435 # the publication is from a followed real XMPP node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1436 ap_object["inReplyTo"] = await self.getReplyToIdFromXMPPNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1437 client,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1438 ap_account,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1439 parent_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1440 mb_data
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1441 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1442
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1443 return ap_item
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1444
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1445 async def publishMessage(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1446 self,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1447 client: SatXMPPEntity,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1448 mess_data: dict,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1449 service: jid.JID
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1450 ) -> None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1451 """Send an AP message
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1452
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1453 .. note::
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1454
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1455 This is a temporary method used for development only
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1456
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1457 @param mess_data: message data. Following keys must be set:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1458
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1459 ``node``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1460 identifier of message which is being replied (this will
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1461 correspond to pubsub node in the future)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1462
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1463 ``content_xhtml`` or ``content``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1464 message body (respectively in XHTML or plain text)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1465
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1466 @param service: JID corresponding to the AP actor.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1467 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1468 if not service.user:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1469 raise ValueError("service must have a local part")
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1470 account = self._e.unescape(service.user)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1471 ap_actor_data = await self.getAPActorDataFromAccount(account)
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1472
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1473 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1474 inbox_url = ap_actor_data["endpoints"]["sharedInbox"]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1475 except KeyError:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1476 raise exceptions.DataError("Can't get ActivityPub actor inbox")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1477
3728
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1478 item_data = await self.mbdata2APitem(client, mess_data)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion:
Goffi <goffi@goffi.org>
parents: 3684
diff changeset
1479 url_actor = item_data["object"]["attributedTo"]
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1480 resp = await self.signAndPost(inbox_url, url_actor, item_data)
3684
8353cc3b8db9 component AP gateway: fix wrong operator when testing HTTP code
Goffi <goffi@goffi.org>
parents: 3682
diff changeset
1481 if resp.code != 202:
3682
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1482 raise exceptions.NetworkError(f"unexpected return code: {resp.code}")
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1483
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1484 async def newReplyToXMPPItem(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1485 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1486 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1487 ap_item: dict,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1488 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1489 """We got an AP item which is a reply to an XMPP item"""
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1490 in_reply_to = ap_item["inReplyTo"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1491 url_type, url_args = self.parseAPURL(in_reply_to)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1492 if url_type != "item":
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1493 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1494 "Ignoring AP item replying to an XMPP item with an unexpected URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1495 f"type({url_type!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1496 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1497 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1498 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1499 parent_item_account, parent_item_id = url_args[0].split("/", 1)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1500 except (IndexError, ValueError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1501 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1502 "Ignoring AP item replying to an XMPP item with invalid inReplyTo URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1503 f"({in_reply_to!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1504 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1505 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1506 parent_item_service, parent_item_node = await self.getJIDAndNode(parent_item_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1507 if parent_item_node is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1508 parent_item_node = self._m.namespace
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1509 items, __ = await self._p.getItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1510 client, parent_item_service, parent_item_node, item_ids=[parent_item_id]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1511 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1512 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1513 parent_item_elt = items[0]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1514 except IndexError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1515 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1516 f"Can't find parent item at {parent_item_service} (node "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1517 f"{parent_item_node!r})\n{pformat(ap_item)}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1518 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1519 parent_item_parsed = await self._m.item2mbdata(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1520 client, parent_item_elt, parent_item_service, parent_item_node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1521 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1522 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1523 comment_service = jid.JID(parent_item_parsed["comments"][0]["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1524 comment_node = parent_item_parsed["comments"][0]["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1525 except (KeyError, IndexError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1526 # we don't have a comment node set for this item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1527 from sat.tools.xml_tools import ppElt
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1528 log.info(f"{ppElt(parent_item_elt.toXml())}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1529 raise NotImplemented()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1530 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1531 __, item_elt = await self.apItem2MbDataAndElt(ap_item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1532 await self._p.publish(client, comment_service, comment_node, [item_elt])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1533
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1534 async def newAPItem(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1535 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1536 client: SatXMPPEntity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1537 destinee: Optional[jid.JID],
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1538 node: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1539 item: dict,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1540 ) -> None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1541 """Analyse, cache and send notification for received AP item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1542
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1543 @param destinee: jid of the destinee,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1544 @param node: XMPP pubsub node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1545 @param item: AP object payload
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1546 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1547 service = client.jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1548 in_reply_to = item.get("inReplyTo")
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1549 if in_reply_to and isinstance(in_reply_to, list):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1550 in_reply_to = in_reply_to[0]
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1551 if in_reply_to and isinstance(in_reply_to, str):
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1552 if self.isLocalURL(in_reply_to):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1553 # this is a reply to an XMPP item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1554 return await self.newReplyToXMPPItem(client, item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1555
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1556 # this item is a reply to an AP item, we use or create a corresponding node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1557 # for comments
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1558 parent_node, __ = await self.getCommentsNodes(item["id"], in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1559 node = parent_node or node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1560 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1561 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1562 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1563 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1564 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1565 cached_node = await self.host.memory.storage.setPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1566 client,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1567 service,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1568 node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1569 subscribed=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1570 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1571 except IntegrityError as e:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1572 if "unique" in str(e.orig).lower():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1573 # the node may already exist, if it has been created just after
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1574 # getPubsubNode above
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1575 log.debug("ignoring UNIQUE constraint error")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1576 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1577 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1578 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1579 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1580 raise e
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1581
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1582 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1583 # it is a root item (i.e. not a reply to an other item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1584 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1585 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1586 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1587 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1588 log.warning(
3764
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1589 f"Received item in unknown node {node!r} at {service}. This may be "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers:
Goffi <goffi@goffi.org>
parents: 3745
diff changeset
1590 f"due to a cache purge. We synchronise the node\n{item}"
3745
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1591
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1592 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1593 return
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1594 mb_data, item_elt = await self.apItem2MbDataAndElt(item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1595 await self.host.memory.storage.cachePubsubItems(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1596 client,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1597 cached_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1598 [item_elt],
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1599 [mb_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1600 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1601
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1602 for subscription in cached_node.subscriptions:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1603 if subscription.state != SubscriptionState.SUBSCRIBED:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1604 continue
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1605 self.pubsub_service.notifyPublish(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1606 service,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1607 node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1608 [(subscription.subscriber, None, [item_elt])]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management:
Goffi <goffi@goffi.org>
parents: 3742
diff changeset
1609 )