libervia-backend: sat/plugins/plugin_comp_ap_gateway/__init_

annotate sat/plugins/plugin_comp_ap_gateway/init.py @ 3793:b5c9021020df

component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: when a `Delete` actity is received, the item is removed from cache, and the corresponding item `retract` is sent to local XMPP subscribers. rel 367

author	Goffi <goffi@goffi.org>
date	Fri, 17 Jun 2022 14:15:23 +0200
parents	865167c34b82
children	d5f343939239

rev	line source
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia ActivityPub Gateway
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 import base64
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	20 import calendar
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	21 import hashlib
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	22 import json
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	23 from pathlib import Path
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	24 from pprint import pformat
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	25 import re
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	26 from typing import (
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	27 Any, Dict, List, Set, Optional, Tuple, Union, Callable, Awaitable, overload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	28 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	29 from urllib import parse
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	30
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	31 from cryptography.exceptions import InvalidSignature
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	32 from cryptography.hazmat.primitives import serialization
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	33 from cryptography.hazmat.primitives import hashes
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	34 from cryptography.hazmat.primitives.asymmetric import rsa
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	35 from cryptography.hazmat.primitives.asymmetric import padding
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	36 import dateutil
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	37 import shortuuid
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	38 from sqlalchemy.exc import IntegrityError
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	39 import treq
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	40 from treq.response import _Response as TReqResponse
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	41 from twisted.internet import defer, reactor, threads
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	42 from twisted.web import http
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	43 from twisted.words.protocols.jabber import error, jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	44 from twisted.words.xish import domish
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	45 from wokkel import rsm
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	46
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	47 from sat.core import exceptions
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 from sat.core.constants import Const as C
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	49 from sat.core.core_types import SatXMPPEntity
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	50 from sat.core.i18n import _
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 from sat.core.log import getLogger
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	52 from sat.memory.sqla_mapping import SubscriptionState
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	53 from sat.tools import utils
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	54 from sat.tools.common import data_format, tls, uri
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	55 from sat.tools.common.async_utils import async_lru
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	56
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	57 from .constants import (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	58 ACTIVITY_OBJECT_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	59 ACTIVITY_TARGET_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	60 ACTIVITY_TYPES,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	61 ACTIVITY_TYPES_LOWER,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	62 AP_MB_MAP,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	63 COMMENTS_MAX_PARENTS,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	64 CONF_SECTION,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	65 IMPORT_NAME,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	66 LRU_MAX_SIZE,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	67 MEDIA_TYPE_AP,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	68 TYPE_ACTOR,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	69 TYPE_ITEM,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	70 TYPE_FOLLOWERS,
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	71 TYPE_TOMBSTONE,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	72 NS_AP_PUBLIC,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	73 PUBLIC_TUPLE
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	74 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	75 from .http_server import HTTPServer
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	76 from .pubsub_service import APPubsubService
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	77
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	78
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	79 log = getLogger(__name__)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	80
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	81 IMPORT_NAME = "ap-gateway"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	82
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	83 PLUGIN_INFO = {
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	84 C.PI_NAME: "ActivityPub Gateway component",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	85 C.PI_IMPORT_NAME: IMPORT_NAME,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	86 C.PI_MODES: [C.PLUG_MODE_COMPONENT],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	87 C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	88 C.PI_PROTOCOLS: [],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	89 C.PI_DEPENDENCIES: [
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	90 "XEP-0106", "XEP-0277", "XEP-0060", "XEP-0465", "PUBSUB_CACHE", "TEXT_SYNTAXES"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	91 ],
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	92 C.PI_RECOMMENDATIONS: [],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	93 C.PI_MAIN: "APGateway",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	94 C.PI_HANDLER: C.BOOL_TRUE,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	95 C.PI_DESCRIPTION: _(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	96 "Gateway for bidirectional communication between XMPP and ActivityPub."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	97 ),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	98 }
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	99
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	100 HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	101 RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	102 RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	103 RE_ALLOWED_UNQUOTED = re.compile(r"^[a-zA-Z0-9_-]+$")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	104
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	105
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	106 class APGateway:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	107
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	108 def __init__(self, host):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	109 self.host = host
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	110 self.initialised = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	111 self.client = None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	112 self._m = host.plugins["XEP-0277"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	113 self._p = host.plugins["XEP-0060"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	114 self._e = host.plugins["XEP-0106"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	115 self._pps = host.plugins["XEP-0465"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	116 self._c = host.plugins["PUBSUB_CACHE"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	117 self._p.addManagedNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	118 "", items_cb=self._itemsReceived
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	119 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	120 self._t = host.plugins["TEXT_SYNTAXES"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	121 self.pubsub_service = APPubsubService(self)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	122
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	123 host.bridge.addMethod(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	124 "APSend",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	125 ".plugin",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	126 in_sign="sss",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	127 out_sign="",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	128 method=self._publishMessage,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	129 async_=True,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	131
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	132 def getHandler(self, __):
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	133 return self.pubsub_service
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	134
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	135 async def init(self, client):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	136 if self.initialised:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	137 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	138
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	139 self.initialised = True
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	140 log.info(_("ActivityPub Gateway initialization"))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	141
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	142 # RSA keys
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	143 stored_data = await self.host.memory.storage.getPrivates(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	144 IMPORT_NAME, ["rsa_key"], profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	145 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	146 private_key_pem = stored_data.get("rsa_key")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	147 if private_key_pem is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	148 self.private_key = await threads.deferToThread(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	149 rsa.generate_private_key,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	150 public_exponent=65537,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	151 key_size=4096,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	152 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	153 private_key_pem = self.private_key.private_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	154 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	155 format=serialization.PrivateFormat.PKCS8,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	156 encryption_algorithm=serialization.NoEncryption()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	157 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	158 await self.host.memory.storage.setPrivateValue(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	159 IMPORT_NAME, "rsa_key", private_key_pem, profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	160 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	161 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	162 self.private_key = serialization.load_pem_private_key(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	163 private_key_pem.encode(),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	164 password=None,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	165 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	166 self.public_key = self.private_key.public_key()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	167 self.public_key_pem = self.public_key.public_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	168 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	169 format=serialization.PublicFormat.SubjectPublicKeyInfo
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	170 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	171
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	172 # params
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	173 # URL and port
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	174 self.public_url = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	175 CONF_SECTION, "public_url"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	176 ) or self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	177 CONF_SECTION, "xmpp_domain"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	178 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	179 if self.public_url is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	180 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	181 '"public_url" not set in configuration, this is mandatory to have'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	182 "ActivityPub Gateway running. Please set this option it to public facing "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	183 f"url in {CONF_SECTION!r} configuration section."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	184 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	185 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	186 if parse.urlparse(self.public_url).scheme:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	187 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	188 "Scheme must not be specified in \"public_url\", please remove it from "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	189 "\"public_url\" configuration option. ActivityPub Gateway won't be run."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	190 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	191 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	192 self.http_port = int(self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	193 CONF_SECTION, 'http_port', 8123))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	194 connection_type = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	195 CONF_SECTION, 'http_connection_type', 'https')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	196 if connection_type not in ('http', 'https'):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	197 raise exceptions.ConfigError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	198 'bad ap-gateay http_connection_type, you must use one of "http" or '
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	199 '"https"'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	200 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	201 self.max_items = int(self.host.memory.getConfig(
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	202 CONF_SECTION, 'new_node_max_items', 50
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	203
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	204 ))
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	205 self.comments_max_depth = int(self.host.memory.getConfig(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	206 CONF_SECTION, 'comments_max_depth', 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	207 ))
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 self.ap_path = self.host.memory.getConfig(CONF_SECTION, 'ap_path', '_ap')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	210 # True (default) if we provide gateway only to entities/services from our server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	211 self.local_only = C.bool(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	212 self.host.memory.getConfig(CONF_SECTION, 'local_only', C.BOOL_TRUE)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	213 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	214
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	215 # HTTP server launch
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	216 self.server = HTTPServer(self)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	217 if connection_type == 'http':
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	218 reactor.listenTCP(self.http_port, self.server)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	219 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	220 options = tls.getOptionsFromConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	221 self.host.memory.config, CONF_SECTION)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	222 tls.TLSOptionsCheck(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	223 context_factory = tls.getTLSContextFactory(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 reactor.listenSSL(self.http_port, self.server, context_factory)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	225
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 async def profileConnecting(self, client):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	227 self.client = client
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 await self.init(client)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	229
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	230 def profileConnected(self, client):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	231 client.xmlstream.addObserver("/message/body", self.onMessage)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	232
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	233 async def _itemsReceived(self, client, itemsEvent):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	234 """Callback called when pubsub items are received
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	235
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	236 if the items are adressed to a JID corresponding to an AP actor, they are
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	237 converted to AP items and sent to the corresponding AP server.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	238
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	239 If comments nodes are linked, they are automatically subscribed to get new items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	240 from there too.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	241 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	242 if client != self.client:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	243 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	244 # we need recipient as JID and not gateway own JID to be able to use methods such
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	245 # as "subscribe"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	246 client = self.client.getVirtualClient(itemsEvent.sender)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	247 recipient = itemsEvent.recipient
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	248 if not recipient.user:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	249 log.debug("ignoring items event without local part specified")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	250 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	251
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	252 ap_account = self._e.unescape(recipient.user)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	253 await self.convertAndPostItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	254 client, ap_account, recipient, itemsEvent.nodeIdentifier, itemsEvent.items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	255 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	256
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	257 async def getVirtualClient(self, actor_id: str) -> SatXMPPEntity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	258 """Get client for this component with a specified jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	259
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	260 This is needed to perform operations with the virtual JID corresponding to the AP
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	261 actor instead of the JID of the gateway itself.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	262 @param actor_id: ID of the actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	263 @return: virtual client
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	264 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	265 local_jid = await self.getJIDFromId(actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	266 return self.client.getVirtualClient(local_jid)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	267
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	268 def isActivity(self, data: dict) -> bool:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	269 """Return True if the data has an activity type"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	270 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	271 return (data.get("type") or "").lower() in ACTIVITY_TYPES_LOWER
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	272 except (KeyError, TypeError):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	273 return False
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	274
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	275 async def apGet(self, url: str) -> dict:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	276 """Retrieve AP JSON from given URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	277
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	278 @raise error.StanzaError: "service-unavailable" is sent when something went wrong
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	279 with AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	280 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	281 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	282 return await treq.json_content(await treq.get(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	283 url,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	284 headers = {
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	285 "Accept": [MEDIA_TYPE_AP],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	286 "Content-Type": [MEDIA_TYPE_AP],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	287 }
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	288 ))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	289 except Exception as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	290 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	291 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	292 text=f"Can't get AP data at {url}: {e}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	293 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	294
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	295 @overload
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	296 async def apGetObject(self, data: dict, key: str) -> Optional[dict]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	297 ...
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	298
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	299 @overload
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	300 async def apGetObject(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	301 self, data: Union[str, dict], key: None = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	302 ) -> dict:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	303 ...
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	304
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	305 async def apGetObject(self, data, key = None):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	306 """Retrieve an AP object, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	307
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	308 This method is to be used with attributes marked as "Functional" in
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	309 https://www.w3.org/TR/activitystreams-vocabulary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	310 @param data: AP object where an other object is looked for, or the object itself
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	311 @param key: name of the object to look for, or None if data is the object directly
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	312 @return: found object if any
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	313 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	314 if key is not None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	315 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	316 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	317 value = data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	318 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	319 if key is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	320 raise ValueError("None can't be used with apGetObject is key is None")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	321 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	322 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	323 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	324 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	325 return await self.apGet(value)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	326 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	327 raise NotImplementedError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	328 "was expecting a string or a dict, got {type(value)}: {value!r}}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	329 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	330
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	331 async def apGetList(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	332 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	333 data: dict,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	334 key: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	335 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	336 ) -> Optional[List[Dict[str, Any]]]:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	337 """Retrieve a list of objects from AP data, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	338
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	339 This method is to be used with non functional vocabularies. Use ``apGetObject``
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	340 otherwise.
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	341 If the value is a dictionary, it will be wrapped in a list
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	342 @param data: AP object where a list of objects is looked for
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	343 @param key: key of the list to look for
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	344 @param only_ids: if Trye, only items IDs are retrieved
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	345 @return: list of objects, or None if the key is not present
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	346 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	347 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	348 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	349 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	350 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	351 value = await self.apGet(value)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	352 if isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	353 return [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	354 if not isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	355 raise ValueError(f"A list was expected, got {type(value)}: {value!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	356 if only_ids:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	357 return [
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	358 {"id": v["id"]} if isinstance(v, dict) else {"id": v}
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	359 for v in value
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	360 ]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	361 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	362 return [await self.apGetObject(i) for i in value]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	363
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	364 async def apGetActors(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	365 self,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	366 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	367 key: str,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	368 as_account: bool = True
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	369 ) -> List[str]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	370 """Retrieve AP actors from data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	371
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	372 @param data: AP object containing a field with actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	373 @param key: field to use to retrieve actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	374 @param as_account: if True returns account handles, otherwise will return actor
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	375 IDs
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	376 @raise exceptions.DataError: there is not actor data or it is invalid
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	377 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	378 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	379 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	380 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	381 f"no actor associated to object {data.get('id')!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	382 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	383 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	384 actor_id = value.get("id")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	385 if actor_id is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	386 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	387 f"invalid actor associated to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	388 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	389 value = [actor_id]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	390 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	391 value = [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	392 elif isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	393 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	394 value = [a if isinstance(a, str) else a["id"] for a in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	395 except (TypeError, KeyError):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	396 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	397 f"invalid actors list to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	398 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	399 if not value:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	400 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	401 f"list of actors is empty"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	402 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	403 if as_account:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	404 return [await self.getAPAccountFromId(actor_id) for actor_id in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	405 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	406 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	407
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	408 async def apGetSenderActor(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	409 self,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	410 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	411 ) -> str:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	412 """Retrieve actor who sent data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	413
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	414 This is done by checking "attributedTo" field first, then "actor" field.
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	415 Only the first found actor is taken into accoun
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	416 @param data: AP object
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	417 @return: actor id of the sender
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	418 @raise exceptions.NotFound: no actor has been found in data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	419 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	420 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	421 actors = await self.apGetActors(data, "attributedTo", as_account=False)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	422 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	423 actors = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	424 if not actors:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	425 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	426 actors = await self.apGetActors(data, "actor", as_account=False)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	427 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	428 raise exceptions.NotFound(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	429 'actor not specified in "attributedTo" or "actor"'
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	430 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	431 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	432 return actors[0]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	433 except IndexError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	434 raise exceptions.NotFound("list of actors is empty")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	435
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	436 def mustEncode(self, text: str) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	437 """Indicate if a text must be period encoded"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	438 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	439 not RE_ALLOWED_UNQUOTED.match(text)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	440 or text.startswith("___")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	441 or "---" in text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	442 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	443
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	444 def periodEncode(self, text: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	445 """Period encode a text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	446
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	447 see [getJIDAndNode] for reasons of period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	448 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	449 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	450 parse.quote(text, safe="")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	451 .replace("---", "%2d%2d%2d")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	452 .replace("___", "%5f%5f%5f")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	453 .replace(".", "%2e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	454 .replace("~", "%7e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	455 .replace("%", ".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	456 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	457
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	458 async def getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	459 self,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	460 jid_: jid.JID,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	461 node: Optional[str]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	462 ) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	463 """Construct AP account from JID and node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	464
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	465 The account construction will use escaping when necessary
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	466 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	467 if not node or node == self._m.namespace:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	468 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	469
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	470 if self.client is None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	471 raise exceptions.InternalError("Client is not set yet")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	472
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	473 if jid_.host == self.client.jid.userhost():
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	474 # this is an proxy JID to an AP Actor
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	475 return self._e.unescape(jid_.user)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	476
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	477 if node and not jid_.user and not self.mustEncode(node):
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	478 is_pubsub = await self.isPubsub(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	479 # when we have a pubsub service, the user part can be used to set the node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	480 # this produces more user-friendly AP accounts
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	481 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	482 jid_.user = node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	483 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	484
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	485 is_local = self.isLocal(jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	486 user = jid_.user if is_local else jid_.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	487 if user is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	488 user = ""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	489 account_elts = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	490 if node and self.mustEncode(node) or self.mustEncode(user):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	491 account_elts = ["___"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	492 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	493 node = self.periodEncode(node)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	494 user = self.periodEncode(user)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	495
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	496 if not user:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	497 raise exceptions.InternalError("there should be a user part")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	498
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	499 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	500 account_elts.extend((node, "---"))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	501
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	502 account_elts.extend((
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	503 user, "@", jid_.host if is_local else self.client.jid.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	504 ))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	505 return "".join(account_elts)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	506
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	507 def isLocal(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	508 """Returns True if jid_ use a domain or subdomain of gateway's host"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	509 local_host = self.client.host.split(".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	510 assert local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	511 return jid_.host.split(".")[-len(local_host):] == local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	512
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	513 async def isPubsub(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	514 """Indicate if a JID is a Pubsub service"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	515 host_disco = await self.host.getDiscoInfos(self.client, jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	516 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	517 ("pubsub", "service") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	518 and not ("pubsub", "pep") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	519 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	520
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	521 async def getJIDAndNode(self, ap_account: str) -> Tuple[jid.JID, Optional[str]]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	522 """Decode raw AP account handle to get XMPP JID and Pubsub Node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	523
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	524 Username are case insensitive.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	525
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	526 By default, the username correspond to local username (i.e. username from
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	527 component's server).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	528
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	529 If local name's domain is a pubsub service (and not PEP), the username is taken as
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	530 a pubsub node.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	531
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	532 If ``---`` is present in username, the part before is used as pubsub node, and the
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	533 rest as a JID user part.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	534
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	535 If username starts with ``___``, characters are encoded using period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	536 (i.e. percent encoding where a ``.`` is used instead of ``%``).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	537
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	538 This horror is necessary due to limitation in some AP implementation (notably
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	539 Mastodon), cf. https://github.com/mastodon/mastodon/issues/17222
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	540
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	541 examples:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	542
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	543 ``toto@example.org`` => JID = toto@example.org, node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	544
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	545 ``___toto.40example.net@example.org`` => JID = toto@example.net (this one is a
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	546 non-local JID, and will work only if setings ``local_only`` is False), node = None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	547
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	548 ``toto@pubsub.example.org`` (with pubsub.example.org being a pubsub service) =>
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	549 JID = pubsub.example.org, node = toto
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	550
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	551 ``tata---toto@example.org`` => JID = toto@example.org, node = tata
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	552
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	553 ``___urn.3axmpp.3amicroblog.3a0@pubsub.example.org`` (with pubsub.example.org
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	554 being a pubsub service) ==> JID = pubsub.example.org, node = urn:xmpp:microblog:0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	555
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	556 @param ap_account: ActivityPub account handle (``username@domain.tld``)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	557 @return: service JID and pubsub node
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	558 if pubsub node is None, default microblog pubsub node (and possibly other
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	559 nodes that plugins may hanlde) will be used
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	560 @raise ValueError: invalid account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	561 @raise PermissionError: non local jid is used when gateway doesn't allow them
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	562 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	563 if ap_account.count("@") != 1:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	564 raise ValueError("Invalid AP account")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	565 if ap_account.startswith("___"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	566 encoded = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	567 ap_account = ap_account[3:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	568 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	569 encoded = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	570
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	571 username, domain = ap_account.split("@")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	572
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	573 if "---" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	574 node, username = username.rsplit("---", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	575 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	576 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	577
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	578 if encoded:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	579 username = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	580 RE_PERIOD_ENC.sub(r"%\g<hex>", username),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	581 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	582 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	583 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	584 node = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	585 RE_PERIOD_ENC.sub(r"%\g<hex>", node),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	586 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	587 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	588
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	589 if "@" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	590 username, domain = username.rsplit("@", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	591
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	592 if not node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	593 # we need to check host disco, because disco request to user may be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	594 # blocked for privacy reason (see
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	595 # https://xmpp.org/extensions/xep-0030.html#security)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	596 is_pubsub = await self.isPubsub(jid.JID(domain))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	597
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	598 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	599 # if the host is a pubsub service and not a PEP, we consider that username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	600 # is in fact the node name
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	601 node = username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	602 username = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	603
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	604 jid_s = f"{username}@{domain}" if username else domain
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	605 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	606 jid_ = jid.JID(jid_s)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	607 except RuntimeError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	608 raise ValueError(f"Invalid jid: {jid_s!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	609
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	610 if self.local_only and not self.isLocal(jid_):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	611 raise exceptions.PermissionError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	612 "This gateway is configured to map only local entities and services"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	613 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	614
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	615 return jid_, node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	616
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	617 def getLocalJIDFromAccount(self, account: str) -> jid.JID:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	618 """Compute JID linking to an AP account
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	619
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	620 The local jid is computer by escaping AP actor handle and using it as local part
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	621 of JID, where domain part is this gateway own JID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	622 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	623 return jid.JID(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	624 None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	625 (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	626 self._e.escape(account),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	627 self.client.jid.host,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	628 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	629 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	630 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	631
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	632 async def getJIDFromId(self, actor_id: str) -> jid.JID:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	633 """Compute JID linking to an AP Actor ID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	634
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	635 The local jid is computer by escaping AP actor handle and using it as local part
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	636 of JID, where domain part is this gateway own JID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	637 If the actor_id comes from local server (checked with self.public_url), it means
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	638 that we have an XMPP entity, and the original JID is returned
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	639 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	640 if self.isLocalURL(actor_id):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	641 request_type, extra_args = self.parseAPURL(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	642 if request_type != TYPE_ACTOR or len(extra_args) != 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	643 raise ValueError(f"invalid actor id: {actor_id!r}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	644 actor_jid, __ = await self.getJIDAndNode(extra_args[0])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	645 return actor_jid
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	646
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	647 account = await self.getAPAccountFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	648 return self.getLocalJIDFromAccount(account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	649
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	650 def parseAPURL(self, url: str) -> Tuple[str, List[str]]:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	651 """Parse an URL leading to an AP endpoint
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	652
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	653 @param url: URL to parse (schema is not mandatory)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	654 @return: endpoint type and extra arguments
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	655 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	656 path = parse.urlparse(url).path.lstrip("/")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	657 type_, *extra_args = path[len(self.ap_path):].lstrip("/").split("/", 1)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	658 return type_, [parse.unquote(a) for a in extra_args]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	659
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	660 def buildAPURL(self, type_:str , *args: str) -> str:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	661 """Build an AP endpoint URL
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	662
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	663 @param type_: type of AP endpoing
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	664 @param arg: endpoint dependant arguments
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	665 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	666 return parse.urljoin(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	667 self.base_ap_url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	668 str(Path(type_).joinpath(*(parse.quote_plus(a) for a in args)))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	669 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	670
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	671 def isLocalURL(self, url: str) -> bool:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	672 """Tells if an URL link to this component
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	673
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	674 ``public_url`` and ``ap_path`` are used to check the URL
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	675 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	676 return url.startswith(self.base_ap_url)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	677
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	678 def buildSignatureHeader(self, values: Dict[str, str]) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	679 """Build key="<value>" signature header from signature data"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	680 fields = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	681 for key, value in values.items():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	682 if key not in ("(created)", "(expired)"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	683 if '"' in value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	684 raise NotImplementedError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	685 "string escaping is not implemented, double-quote can't be used "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	686 f"in {value!r}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	687 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	688 value = f'"{value}"'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	689 fields.append(f"{key}={value}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	690
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	691 return ",".join(fields)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	692
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	693 def getDigest(self, body: bytes, algo="SHA-256") -> Tuple[str, str]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	694 """Get digest data to use in header and signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	695
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	696 @param body: body of the request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	697 @return: hash name and digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	698 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	699 if algo != "SHA-256":
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	700 raise NotImplementedError("only SHA-256 is implemented for now")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	701 return algo, base64.b64encode(hashlib.sha256(body).digest()).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	702
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	703 @async_lru(maxsize=LRU_MAX_SIZE)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	704 async def getActorData(self, actor_id) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	705 """Retrieve actor data with LRU cache"""
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	706 return await self.apGet(actor_id)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	707
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	708 @async_lru(maxsize=LRU_MAX_SIZE)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	709 async def getActorPubKeyData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	710 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	711 actor_id: str
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	712 ) -> Tuple[str, str, rsa.RSAPublicKey]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	713 """Retrieve Public Key data from actor ID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	714
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	715 @param actor_id: actor ID (url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	716 @return: key_id, owner and public_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	717 @raise KeyError: publicKey is missing from actor data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	718 """
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	719 actor_data = await self.getActorData(actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	720 pub_key_data = actor_data["publicKey"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	721 key_id = pub_key_data["id"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	722 owner = pub_key_data["owner"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	723 pub_key_pem = pub_key_data["publicKeyPem"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	724 pub_key = serialization.load_pem_public_key(pub_key_pem.encode())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	725 return key_id, owner, pub_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	726
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	727 def createActivity(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	728 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	729 activity: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	730 actor_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	731 object_: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	732 target: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	733 **kwargs,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	734 ) -> Dict[str, Any]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	735 """Generate base data for an activity
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	736
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	737 @param activity: one of ACTIVITY_TYPES
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	738 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	739 if activity not in ACTIVITY_TYPES:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	740 raise exceptions.InternalError(f"invalid activity: {activity!r}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	741 if object_ is None and activity in ACTIVITY_OBJECT_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	742 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	743 f'"object_" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	744 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	745 if target is None and activity in ACTIVITY_TARGET_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	746 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	747 f'"target" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	748 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	749 activity_id = f"{actor_id}#{activity.lower()}_{shortuuid.uuid()}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	750 data: Dict[str, Any] = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	751 "@context": "https://www.w3.org/ns/activitystreams",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	752 "actor": actor_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	753 "id": activity_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	754 "type": activity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	755 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	756 data.update(kwargs)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	757 if object_ is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	758 data["object"] = object_
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	759 if target is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	760 data["target"] = target
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	761
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	762 return data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	763
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	764 def getKeyId(self, actor_id: str) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	765 """Get local key ID from actor ID"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	766 return f"{actor_id}#main-key"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	767
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	768 async def checkSignature(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	769 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	770 signature: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	771 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	772 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	773 ) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	774 """Verify that signature matches given headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	775
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	776 see https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-06#section-3.1.2
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	777
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	778 @param signature: Base64 encoded signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	779 @param key_id: ID of the key used to sign the data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	780 @param headers: headers and their values, including pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	781 @return: id of the signing actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	782
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	783 @raise InvalidSignature: signature doesn't match headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	784 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	785 to_sign = "\n".join(f"{k.lower()}: {v}" for k,v in headers.items())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	786 if key_id.startswith("acct:"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	787 actor = key_id[5:]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	788 actor_id = await self.getAPActorIdFromAccount(actor)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	789 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	790 actor_id = key_id.split("#", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	791
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	792 pub_key_id, pub_key_owner, pub_key = await self.getActorPubKeyData(actor_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	793 if pub_key_id != key_id or pub_key_owner != actor_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	794 raise exceptions.EncryptionError("Public Key mismatch")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	795
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	796 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	797 pub_key.verify(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	798 base64.b64decode(signature),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	799 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	800 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	801 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	802 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	803 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	804 except InvalidSignature:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	805 raise exceptions.EncryptionError("Invalid signature (using PKC0S1 v1.5 and SHA-256)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	806
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	807 return actor_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	808
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	809 def getSignatureData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	810 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	811 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	812 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	813 ) -> Tuple[Dict[str, str], Dict[str, str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	814 """Generate and return signature and corresponding headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	815
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	816 @param parsed_url: URL where the request is sent/has been received
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	817 @param key_id: ID of the key (URL linking to the data with public key)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	818 @param date: HTTP datetime string of signature generation
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	819 @param body: body of the HTTP request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	820 @param headers: headers to sign and their value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	821 default value will be used if not specified
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	822
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	823 @return: headers and signature data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	824 ``headers`` is an updated copy of ``headers`` arguments, with pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	825 removed, and ``Signature`` added.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	826 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	827 to_sign = "\n".join(f"{k.lower()}: {v}" for k,v in headers.items())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	828 signature = base64.b64encode(self.private_key.sign(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	829 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	830 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	831 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	832 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	833 )).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	834 sign_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	835 "keyId": key_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	836 "Algorithm": "rsa-sha256",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	837 "headers": " ".join(headers.keys()),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	838 "signature": signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	839 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	840 new_headers = {k: v for k,v in headers.items() if not k.startswith("(")}
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	841 new_headers["Signature"] = self.buildSignatureHeader(sign_data)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	842 return new_headers, sign_data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	843
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	844 async def convertAndPostItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	845 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	846 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	847 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	848 service: jid.JID,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	849 node: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	850 items: List[domish.Element],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	851 subscribe_comments_nodes: bool = False,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	852 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	853 """Convert XMPP items to AP items and post them to actor inbox
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	854
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	855 @param ap_account: account of ActivityPub actor
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	856 @param service: JID of the virtual pubsub service corresponding to the AP actor
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	857 @param node: virtual node corresponding to the AP actor and items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	858 @param subscribe_comments_nodes: if True, comment nodes present in given items,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	859 they will be automatically subscribed
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	860 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	861 actor_id = await self.getAPActorIdFromAccount(ap_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	862 inbox = await self.getAPInboxFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	863 for item in items:
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	864 if item.name == "item":
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	865 mb_data = await self._m.item2mbdata(client, item, service, node)
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	866 if subscribe_comments_nodes:
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	867 # we subscribe automatically to comment nodes if any
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	868 for comment_data in mb_data.get("comments", []):
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	869 comment_service = jid.JID(comment_data["service"])
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	870 comment_node = comment_data["node"]
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	871 await self._p.subscribe(client, comment_service, comment_node)
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	872 ap_item = await self.mbdata2APitem(client, mb_data)
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	873 url_actor = ap_item["object"]["attributedTo"]
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	874 elif item.name == "retract":
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	875 author_account = await self.getAPAccountFromJidAndNode(client.jid, node)
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	876 author_actor_id = self.buildAPURL(TYPE_ACTOR, author_account)
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	877 url_item = self.buildAPURL(TYPE_ITEM, author_account, item["id"])
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	878 ap_item = self.createActivity(
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	879 "Delete",
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	880 author_actor_id,
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	881 {
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	882 "id": url_item,
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	883 "type": TYPE_TOMBSTONE
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	884 }
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	885 )
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	886 ap_item["to"] = [NS_AP_PUBLIC]
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	887 url_actor = author_actor_id
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	888 else:
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	889 raise exceptions.InternalError(f"unexpected element: {item.toXml()}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	890 resp = await self.signAndPost(inbox, url_actor, ap_item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	891 if resp.code >= 300:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	892 text = await resp.text()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	893 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	894 f"unexpected return code while sending AP item: {resp.code}\n{text}\n"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	895 f"{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	896 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	897
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	898 async def signAndPost(self, url: str, actor_id: str, doc: dict) -> TReqResponse:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	899 """Sign a documentent and post it to AP server
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	900
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	901 @param url: AP server endpoint
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	902 @param actor_id: originating actor ID (URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	903 @param doc: document to send
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	904 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	905 p_url = parse.urlparse(url)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	906 body = json.dumps(doc).encode()
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	907 digest_algo, digest_hash = self.getDigest(body)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	908 digest = f"{digest_algo}={digest_hash}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	909
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	910 headers = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	911 "(request-target)": f"post {p_url.path}",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	912 "Host": p_url.hostname,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	913 "Date": http.datetimeToString().decode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	914 "Digest": digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	915 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	916 headers, __ = self.getSignatureData(self.getKeyId(actor_id), headers)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	917
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	918 headers["Content-Type"] = (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	919 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"'
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	920 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	921 resp = await treq.post(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	922 url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	923 body,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	924 headers=headers,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	925 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	926 if resp.code >= 400:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	927 text = await resp.text()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	928 log.warning(f"POST request to {url} failed: {text}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	929 return resp
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	930
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	931 def _publishMessage(self, mess_data_s: str, service_s: str, profile: str):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	932 mess_data: dict = data_format.deserialise(mess_data_s) # type: ignore
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	933 service = jid.JID(service_s)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	934 client = self.host.getClient(profile)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	935 return defer.ensureDeferred(self.publishMessage(client, mess_data, service))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	936
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	937 @async_lru(maxsize=LRU_MAX_SIZE)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	938 async def getAPActorIdFromAccount(self, account: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	939 """Retrieve account ID from it's handle using WebFinger
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	940
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	941 @param account: AP handle (user@domain.tld)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	942 @return: Actor ID (which is an URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	943 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	944 if account.count("@") != 1 or "/" in account:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	945 raise ValueError(f"Invalid account: {account!r}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	946 host = account.split("@")[1]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	947 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	948 finger_data = await treq.json_content(await treq.get(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	949 f"https://{host}/.well-known/webfinger?"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	950 f"resource=acct:{parse.quote_plus(account)}",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	951 ))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	952 except Exception as e:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	953 raise exceptions.DataError(f"Can't get webfinger data for {account!r}: {e}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	954 for link in finger_data.get("links", []):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	955 if (
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	956 link.get("type") == "application/activity+json"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	957 and link.get("rel") == "self"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	958 ):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	959 href = link.get("href", "").strip()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	960 if not href:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	961 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	962 f"Invalid webfinger data for {account:r}: missing href"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	963 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	964 break
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	965 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	966 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	967 f"No ActivityPub link found for {account!r}"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	968 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	969 return href
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	970
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	971 async def getAPActorDataFromAccount(self, account: str) -> dict:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	972 """Retrieve ActivityPub Actor data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	973
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	974 @param account: ActivityPub Actor identifier
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	975 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	976 href = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	977 return await self.apGet(href)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	978
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	979 async def getAPInboxFromId(self, actor_id: str) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	980 """Retrieve inbox of an actor_id"""
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	981 data = await self.getActorData(actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	982 return data["inbox"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	983
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	984 @async_lru(maxsize=LRU_MAX_SIZE)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	985 async def getAPAccountFromId(self, actor_id: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	986 """Retrieve AP account from the ID URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	987
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	988 @param actor_id: AP ID of the actor (URL to the actor data)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	989 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	990 url_parsed = parse.urlparse(actor_id)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	991 actor_data = await self.getActorData(actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	992 username = actor_data.get("preferredUsername")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	993 if not username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	994 raise exceptions.DataError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	995 'No "preferredUsername" field found, can\'t retrieve actor account'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	996 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	997 account = f"{username}@{url_parsed.hostname}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	998 # we try to retrieve the actor ID from the account to check it
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	999 found_id = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1000 if found_id != actor_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1001 # cf. https://socialhub.activitypub.rocks/t/how-to-retrieve-user-server-tld-handle-from-actors-url/2196
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1002 msg = (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1003 f"Account ID found on WebFinger {found_id!r} doesn't match our actor ID "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1004 f"({actor_id!r}). This AP instance doesn't seems to use "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1005 '"preferredUsername" as we expect.'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1006 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1007 log.warning(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1008 raise exceptions.DataError(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1009 return account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1010
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1011 async def getAPItems(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1012 self,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1013 collection: dict,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1014 max_items: Optional[int] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1015 chronological_pagination: bool = True,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1016 after_id: Optional[str] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1017 start_index: Optional[int] = None,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1018 parser: Optional[Callable[[dict], Awaitable[domish.Element]]] = None,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1019 only_ids: bool = False,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1020 ) -> Tuple[List[domish.Element], rsm.RSMResponse]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1021 """Retrieve AP items and convert them to XMPP items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1022
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1023 @param account: AP account handle to get items from
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1024 @param max_items: maximum number of items to retrieve
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1025 retrieve all items by default
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1026 @param chronological_pagination: get pages in chronological order
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1027 AP use reversed chronological order for pagination, "first" page returns more
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1028 recent items. If "chronological_pagination" is True, "last" AP page will be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1029 retrieved first.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1030 @param after_id: if set, retrieve items starting from given ID
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1031 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1032 if ``after_id`` is not already in cache, we have to retrieve every page until
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1033 we find it.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1034 In most common cases, ``after_id`` should be in cache though (client usually
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1035 use known ID when in-order pagination is used).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1036 @param start_index: start retrieving items from the one with given index
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1037 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1038 all pages before the requested index will be retrieved to count items.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1039 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1040 if None, use default generic parser
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1041 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1042 Retrieving only item IDs avoid HTTP requests to retrieve items, it may be
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1043 sufficient in some use cases (e.g. when retrieving following/followers
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1044 collections)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1045 @return: XMPP Pubsub items and corresponding RSM Response
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1046 Items are always returned in chronological order in the result
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1047 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1048 if parser is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1049 parser = self.apItem2Elt
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1050
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1051 rsm_resp: Dict[str, Union[bool, int]] = {}
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1052 try:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1053 count = collection["totalItems"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1054 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1055 log.warning(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1056 f'"totalItems" not found in collection {collection.get("id")}, '
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1057 "defaulting to 20"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1058 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1059 count = 20
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1060 else:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1061 log.info(f"{collection.get('id')} has {count} item(s)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1062
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1063 rsm_resp["count"] = count
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1064
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1065 if start_index is not None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1066 assert chronological_pagination and after_id is None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1067 if start_index >= count:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1068 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1069 elif start_index == 0:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1070 # this is the default behaviour
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1071 pass
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1072 elif start_index > 5000:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1073 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1074 "feature-not-implemented",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1075 text="Maximum limit for previous_index has been reached, this limit"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1076 "is set to avoid DoS"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1077 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1078 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1079 # we'll convert "start_index" to "after_id", thus we need the item just
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1080 # before "start_index"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1081 previous_index = start_index - 1
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1082 retrieved_items = 0
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1083 current_page = collection["last"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1084 while retrieved_items < count:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1085 page_data, items = await self.parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1086 current_page, parser, only_ids
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1087 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1088 if not items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1089 log.warning(f"found an empty AP page at {current_page}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1090 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1091 page_start_idx = retrieved_items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1092 retrieved_items += len(items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1093 if previous_index <= retrieved_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1094 after_id = items[previous_index - page_start_idx]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1095 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1096 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1097 current_page = page_data["prev"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1098 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1099 log.warning(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1100 f"missing previous page link at {current_page}: {page_data!r}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1101 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1102 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1103 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1104 "Error while retrieving previous page from AP service at "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1105 f"{current_page}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1106 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1107
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1108 init_page = "last" if chronological_pagination else "first"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1109 page = collection.get(init_page)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1110 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1111 raise exceptions.DataError(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1112 f"Initial page {init_page!r} not found for collection "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1113 f"{collection.get('id')})"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1114 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1115 items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1116 page_items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1117 retrieved_items = 0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1118 found_after_id = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1119
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1120 while retrieved_items < count:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1121 __, page_items = await self.parseAPPage(page, parser, only_ids)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1122 if not page_items:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1123 break
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1124 retrieved_items += len(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1125 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1126 # if we have an after_id, we ignore all items until the requested one is
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1127 # found
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1128 try:
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1129 limit_idx = [i["id"] for i in page_items].index(after_id)
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1130 except ValueError:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1131 # if "after_id" is not found, we don't add any item from this page
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1132 page_id = page.get("id") if isinstance(page, dict) else page
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1133 log.debug(f"{after_id!r} not found at {page_id}, skipping")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1134 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1135 found_after_id = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1136 if chronological_pagination:
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1137 start_index = retrieved_items - len(page_items) + limit_idx + 1
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1138 page_items = page_items[limit_idx+1:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1139 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1140 start_index = count - (retrieved_items - len(page_items) +
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1141 limit_idx + 1)
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1142 page_items = page_items[:limit_idx]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1143 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1144 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1145 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1146 if max_items is not None and len(items) >= max_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1147 if chronological_pagination:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1148 items = items[:max_items]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1149 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1150 items = items[-max_items:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1151 break
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1152 page = collection.get("prev" if chronological_pagination else "next")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1153 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1154 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1155
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1156 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1157 raise error.StanzaError("item-not-found")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1158
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1159 if items:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1160 if after_id is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1161 rsm_resp["index"] = 0 if chronological_pagination else count - len(items)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1162 if start_index is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1163 rsm_resp["index"] = start_index
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1164 elif after_id is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1165 log.warning("Can't determine index of first element")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1166 elif chronological_pagination:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1167 rsm_resp["index"] = 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1168 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1169 rsm_resp["index"] = count - len(items)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1170 rsm_resp.update({
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1171 "first": items[0]["id"],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1172 "last": items[-1]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1173 })
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1174
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1175 return items, rsm.RSMResponse(**rsm_resp)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1176
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1177 async def apItem2MbDataAndElt(self, ap_item: dict) -> Tuple[dict, domish.Element]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1178 """Convert AP item to parsed microblog data and corresponding item element"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1179 mb_data = await self.apItem2MBdata(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1180 item_elt = await self._m.data2entry(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1181 self.client, mb_data, mb_data["id"], None, self._m.namespace
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1182 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1183 item_elt["publisher"] = mb_data["author_jid"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1184 return mb_data, item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1185
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1186 async def apItem2Elt(self, ap_item: dict) -> domish.Element:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1187 """Convert AP item to XMPP item element"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1188 __, item_elt = await self.apItem2MbDataAndElt(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1189 return item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1190
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1191 async def parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1192 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1193 page: Union[str, dict],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1194 parser: Callable[[dict], Awaitable[domish.Element]],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1195 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1196 ) -> Tuple[dict, List[domish.Element]]:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1197 """Convert AP objects from an AP page to XMPP items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1198
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1199 @param page: Can be either url linking and AP page, or the page data directly
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1200 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1201 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1202 @return: page data, pubsub items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1203 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1204 page_data = await self.apGetObject(page)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1205 if page_data is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1206 log.warning('No data found in collection')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1207 return {}, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1208 ap_items = await self.apGetList(page_data, "orderedItems", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1209 if ap_items is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1210 ap_items = await self.apGetList(page_data, "items", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1211 if not ap_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1212 log.warning(f'No item field found in collection: {page_data!r}')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1213 return page_data, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1214 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1215 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1216 "Items are not ordered, this is not spec compliant"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1217 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1218 items = []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1219 # AP Collections are in antichronological order, but we expect chronological in
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1220 # Pubsub, thus we reverse it
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1221 for ap_item in reversed(ap_items):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1222 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1223 items.append(await parser(ap_item))
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1224 except (exceptions.DataError, NotImplementedError, error.StanzaError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1225 continue
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1226
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1227 return page_data, items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1228
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1229 async def getCommentsNodes(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1230 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1231 item_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1232 parent_id: Optional[str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1233 ) -> Tuple[Optional[str], Optional[str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1234 """Get node where this item is and node to use for comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1235
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1236 if config option "comments_max_depth" is set, a common node will be used below the
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1237 given depth
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1238 @param item_id: ID of the reference item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1239 @param parent_id: ID of the parent item if any (the ID set in "inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1240 @return: a tuple with parent_node_id, comments_node_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1241 - parent_node_id is the ID of the node where reference item must be. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1242 returned when the root node (i.e. not a comments node) must be used.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1243 - comments_node_id: is the ID of the node to use for comments. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1244 returned when no comment node must be used (happens when we have reached
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1245 "comments_max_depth")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1246 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1247 if parent_id is None or not self.comments_max_depth:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1248 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1249 self._m.getCommentsNode(parent_id) if parent_id is not None else None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1250 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1251 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1252 parent_url = parent_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1253 parents = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1254 for __ in range(COMMENTS_MAX_PARENTS):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1255 parent_item = await self.apGet(parent_url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1256 parents.insert(0, parent_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1257 parent_url = parent_item.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1258 if parent_url is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1259 break
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1260 parent_limit = self.comments_max_depth-1
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1261 if len(parents) <= parent_limit:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1262 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1263 self._m.getCommentsNode(parents[-1]["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1264 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1265 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1266 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1267 last_level_item = parents[parent_limit]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1268 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1269 self._m.getCommentsNode(last_level_item["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1270 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1271 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1272
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1273 async def apItem2MBdata(self, ap_item: dict) -> dict:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1274 """Convert AP activity or object to microblog data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1275
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1276 @return: AP Item's Object and microblog data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1277 @raise exceptions.DataError: something is invalid in the AP item
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1278 @raise NotImplemented: some AP data is not handled yet
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1279 @raise error.StanzaError: error while contacting the AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1280 """
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1281 is_activity = self.isActivity(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1282 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1283 ap_object = await self.apGetObject(ap_item, "object")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1284 if not ap_object:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1285 log.warning(f'No "object" found in AP item {ap_item!r}')
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1286 raise exceptions.DataError
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1287 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1288 ap_object = ap_item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1289 item_id = ap_object.get("id")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1290 if not item_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1291 log.warning(f'No "id" found in AP item: {ap_object!r}')
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1292 raise exceptions.DataError
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1293 mb_data = {"id": item_id}
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1294 for ap_key, mb_key in AP_MB_MAP.items():
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1295 data = ap_object.get(ap_key)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1296 if data is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1297 continue
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1298 mb_data[mb_key] = data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1299
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1300 # content
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1301 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1302 language, content_xhtml = ap_object["contentMap"].popitem()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1303 except (KeyError, AttributeError):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1304 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1305 mb_data["content_xhtml"] = mb_data["content"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1306 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1307 log.warning(f"no content found:\n{ap_object!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1308 raise exceptions.DataError
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1309 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1310 mb_data["language"] = language
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1311 mb_data["content_xhtml"] = content_xhtml
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1312 if not mb_data.get("content"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1313 mb_data["content"] = await self._t.convert(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1314 content_xhtml,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1315 self._t.SYNTAX_XHTML,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1316 self._t.SYNTAX_TEXT,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1317 False,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1318 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1319
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1320 # author
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1321 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1322 authors = await self.apGetActors(ap_item, "actor")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1323 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1324 authors = await self.apGetActors(ap_object, "attributedTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1325 if len(authors) > 1:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1326 # we only keep first item as author
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1327 # TODO: handle multiple actors
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1328 log.warning("multiple actors are not managed")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1329
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1330 account = authors[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1331 author_jid = self.getLocalJIDFromAccount(account).full()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1332
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1333 mb_data["author"] = account.split("@", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1334 mb_data["author_jid"] = author_jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1335
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1336 # published/updated
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1337 for field in ("published", "updated"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1338 value = ap_object.get(field)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1339 if not value and field == "updated":
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1340 value = ap_object.get("published")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1341 if value:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1342 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1343 mb_data[field] = calendar.timegm(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1344 dateutil.parser.parse(str(value)).utctimetuple()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1345 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1346 except dateutil.parser.ParserError as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1347 log.warning(f"Can't parse {field!r} field: {e}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1348
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1349 # comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1350 in_reply_to = ap_object.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1351 __, comments_node = await self.getCommentsNodes(item_id, in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1352 if comments_node is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1353 comments_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1354 "service": author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1355 "node": comments_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1356 "uri": uri.buildXMPPUri(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1357 "pubsub",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1358 path=author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1359 node=comments_node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1360 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1361 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1362 mb_data["comments"] = [comments_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1363
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1364 return mb_data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1365
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1366 async def getReplyToIdFromXMPPNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1367 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1368 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1369 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1370 parent_item: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1371 mb_data: dict
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1372 ) -> str:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1373 """Get URL to use for ``inReplyTo`` field in AP item.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1374
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1375 There is currently no way to know the parent service of a comment with XEP-0277.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1376 To work around that, we try to check if we have this item in the cache (we
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1377 should). If there is more that one item with this ID, we first try to find one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1378 with this author_jid. If nothing is found, we use ap_account to build `inReplyTo`.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1379
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1380 @param ap_account: AP account corresponding to the publication author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1381 @param parent_item: ID of the node where the publication this item is replying to
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1382 has been posted
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1383 @param mb_data: microblog data of the publication
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1384 @return: URL to use in ``inReplyTo`` field
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1385 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1386 # FIXME: propose a protoXEP to properly get parent item, node and service
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1387
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1388 found_items = await self.host.memory.storage.searchPubsubItems({
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1389 "profiles": [client.profile],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1390 "names": [parent_item]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1391 })
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1392 if not found_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1393 log.warning(f"parent item {parent_item!r} not found in cache")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1394 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1395 elif len(found_items) == 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1396 cached_node = found_items[0].node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1397 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1398 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1399 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1400 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1401 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1402 # we found several cached item with given ID, we check if there is one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1403 # corresponding to this author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1404 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1405 author = jid.JID(mb_data["author_jid"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1406 cached_item = next(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1407 i for i in found_items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1408 if jid.JID(i.data["publisher"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1409 == author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1410 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1411 except StopIteration:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1412 # no item corresponding to this author, we use ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1413 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1414 "Can't find a single cached item for parent item "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1415 f"{parent_item!r}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1416 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1417 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1418 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1419 cached_node = cached_item.node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1420 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1421 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1422 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1423 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1424
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1425 return self.buildAPURL(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1426 TYPE_ITEM, parent_ap_account, parent_item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1427 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1428
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1429 async def mbdata2APitem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1430 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1431 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1432 mb_data: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1433 public=True
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1434 ) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1435 """Convert Libervia Microblog Data to ActivityPub item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1436
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1437 @param mb_data: microblog data (as used in plugin XEP-0277) to convert
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1438 If ``public`` is True, ``service`` and ``node`` keys must be set.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1439 If ``published`` is not set, current datetime will be used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1440 @param public: True if the message is not a private/direct one
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1441 if True, the AP Item will be marked as public, and AP followers of target AP
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1442 account (which retrieve from ``service``) will be put in ``cc``.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1443 ``inReplyTo`` will also be set if suitable
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1444 if False, no destinee will be set (i.e., no ``to`` or ``cc`` or public flag).
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1445 This is usually used for direct messages.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1446 @return: AP item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1447 """
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1448 if not mb_data.get("id"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1449 mb_data["id"] = shortuuid.uuid()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1450 if not mb_data.get("author_jid"):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1451 mb_data["author_jid"] = client.jid.userhost()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1452 ap_account = await self.getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1453 jid.JID(mb_data["author_jid"]),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1454 None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1455 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1456 url_actor = self.buildAPURL(TYPE_ACTOR, ap_account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1457 url_item = self.buildAPURL(TYPE_ITEM, ap_account, mb_data["id"])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1458 ap_object = {
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1459 "id": url_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1460 "type": "Note",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1461 "published": utils.xmpp_date(mb_data.get("published")),
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1462 "attributedTo": url_actor,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1463 "content": mb_data.get("content_xhtml") or mb_data["content"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1464 }
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1465
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1466 language = mb_data.get("language")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1467 if language:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1468 ap_object["contentMap"] = {language: ap_object["content"]}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1469
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1470 if public:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1471 ap_object["to"] = [NS_AP_PUBLIC]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1472 try:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1473 node = mb_data["node"]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1474 service = jid.JID(mb_data["service"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1475 except KeyError:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1476 # node and service must always be specified when this method is used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1477 raise exceptions.InternalError(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1478 "node or service is missing in mb_data"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1479 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1480 target_ap_account = await self.getAPAccountFromJidAndNode(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1481 service, node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1482 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1483 if service.host == self.client.jid.userhost:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1484 # service is a proxy JID for AP account
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1485 actor_data = await self.getAPActorDataFromAccount(target_ap_account)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1486 followers = actor_data.get("followers")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1487 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1488 # service is a real XMPP entity
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1489 followers = self.buildAPURL(TYPE_FOLLOWERS, target_ap_account)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1490 if followers:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1491 ap_object["cc"] = [followers]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1492 if self._m.isCommentNode(node):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1493 parent_item = self._m.getParentItem(node)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1494 if service.host == self.client.jid.userhost():
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1495 # the publication is on a virtual node (i.e. an XMPP node managed by
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1496 # this gateway and linking to an ActivityPub actor)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1497 ap_object["inReplyTo"] = parent_item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1498 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1499 # the publication is from a followed real XMPP node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1500 ap_object["inReplyTo"] = await self.getReplyToIdFromXMPPNode(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1501 client,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1502 ap_account,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1503 parent_item,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1504 mb_data
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1505 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1506
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1507 ap_item = {
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1508 "@context": "https://www.w3.org/ns/activitystreams",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1509 "id": url_item,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1510 "type": "Create",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1511 "actor": url_actor,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1512 "object": ap_object
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1513 }
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1514
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1515 return ap_item
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1516
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1517 async def publishMessage(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1518 self,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1519 client: SatXMPPEntity,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1520 mess_data: dict,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1521 service: jid.JID
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1522 ) -> None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1523 """Send an AP message
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1524
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1525 .. note::
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1526
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1527 This is a temporary method used for development only
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1528
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1529 @param mess_data: message data. Following keys must be set:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1530
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1531 ``node``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1532 identifier of message which is being replied (this will
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1533 correspond to pubsub node in the future)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1534
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1535 ``content_xhtml`` or ``content``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1536 message body (respectively in XHTML or plain text)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1537
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1538 @param service: JID corresponding to the AP actor.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1539 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1540 if not service.user:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1541 raise ValueError("service must have a local part")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1542 account = self._e.unescape(service.user)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1543 ap_actor_data = await self.getAPActorDataFromAccount(account)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1544
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1545 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1546 inbox_url = ap_actor_data["endpoints"]["sharedInbox"]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1547 except KeyError:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1548 raise exceptions.DataError("Can't get ActivityPub actor inbox")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1549
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1550 item_data = await self.mbdata2APitem(client, mess_data)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1551 url_actor = item_data["object"]["attributedTo"]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1552 resp = await self.signAndPost(inbox_url, url_actor, item_data)
3684 8353cc3b8db9 component AP gateway: fix wrong operator when testing HTTP code Goffi <goffi@goffi.org> parents: 3682 diff changeset	1553 if resp.code != 202:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1554 raise exceptions.NetworkError(f"unexpected return code: {resp.code}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1555
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1556 @utils.ensure_deferred
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1557 async def onMessage(self, message_elt: domish.Element) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1558 """Called when a XMPP message is received"""
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1559 if not self.client:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1560 log.warning(f"no client set, ignoring message: {message_elt.toXml()}")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1561 return
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1562 mess_type = message_elt.getAttribute("type")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1563 if mess_type and mess_type not in ("chat", "normal"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1564 log.warning(f"ignoring message with unexpected type: {message_elt.toXml()}")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1565 return
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1566 from_jid = jid.JID(message_elt["from"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1567 if not self.isLocal(from_jid):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1568 log.warning(f"ignoring non local message: {message_elt.toXml()}")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1569 return
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1570 to_jid = jid.JID(message_elt["to"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1571 if not to_jid.user:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1572 log.warning(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1573 f"ignoring message addressed to gateway itself: {message_elt.toXml()}"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1574 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1575 return
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1576
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1577 actor_account = self._e.unescape(to_jid.user)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1578 actor_id = await self.getAPActorIdFromAccount(actor_account)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1579 inbox = await self.getAPInboxFromId(actor_id)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1580
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1581 mb_data = {
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1582 "content": str(message_elt.body),
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1583 }
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1584 client = self.client.getVirtualClient(from_jid)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1585 ap_item = await self.mbdata2APitem(client, mb_data, public=False)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1586 ap_item["object"]["to"] = ap_item["to"] = [actor_id]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1587 await self.signAndPost(inbox, ap_item["actor"], ap_item)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1588
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1589 async def newReplyToXMPPItem(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1590 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1591 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1592 ap_item: dict,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1593 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1594 """We got an AP item which is a reply to an XMPP item"""
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1595 in_reply_to = ap_item["inReplyTo"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1596 url_type, url_args = self.parseAPURL(in_reply_to)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1597 if url_type != "item":
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1598 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1599 "Ignoring AP item replying to an XMPP item with an unexpected URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1600 f"type({url_type!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1601 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1602 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1603 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1604 parent_item_account, parent_item_id = url_args[0].split("/", 1)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1605 except (IndexError, ValueError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1606 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1607 "Ignoring AP item replying to an XMPP item with invalid inReplyTo URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1608 f"({in_reply_to!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1609 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1610 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1611 parent_item_service, parent_item_node = await self.getJIDAndNode(parent_item_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1612 if parent_item_node is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1613 parent_item_node = self._m.namespace
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1614 items, __ = await self._p.getItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1615 client, parent_item_service, parent_item_node, item_ids=[parent_item_id]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1616 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1617 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1618 parent_item_elt = items[0]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1619 except IndexError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1620 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1621 f"Can't find parent item at {parent_item_service} (node "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1622 f"{parent_item_node!r})\n{pformat(ap_item)}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1623 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1624 parent_item_parsed = await self._m.item2mbdata(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1625 client, parent_item_elt, parent_item_service, parent_item_node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1626 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1627 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1628 comment_service = jid.JID(parent_item_parsed["comments"][0]["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1629 comment_node = parent_item_parsed["comments"][0]["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1630 except (KeyError, IndexError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1631 # we don't have a comment node set for this item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1632 from sat.tools.xml_tools import ppElt
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1633 log.info(f"{ppElt(parent_item_elt.toXml())}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1634 raise NotImplemented()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1635 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1636 __, item_elt = await self.apItem2MbDataAndElt(ap_item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1637 await self._p.publish(client, comment_service, comment_node, [item_elt])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1638
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1639 async def newAPItem(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1640 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1641 client: SatXMPPEntity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1642 destinee: Optional[jid.JID],
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1643 node: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1644 item: dict,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1645 ) -> None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1646 """Analyse, cache and send notification for received AP item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1647
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1648 @param destinee: jid of the destinee,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1649 @param node: XMPP pubsub node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1650 @param item: AP object payload
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1651 """
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1652 targets: Set[str] = set()
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1653 is_public = False
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1654 # TODO: handle "audience"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1655 for key in ("to", "bto", "cc", "bcc"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1656 values = item.get(key)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1657 if not values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1658 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1659 if isinstance(values, str):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1660 values = [values]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1661 for value in values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1662 if value in PUBLIC_TUPLE:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1663 is_public = True
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1664 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1665 if not value:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1666 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1667 if not self.isLocalURL(value):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1668 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1669 targets.add(value)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1670
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1671 targets_types = {self.parseAPURL(t)[0] for t in targets}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1672 if not is_public and targets_types == {TYPE_ACTOR}:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1673 # this is a direct message
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1674 await self.handleMessageAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1675 client, targets, destinee, item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1676 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1677 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1678 await self.handlePubsubAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1679 client, targets, destinee, node, item, is_public
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1680 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1681
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1682 async def handleMessageAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1683 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1684 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1685 targets: Set[str],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1686 destinee: Optional[jid.JID],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1687 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1688 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1689 """Parse and deliver direct AP items translating to XMPP messages
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1690
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1691 @param targets: actors where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1692 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1693 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1694 """
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1695 targets_jids = {await self.getJIDFromId(t) for t in targets}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1696 if destinee is not None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1697 targets_jids.add(destinee)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1698 mb_data = await self.apItem2MBdata(item)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1699 defer_l = []
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1700 for target_jid in targets_jids:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1701 defer_l.append(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1702 client.sendMessage(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1703 target_jid,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1704 {'': mb_data.get("content", "")},
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1705 mb_data.get("title"),
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1706
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1707 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1708 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1709 await defer.DeferredList(defer_l)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1710
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1711 async def handlePubsubAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1712 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1713 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1714 targets: Set[str],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1715 destinee: Optional[jid.JID],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1716 node: str,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1717 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1718 public: bool
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1719 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1720 """Analyse, cache and deliver AP items translating to Pubsub
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1721
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1722 @param targets: actors/collections where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1723 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1724 @param node: XMPP pubsub node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1725 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1726 @param public: True if the item is public
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1727 """
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1728 # XXX: "public" is not used for now
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1729
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1730 service = client.jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1731 in_reply_to = item.get("inReplyTo")
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1732
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1733 if in_reply_to and isinstance(in_reply_to, list):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1734 in_reply_to = in_reply_to[0]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1735 if in_reply_to and isinstance(in_reply_to, str):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1736 if self.isLocalURL(in_reply_to):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1737 # this is a reply to an XMPP item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1738 return await self.newReplyToXMPPItem(client, item)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1739
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1740 # this item is a reply to an AP item, we use or create a corresponding node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1741 # for comments
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1742 parent_node, __ = await self.getCommentsNodes(item["id"], in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1743 node = parent_node or node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1744 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1745 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1746 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1747 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1748 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1749 cached_node = await self.host.memory.storage.setPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1750 client,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1751 service,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1752 node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1753 subscribed=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1754 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1755 except IntegrityError as e:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1756 if "unique" in str(e.orig).lower():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1757 # the node may already exist, if it has been created just after
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1758 # getPubsubNode above
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1759 log.debug("ignoring UNIQUE constraint error")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1760 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1761 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1762 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1763 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1764 raise e
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1765
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1766 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1767 # it is a root item (i.e. not a reply to an other item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1768 cached_node = await self.host.memory.storage.getPubsubNode(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1769 client, service, node, with_subscriptions=True
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1770 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1771 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1772 log.warning(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1773 f"Received item in unknown node {node!r} at {service}. This may be "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1774 f"due to a cache purge. We synchronise the node\n{item}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1775
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1776 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1777 return
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1778 mb_data, item_elt = await self.apItem2MbDataAndElt(item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1779 await self.host.memory.storage.cachePubsubItems(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1780 client,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1781 cached_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1782 [item_elt],
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1783 [mb_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1784 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1785
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1786 for subscription in cached_node.subscriptions:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1787 if subscription.state != SubscriptionState.SUBSCRIBED:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1788 continue
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1789 self.pubsub_service.notifyPublish(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1790 service,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1791 node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1792 [(subscription.subscriber, None, [item_elt])]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1793 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1794
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1795 async def newAPDeleteItem(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1796 self,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1797 client: SatXMPPEntity,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1798 destinee: Optional[jid.JID],
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1799 node: str,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1800 item: dict,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1801 ) -> None:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1802 """Analyse, cache and send notification for received AP item
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1803
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1804 @param destinee: jid of the destinee,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1805 @param node: XMPP pubsub node
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1806 @param item: AP object payload
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1807 """
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1808 item_id = item.get("id")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1809 if not item_id:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1810 raise exceptions.DataError('"id" attribute is missing in item')
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1811 if self.isLocalURL(item_id):
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1812 raise ValueError("Local IDs should not be used")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1813
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1814 cached_node = await self.host.memory.storage.getPubsubNode(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1815 client, client.jid, node, with_subscriptions=True
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1816 )
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1817 if cached_node is None:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1818 log.warning(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1819 f"Received an item retract for node {node!r} at {client.jid} which is "
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1820 "not cached"
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1821 )
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1822 else:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1823 await self.host.memory.storage.deletePubsubItems(cached_node, [item_id])
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1824 # notifyRetract is expecting domish.Element instances
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1825 item_elt = domish.Element((None, "item"))
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1826 item_elt["id"] = item_id
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1827 for subscription in cached_node.subscriptions:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1828 if subscription.state != SubscriptionState.SUBSCRIBED:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1829 continue
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1830 self.pubsub_service.notifyRetract(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1831 client.jid,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1832 node,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1833 [(subscription.subscriber, None, [item_elt])]
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	1834 )

Mercurial > libervia-backend

annotate sat/plugins/plugin_comp_ap_gateway/__init__.py @ 3793:b5c9021020df

annotate sat/plugins/plugin_comp_ap_gateway/init.py @ 3793:b5c9021020df