libervia-backend: sat/plugins/plugin_comp_ap_gateway/__init_

annotate sat/plugins/plugin_comp_ap_gateway/init.py @ 3934:e345d93fb6e5

plugin OXPS: OpenPGP for XMPP Pubsub implementation: OpenPGP for XMPP Pubsub (https://xmpp.org/extensions/inbox/pubsub-encryption.html, currently a protoXEP) is implemented and activated when `encrypted` is set to `True` in pubsub's `extra` data. On item retrieval, the decryption is transparent if the key is known, except if the `decrypt` key in `extra` is set to `False` (notably useful when one wants to checks that data is well encrypted). Methods and corresponding bridge methods have been implemented to manage shared secrets (to share, revoke or rotate the secrets). plugin XEP-0060's `XEP-0060_publish` trigger point as been move before actual publish so item can be modified (here e2ee) by the triggers. A new `XEP-0060_items` trigger point has also been added. `encrypted` flag can be used with plugin XEP-0277's microblog data rel 380

author	Goffi <goffi@goffi.org>
date	Sat, 15 Oct 2022 20:36:53 +0200
parents	0aa7023dcd08
children	6fa4ca0c047e

rev	line source
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia ActivityPub Gateway
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 import base64
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	20 import calendar
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	21 import hashlib
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	22 import json
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	23 from os import access
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	24 from pathlib import Path
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	25 from pprint import pformat
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	26 import re
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	27 from typing import (
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	28 Any,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	29 Awaitable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	30 Callable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	31 Dict,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	32 List,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	33 Optional,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	34 Set,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	35 Tuple,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	36 Type,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	37 Union,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	38 overload,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	39 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	40 from urllib import parse
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	41
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	42 from cryptography.exceptions import InvalidSignature
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	43 from cryptography.hazmat.primitives import serialization
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	44 from cryptography.hazmat.primitives import hashes
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	45 from cryptography.hazmat.primitives.asymmetric import rsa
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	46 from cryptography.hazmat.primitives.asymmetric import padding
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	47 import dateutil
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	48 from dateutil.parser import parserinfo
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	49 import shortuuid
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	50 from sqlalchemy.exc import IntegrityError
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 import treq
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	52 from treq.response import _Response as TReqResponse
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	53 from twisted.internet import defer, reactor, threads
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	54 from twisted.web import http
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	55 from twisted.words.protocols.jabber import error, jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	56 from twisted.words.xish import domish
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	57 from wokkel import pubsub, rsm
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	58
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	59 from sat.core import exceptions
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	60 from sat.core.constants import Const as C
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	61 from sat.core.core_types import SatXMPPEntity
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	62 from sat.core.i18n import _
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	63 from sat.core.log import getLogger
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	64 from sat.memory import persistent
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	65 from sat.memory.sqla_mapping import History, SubscriptionState
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	66 from sat.tools import utils
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	67 from sat.tools.common import data_format, date_utils, tls, uri
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	68 from sat.tools.common.async_utils import async_lru
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	69
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	70 from .ad_hoc import APAdHocService
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	71 from .events import APEvents
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	72 from .constants import (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	73 ACTIVITY_OBJECT_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	74 ACTIVITY_TARGET_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	75 ACTIVITY_TYPES,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	76 ACTIVITY_TYPES_LOWER,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	77 COMMENTS_MAX_PARENTS,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	78 CONF_SECTION,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	79 IMPORT_NAME,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	80 LRU_MAX_SIZE,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	81 MEDIA_TYPE_AP,
3846 cc13efdd8360 component AP gateway: return item when `item` URL is used: Goffi <goffi@goffi.org> parents: 3845 diff changeset	82 NS_AP,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	83 NS_AP_PUBLIC,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	84 PUBLIC_TUPLE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	85 TYPE_ACTOR,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	86 TYPE_EVENT,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	87 TYPE_FOLLOWERS,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	88 TYPE_ITEM,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	89 TYPE_LIKE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	90 TYPE_MENTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	91 TYPE_REACTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	92 TYPE_TOMBSTONE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	93 TYPE_JOIN,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	94 TYPE_LEAVE
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	95 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	96 from .http_server import HTTPServer
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	97 from .pubsub_service import APPubsubService
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	98 from .regex import RE_MENTION
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	99
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	100
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	101 log = getLogger(__name__)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	102
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	103 IMPORT_NAME = "ap-gateway"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	104
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	105 PLUGIN_INFO = {
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	106 C.PI_NAME: "ActivityPub Gateway component",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	107 C.PI_IMPORT_NAME: IMPORT_NAME,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	108 C.PI_MODES: [C.PLUG_MODE_COMPONENT],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	109 C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	110 C.PI_PROTOCOLS: [],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	111 C.PI_DEPENDENCIES: [
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	112 "XEP-0050", "XEP-0054", "XEP-0060", "XEP-0084", "XEP-0106", "XEP-0277",
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	113 "XEP-0292", "XEP-0329", "XEP-0372", "XEP-0424", "XEP-0465", "XEP-0470",
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	114 "PUBSUB_CACHE", "TEXT_SYNTAXES", "IDENTITY", "EVENTS"
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	115 ],
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	116 C.PI_RECOMMENDATIONS: [],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	117 C.PI_MAIN: "APGateway",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	118 C.PI_HANDLER: C.BOOL_TRUE,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	119 C.PI_DESCRIPTION: _(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	120 "Gateway for bidirectional communication between XMPP and ActivityPub."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	121 ),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	122 }
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	123
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	124 HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	125 RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	126 RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	127 RE_ALLOWED_UNQUOTED = re.compile(r"^[a-zA-Z0-9_-]+$")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	128
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	129
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 class APGateway:
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	131 IMPORT_NAME = IMPORT_NAME
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	132 # show data send or received through HTTP, used for debugging
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	133 # 1: log POST objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	134 # 2: log POST and GET objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	135 # 3: log POST and GET objects with HTTP headers for GET requests
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	136 verbose = 0
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	137
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	138 def __init__(self, host):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	139 self.host = host
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	140 self.initialised = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	141 self.client = None
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	142 self._p = host.plugins["XEP-0060"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	143 self._a = host.plugins["XEP-0084"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	144 self._e = host.plugins["XEP-0106"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	145 self._m = host.plugins["XEP-0277"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	146 self._v = host.plugins["XEP-0292"]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	147 self._refs = host.plugins["XEP-0372"]
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	148 self._r = host.plugins["XEP-0424"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	149 self._pps = host.plugins["XEP-0465"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	150 self._c = host.plugins["PUBSUB_CACHE"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	151 self._t = host.plugins["TEXT_SYNTAXES"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	152 self._i = host.plugins["IDENTITY"]
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	153 self._pa = host.plugins["XEP-0470"]
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	154 self._events = host.plugins["EVENTS"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	155 self._p.addManagedNode(
3850 4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	156 "",
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	157 items_cb=self._itemsReceived,
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	158 # we want to be sure that the callbacks are launched before pubsub cache's
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	159 # one, as we need to inspect items before they are actually removed from cache
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	160 # or updated
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	161 priority=1000
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	162 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	163 self.pubsub_service = APPubsubService(self)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	164 self.ad_hoc = APAdHocService(self)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	165 self.ap_events = APEvents(self)
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	166 host.trigger.add("messageReceived", self._messageReceivedTrigger, priority=-1000)
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	167 host.trigger.add("XEP-0424_retractReceived", self._onMessageRetract)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	168 host.trigger.add("XEP-0372_ref_received", self._onReferenceReceived)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	169
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	170 host.bridge.addMethod(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	171 "APSend",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	172 ".plugin",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	173 in_sign="sss",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	174 out_sign="",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	175 method=self._publishMessage,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	176 async_=True,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	177 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	178
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	179 def getHandler(self, __):
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	180 return self.pubsub_service
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	181
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	182 async def init(self, client):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	183 if self.initialised:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	184 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	185
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	186 self.initialised = True
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	187 log.info(_("ActivityPub Gateway initialization"))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	188
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	189 # RSA keys
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	190 stored_data = await self.host.memory.storage.getPrivates(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	191 IMPORT_NAME, ["rsa_key"], profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	192 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	193 private_key_pem = stored_data.get("rsa_key")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	194 if private_key_pem is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	195 self.private_key = await threads.deferToThread(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	196 rsa.generate_private_key,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	197 public_exponent=65537,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	198 key_size=4096,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	199 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	200 private_key_pem = self.private_key.private_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	201 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	202 format=serialization.PrivateFormat.PKCS8,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	203 encryption_algorithm=serialization.NoEncryption()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	204 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	205 await self.host.memory.storage.setPrivateValue(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	206 IMPORT_NAME, "rsa_key", private_key_pem, profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	207 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 self.private_key = serialization.load_pem_private_key(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	210 private_key_pem.encode(),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	211 password=None,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	212 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	213 self.public_key = self.private_key.public_key()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	214 self.public_key_pem = self.public_key.public_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	215 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	216 format=serialization.PublicFormat.SubjectPublicKeyInfo
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	217 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	218
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	219 # params
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	220 # URL and port
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	221 self.public_url = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	222 CONF_SECTION, "public_url"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	223 ) or self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 CONF_SECTION, "xmpp_domain"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	225 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 if self.public_url is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	227 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 '"public_url" not set in configuration, this is mandatory to have'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	229 "ActivityPub Gateway running. Please set this option it to public facing "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	230 f"url in {CONF_SECTION!r} configuration section."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	231 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	232 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	233 if parse.urlparse(self.public_url).scheme:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	234 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	235 "Scheme must not be specified in \"public_url\", please remove it from "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	236 "\"public_url\" configuration option. ActivityPub Gateway won't be run."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	237 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	238 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	239 self.http_port = int(self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	240 CONF_SECTION, 'http_port', 8123))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	241 connection_type = self.host.memory.getConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	242 CONF_SECTION, 'http_connection_type', 'https')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	243 if connection_type not in ('http', 'https'):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	244 raise exceptions.ConfigError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	245 'bad ap-gateay http_connection_type, you must use one of "http" or '
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 '"https"'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	247 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	248 self.max_items = int(self.host.memory.getConfig(
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	249 CONF_SECTION, 'new_node_max_items', 50
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	250
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	251 ))
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	252 self.comments_max_depth = int(self.host.memory.getConfig(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	253 CONF_SECTION, 'comments_max_depth', 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	254 ))
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	255 self.ap_path = self.host.memory.getConfig(CONF_SECTION, 'ap_path', '_ap')
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	256 self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	257 # True (default) if we provide gateway only to entities/services from our server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	258 self.local_only = C.bool(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	259 self.host.memory.getConfig(CONF_SECTION, 'local_only', C.BOOL_TRUE)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	260 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	261 # if True (default), mention will be parsed in non-private content coming from
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	262 # XMPP. This is necessary as XEP-0372 are coming separately from item where the
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	263 # mention is done, which is hard to impossible to translate to ActivityPub (where
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	264 # mention specified inside the item directly). See documentation for details.
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	265 self.auto_mentions = C.bool(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	266 self.host.memory.getConfig(CONF_SECTION, "auto_mentions", C.BOOL_TRUE)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	267 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	268
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	269 # HTTP server launch
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	270 self.server = HTTPServer(self)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	271 if connection_type == 'http':
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	272 reactor.listenTCP(self.http_port, self.server)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	273 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	274 options = tls.getOptionsFromConfig(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	275 self.host.memory.config, CONF_SECTION)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	276 tls.TLSOptionsCheck(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	277 context_factory = tls.getTLSContextFactory(options)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	278 reactor.listenSSL(self.http_port, self.server, context_factory)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	279
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	280 async def profileConnecting(self, client):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	281 self.client = client
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	282 client.sendHistory = True
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	283 client._ap_storage = persistent.LazyPersistentBinaryDict(
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	284 IMPORT_NAME,
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	285 client.profile
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	286 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	287 await self.init(client)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	288
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	289 def profileConnected(self, client):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	290 self.ad_hoc.init(client)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	291
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	292 async def _itemsReceived(
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	293 self,
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	294 client: SatXMPPEntity,
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	295 itemsEvent: pubsub.ItemsEvent
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	296 ) -> None:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	297 """Callback called when pubsub items are received
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	298
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	299 if the items are adressed to a JID corresponding to an AP actor, they are
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	300 converted to AP items and sent to the corresponding AP server.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	301
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	302 If comments nodes are linked, they are automatically subscribed to get new items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	303 from there too.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	304 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	305 if client != self.client:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	306 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	307 # we need recipient as JID and not gateway own JID to be able to use methods such
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	308 # as "subscribe"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	309 client = self.client.getVirtualClient(itemsEvent.sender)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	310 recipient = itemsEvent.recipient
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	311 if not recipient.user:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	312 log.debug("ignoring items event without local part specified")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	313 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	314
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	315 ap_account = self._e.unescape(recipient.user)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	316
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	317 if self._pa.isAttachmentNode(itemsEvent.nodeIdentifier):
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	318 await self.convertAndPostAttachments(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	319 client, ap_account, itemsEvent.sender, itemsEvent.nodeIdentifier,
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	320 itemsEvent.items
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	321 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	322 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	323 await self.convertAndPostItems(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	324 client, ap_account, itemsEvent.sender, itemsEvent.nodeIdentifier,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	325 itemsEvent.items
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	326 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	327
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	328 async def getVirtualClient(self, actor_id: str) -> SatXMPPEntity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	329 """Get client for this component with a specified jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	330
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	331 This is needed to perform operations with the virtual JID corresponding to the AP
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	332 actor instead of the JID of the gateway itself.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	333 @param actor_id: ID of the actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	334 @return: virtual client
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	335 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	336 local_jid = await self.getJIDFromId(actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	337 return self.client.getVirtualClient(local_jid)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	338
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	339 def is_activity(self, data: dict) -> bool:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	340 """Return True if the data has an activity type"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	341 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	342 return (data.get("type") or "").lower() in ACTIVITY_TYPES_LOWER
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	343 except (KeyError, TypeError):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	344 return False
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	345
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	346 async def apGet(self, url: str) -> dict:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	347 """Retrieve AP JSON from given URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	348
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	349 @raise error.StanzaError: "service-unavailable" is sent when something went wrong
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	350 with AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	351 """
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	352 resp = await treq.get(
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	353 url,
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	354 headers = {
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	355 "Accept": [MEDIA_TYPE_AP],
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	356 "Content-Type": [MEDIA_TYPE_AP],
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	357 }
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	358 )
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	359 if resp.code >= 300:
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	360 text = await resp.text()
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	361 if resp.code == 404:
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	362 raise exceptions.NotFound(f"Can't find resource at {url}")
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	363 else:
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	364 msg = f"HTTP error {resp.code}: {text}"
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	365 raise exceptions.ExternalRequestError(msg)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	366 try:
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	367 return await treq.json_content(resp)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	368 except Exception as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	369 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	370 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	371 text=f"Can't get AP data at {url}: {e}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	372 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	373
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	374 @overload
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	375 async def apGetObject(self, data: dict, key: str) -> Optional[dict]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	376 ...
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	377
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	378 @overload
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	379 async def apGetObject(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	380 self, data: Union[str, dict], key: None = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	381 ) -> dict:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	382 ...
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	383
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	384 async def apGetObject(self, data, key = None):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	385 """Retrieve an AP object, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	386
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	387 This method is to be used with attributes marked as "Functional" in
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	388 https://www.w3.org/TR/activitystreams-vocabulary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	389 @param data: AP object where an other object is looked for, or the object itself
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	390 @param key: name of the object to look for, or None if data is the object directly
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	391 @return: found object if any
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	392 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	393 if key is not None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	394 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	395 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	396 value = data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	397 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	398 if key is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	399 raise ValueError("None can't be used with apGetObject is key is None")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	400 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	401 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	402 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	403 elif isinstance(value, str):
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	404 if self.isLocalURL(value):
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	405 return await self.apGetLocalObject(value)
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	406 else:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	407 return await self.apGet(value)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	408 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	409 raise NotImplementedError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	410 "was expecting a string or a dict, got {type(value)}: {value!r}}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	411 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	412
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	413 async def apGetLocalObject(
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	414 self,
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	415 url: str
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	416 ) -> dict:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	417 """Retrieve or generate local object
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	418
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	419 for now, only handle XMPP items to convert to AP
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	420 """
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	421 url_type, url_args = self.parseAPURL(url)
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	422 if url_type == TYPE_ITEM:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	423 try:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	424 account, item_id = url_args
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	425 except ValueError:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	426 raise ValueError(f"invalid URL: {url}")
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	427 author_jid, node = await self.getJIDAndNode(account)
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	428 if node is None:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	429 node = self._m.namespace
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	430 cached_node = await self.host.memory.storage.getPubsubNode(
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	431 self.client, author_jid, node
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	432 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	433 if not cached_node:
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	434 log.debug(f"node {node!r} at {author_jid} is not found in cache")
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	435 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	436 else:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	437 cached_items, __ = await self.host.memory.storage.getItems(
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	438 cached_node, item_ids=[item_id]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	439 )
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	440 if not cached_items:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	441 log.debug(
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	442 f"item {item_id!r} of {node!r} at {author_jid} is not found in "
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	443 "cache"
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	444 )
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	445 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	446 else:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	447 found_item = cached_items[0].data
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	448
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	449 if found_item is None:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	450 # the node is not in cache, we have to make a request to retrieve the item
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	451 # If the node doesn't exist, getItems will raise a NotFound exception
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	452 found_items, __ = await self._p.getItems(
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	453 self.client, author_jid, node, item_ids=[item_id]
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	454 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	455 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	456 found_item = found_items[0]
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	457 except IndexError:
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	458 raise exceptions.NotFound("requested items can't be found")
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	459
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	460 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	461 # this is an event
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	462 event_data = self._events.event_elt_2_event_data(found_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	463 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	464 event_data, author_jid
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	465 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	466 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	467 ap_item["object"]["@context"] = ap_item["@context"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	468 return ap_item["object"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	469 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	470 # this is a blog item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	471 mb_data = await self._m.item2mbdata(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	472 self.client, found_item, author_jid, node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	473 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	474 ap_item = await self.mb_data_2_ap_item(self.client, mb_data)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	475 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	476 return ap_item["object"]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	477 else:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	478 raise NotImplementedError(
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	479 'only object from "item" URLs can be retrieved for now'
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	480 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	481
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	482 async def apGetList(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	483 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	484 data: dict,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	485 key: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	486 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	487 ) -> Optional[List[Dict[str, Any]]]:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	488 """Retrieve a list of objects from AP data, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	489
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	490 This method is to be used with non functional vocabularies. Use ``apGetObject``
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	491 otherwise.
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	492 If the value is a dictionary, it will be wrapped in a list
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	493 @param data: AP object where a list of objects is looked for
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	494 @param key: key of the list to look for
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	495 @param only_ids: if Trye, only items IDs are retrieved
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	496 @return: list of objects, or None if the key is not present
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	497 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	498 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	499 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	500 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	501 elif isinstance(value, str):
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	502 if self.isLocalURL(value):
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	503 value = await self.apGetLocalObject(value)
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	504 else:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	505 value = await self.apGet(value)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	506 if isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	507 return [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	508 if not isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	509 raise ValueError(f"A list was expected, got {type(value)}: {value!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	510 if only_ids:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	511 return [
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	512 {"id": v["id"]} if isinstance(v, dict) else {"id": v}
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	513 for v in value
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	514 ]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	515 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	516 return [await self.apGetObject(i) for i in value]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	517
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	518 async def apGetActors(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	519 self,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	520 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	521 key: str,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	522 as_account: bool = True
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	523 ) -> List[str]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	524 """Retrieve AP actors from data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	525
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	526 @param data: AP object containing a field with actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	527 @param key: field to use to retrieve actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	528 @param as_account: if True returns account handles, otherwise will return actor
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	529 IDs
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	530 @raise exceptions.DataError: there is not actor data or it is invalid
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	531 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	532 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	533 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	534 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	535 f"no actor associated to object {data.get('id')!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	536 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	537 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	538 actor_id = value.get("id")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	539 if actor_id is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	540 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	541 f"invalid actor associated to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	542 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	543 value = [actor_id]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	544 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	545 value = [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	546 elif isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	547 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	548 value = [a if isinstance(a, str) else a["id"] for a in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	549 except (TypeError, KeyError):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	550 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	551 f"invalid actors list to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	552 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	553 if not value:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	554 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	555 f"list of actors is empty"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	556 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	557 if as_account:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	558 return [await self.getAPAccountFromId(actor_id) for actor_id in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	559 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	560 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	561
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	562 async def apGetSenderActor(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	563 self,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	564 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	565 ) -> str:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	566 """Retrieve actor who sent data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	567
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	568 This is done by checking "actor" field first, then "attributedTo" field.
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	569 Only the first found actor is taken into account
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	570 @param data: AP object
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	571 @return: actor id of the sender
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	572 @raise exceptions.NotFound: no actor has been found in data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	573 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	574 try:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	575 actors = await self.apGetActors(data, "actor", as_account=False)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	576 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	577 actors = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	578 if not actors:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	579 try:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	580 actors = await self.apGetActors(data, "attributedTo", as_account=False)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	581 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	582 raise exceptions.NotFound(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	583 'actor not specified in "actor" or "attributedTo"'
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	584 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	585 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	586 return actors[0]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	587 except IndexError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	588 raise exceptions.NotFound("list of actors is empty")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	589
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	590 def mustEncode(self, text: str) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	591 """Indicate if a text must be period encoded"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	592 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	593 not RE_ALLOWED_UNQUOTED.match(text)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	594 or text.startswith("___")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	595 or "---" in text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	596 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	597
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	598 def periodEncode(self, text: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	599 """Period encode a text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	600
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	601 see [getJIDAndNode] for reasons of period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	602 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	603 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	604 parse.quote(text, safe="")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	605 .replace("---", "%2d%2d%2d")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	606 .replace("___", "%5f%5f%5f")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	607 .replace(".", "%2e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	608 .replace("~", "%7e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	609 .replace("%", ".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	610 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	611
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	612 async def getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	613 self,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	614 jid_: jid.JID,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	615 node: Optional[str]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	616 ) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	617 """Construct AP account from JID and node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	618
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	619 The account construction will use escaping when necessary
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	620 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	621 if not node or node == self._m.namespace:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	622 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	623
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	624 if self.client is None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	625 raise exceptions.InternalError("Client is not set yet")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	626
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	627 if self.isVirtualJID(jid_):
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	628 # this is an proxy JID to an AP Actor
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	629 return self._e.unescape(jid_.user)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	630
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	631 if node and not jid_.user and not self.mustEncode(node):
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	632 is_pubsub = await self.isPubsub(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	633 # when we have a pubsub service, the user part can be used to set the node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	634 # this produces more user-friendly AP accounts
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	635 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	636 jid_.user = node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	637 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	638
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	639 is_local = self.isLocal(jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	640 user = jid_.user if is_local else jid_.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	641 if user is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	642 user = ""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	643 account_elts = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	644 if node and self.mustEncode(node) or self.mustEncode(user):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	645 account_elts = ["___"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	646 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	647 node = self.periodEncode(node)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	648 user = self.periodEncode(user)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	649
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	650 if not user:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	651 raise exceptions.InternalError("there should be a user part")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	652
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	653 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	654 account_elts.extend((node, "---"))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	655
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	656 account_elts.extend((
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	657 user, "@", jid_.host if is_local else self.client.jid.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	658 ))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	659 return "".join(account_elts)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	660
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	661 def isLocal(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	662 """Returns True if jid_ use a domain or subdomain of gateway's host"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	663 local_host = self.client.host.split(".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	664 assert local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	665 return jid_.host.split(".")[-len(local_host):] == local_host
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	666
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	667 async def isPubsub(self, jid_: jid.JID) -> bool:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	668 """Indicate if a JID is a Pubsub service"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	669 host_disco = await self.host.getDiscoInfos(self.client, jid_)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	670 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	671 ("pubsub", "service") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	672 and not ("pubsub", "pep") in host_disco.identities
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	673 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	674
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	675 async def getJIDAndNode(self, ap_account: str) -> Tuple[jid.JID, Optional[str]]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	676 """Decode raw AP account handle to get XMPP JID and Pubsub Node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	677
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	678 Username are case insensitive.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	679
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	680 By default, the username correspond to local username (i.e. username from
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	681 component's server).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	682
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	683 If local name's domain is a pubsub service (and not PEP), the username is taken as
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	684 a pubsub node.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	685
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	686 If ``---`` is present in username, the part before is used as pubsub node, and the
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	687 rest as a JID user part.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	688
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	689 If username starts with ``___``, characters are encoded using period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	690 (i.e. percent encoding where a ``.`` is used instead of ``%``).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	691
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	692 This horror is necessary due to limitation in some AP implementation (notably
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	693 Mastodon), cf. https://github.com/mastodon/mastodon/issues/17222
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	694
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	695 examples:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	696
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	697 ``toto@example.org`` => JID = toto@example.org, node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	698
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	699 ``___toto.40example.net@example.org`` => JID = toto@example.net (this one is a
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	700 non-local JID, and will work only if setings ``local_only`` is False), node = None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	701
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	702 ``toto@pubsub.example.org`` (with pubsub.example.org being a pubsub service) =>
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	703 JID = pubsub.example.org, node = toto
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	704
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	705 ``tata---toto@example.org`` => JID = toto@example.org, node = tata
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	706
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	707 ``___urn.3axmpp.3amicroblog.3a0@pubsub.example.org`` (with pubsub.example.org
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	708 being a pubsub service) ==> JID = pubsub.example.org, node = urn:xmpp:microblog:0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	709
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	710 @param ap_account: ActivityPub account handle (``username@domain.tld``)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	711 @return: service JID and pubsub node
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	712 if pubsub node is None, default microblog pubsub node (and possibly other
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	713 nodes that plugins may hanlde) will be used
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	714 @raise ValueError: invalid account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	715 @raise PermissionError: non local jid is used when gateway doesn't allow them
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	716 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	717 if ap_account.count("@") != 1:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	718 raise ValueError("Invalid AP account")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	719 if ap_account.startswith("___"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	720 encoded = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	721 ap_account = ap_account[3:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	722 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	723 encoded = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	724
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	725 username, domain = ap_account.split("@")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	726
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	727 if "---" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	728 node, username = username.rsplit("---", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	729 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	730 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	731
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	732 if encoded:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	733 username = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	734 RE_PERIOD_ENC.sub(r"%\g<hex>", username),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	735 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	736 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	737 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	738 node = parse.unquote(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	739 RE_PERIOD_ENC.sub(r"%\g<hex>", node),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	740 errors="strict"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	741 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	742
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	743 if "@" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	744 username, domain = username.rsplit("@", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	745
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	746 if not node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	747 # we need to check host disco, because disco request to user may be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	748 # blocked for privacy reason (see
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	749 # https://xmpp.org/extensions/xep-0030.html#security)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	750 is_pubsub = await self.isPubsub(jid.JID(domain))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	751
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	752 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	753 # if the host is a pubsub service and not a PEP, we consider that username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	754 # is in fact the node name
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	755 node = username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	756 username = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	757
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	758 jid_s = f"{username}@{domain}" if username else domain
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	759 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	760 jid_ = jid.JID(jid_s)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	761 except RuntimeError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	762 raise ValueError(f"Invalid jid: {jid_s!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	763
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	764 if self.local_only and not self.isLocal(jid_):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	765 raise exceptions.PermissionError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	766 "This gateway is configured to map only local entities and services"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	767 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	768
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	769 return jid_, node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	770
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	771 def getLocalJIDFromAccount(self, account: str) -> jid.JID:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	772 """Compute JID linking to an AP account
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	773
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	774 The local jid is computer by escaping AP actor handle and using it as local part
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	775 of JID, where domain part is this gateway own JID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	776 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	777 return jid.JID(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	778 None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	779 (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	780 self._e.escape(account),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	781 self.client.jid.host,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	782 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	783 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	784 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	785
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	786 async def getJIDFromId(self, actor_id: str) -> jid.JID:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	787 """Compute JID linking to an AP Actor ID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	788
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	789 The local jid is computer by escaping AP actor handle and using it as local part
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	790 of JID, where domain part is this gateway own JID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	791 If the actor_id comes from local server (checked with self.public_url), it means
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	792 that we have an XMPP entity, and the original JID is returned
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	793 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	794 if self.isLocalURL(actor_id):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	795 request_type, extra_args = self.parseAPURL(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	796 if request_type != TYPE_ACTOR or len(extra_args) != 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	797 raise ValueError(f"invalid actor id: {actor_id!r}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	798 actor_jid, __ = await self.getJIDAndNode(extra_args[0])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	799 return actor_jid
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	800
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	801 account = await self.getAPAccountFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	802 return self.getLocalJIDFromAccount(account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	803
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	804 def parseAPURL(self, url: str) -> Tuple[str, List[str]]:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	805 """Parse an URL leading to an AP endpoint
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	806
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	807 @param url: URL to parse (schema is not mandatory)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	808 @return: endpoint type and extra arguments
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	809 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	810 path = parse.urlparse(url).path.lstrip("/")
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	811 type_, *extra_args = path[len(self.ap_path):].lstrip("/").split("/")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	812 return type_, [parse.unquote(a) for a in extra_args]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	813
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	814 def buildAPURL(self, type_:str , *args: str) -> str:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	815 """Build an AP endpoint URL
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	816
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	817 @param type_: type of AP endpoing
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	818 @param arg: endpoint dependant arguments
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	819 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	820 return parse.urljoin(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	821 self.base_ap_url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	822 str(Path(type_).joinpath(*(parse.quote_plus(a) for a in args)))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	823 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	824
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	825 def isLocalURL(self, url: str) -> bool:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	826 """Tells if an URL link to this component
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	827
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	828 ``public_url`` and ``ap_path`` are used to check the URL
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	829 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	830 return url.startswith(self.base_ap_url)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	831
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	832 def isVirtualJID(self, jid_: jid.JID) -> bool:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	833 """Tell if a JID is an AP actor mapped through this gateway"""
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	834 return jid_.host == self.client.jid.userhost()
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	835
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	836 def buildSignatureHeader(self, values: Dict[str, str]) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	837 """Build key="<value>" signature header from signature data"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	838 fields = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	839 for key, value in values.items():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	840 if key not in ("(created)", "(expired)"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	841 if '"' in value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	842 raise NotImplementedError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	843 "string escaping is not implemented, double-quote can't be used "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	844 f"in {value!r}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	845 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	846 value = f'"{value}"'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	847 fields.append(f"{key}={value}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	848
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	849 return ",".join(fields)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	850
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	851 def getDigest(self, body: bytes, algo="SHA-256") -> Tuple[str, str]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	852 """Get digest data to use in header and signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	853
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	854 @param body: body of the request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	855 @return: hash name and digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	856 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	857 if algo != "SHA-256":
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	858 raise NotImplementedError("only SHA-256 is implemented for now")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	859 return algo, base64.b64encode(hashlib.sha256(body).digest()).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	860
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	861 @async_lru(maxsize=LRU_MAX_SIZE)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	862 async def getActorData(self, actor_id) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	863 """Retrieve actor data with LRU cache"""
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	864 return await self.apGet(actor_id)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	865
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	866 @async_lru(maxsize=LRU_MAX_SIZE)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	867 async def getActorPubKeyData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	868 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	869 actor_id: str
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	870 ) -> Tuple[str, str, rsa.RSAPublicKey]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	871 """Retrieve Public Key data from actor ID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	872
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	873 @param actor_id: actor ID (url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	874 @return: key_id, owner and public_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	875 @raise KeyError: publicKey is missing from actor data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	876 """
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	877 actor_data = await self.getActorData(actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	878 pub_key_data = actor_data["publicKey"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	879 key_id = pub_key_data["id"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	880 owner = pub_key_data["owner"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	881 pub_key_pem = pub_key_data["publicKeyPem"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	882 pub_key = serialization.load_pem_public_key(pub_key_pem.encode())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	883 return key_id, owner, pub_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	884
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	885 def createActivity(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	886 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	887 activity: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	888 actor_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	889 object_: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	890 target: Optional[Union[str, dict]] = None,
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	891 activity_id: Optional[str] = None,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	892 **kwargs,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	893 ) -> Dict[str, Any]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	894 """Generate base data for an activity
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	895
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	896 @param activity: one of ACTIVITY_TYPES
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	897 @param actor_id: AP actor ID of the sender
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	898 @param object_: content of "object" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	899 @param target: content of "target" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	900 @param activity_id: ID to use for the activity
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	901 if not set it will be automatically generated, but it is usually desirable to
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	902 set the ID manually so it can be retrieved (e.g. for Undo)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	903 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	904 if activity not in ACTIVITY_TYPES:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	905 raise exceptions.InternalError(f"invalid activity: {activity!r}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	906 if object_ is None and activity in ACTIVITY_OBJECT_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	907 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	908 f'"object_" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	909 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	910 if target is None and activity in ACTIVITY_TARGET_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	911 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	912 f'"target" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	913 )
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	914 if activity_id is None:
4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	915 activity_id = f"{actor_id}#{activity.lower()}_{shortuuid.uuid()}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	916 data: Dict[str, Any] = {
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	917 "@context": [NS_AP],
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	918 "actor": actor_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	919 "id": activity_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	920 "type": activity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	921 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	922 data.update(kwargs)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	923 if object_ is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	924 data["object"] = object_
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	925 if target is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	926 data["target"] = target
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	927
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	928 return data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	929
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	930 def getKeyId(self, actor_id: str) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	931 """Get local key ID from actor ID"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	932 return f"{actor_id}#main-key"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	933
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	934 async def checkSignature(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	935 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	936 signature: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	937 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	938 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	939 ) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	940 """Verify that signature matches given headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	941
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	942 see https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-06#section-3.1.2
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	943
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	944 @param signature: Base64 encoded signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	945 @param key_id: ID of the key used to sign the data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	946 @param headers: headers and their values, including pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	947 @return: id of the signing actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	948
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	949 @raise InvalidSignature: signature doesn't match headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	950 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	951 to_sign = "\n".join(f"{k.lower()}: {v}" for k,v in headers.items())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	952 if key_id.startswith("acct:"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	953 actor = key_id[5:]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	954 actor_id = await self.getAPActorIdFromAccount(actor)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	955 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	956 actor_id = key_id.split("#", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	957
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	958 pub_key_id, pub_key_owner, pub_key = await self.getActorPubKeyData(actor_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	959 if pub_key_id != key_id or pub_key_owner != actor_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	960 raise exceptions.EncryptionError("Public Key mismatch")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	961
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	962 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	963 pub_key.verify(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	964 base64.b64decode(signature),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	965 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	966 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	967 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	968 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	969 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	970 except InvalidSignature:
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	971 raise exceptions.EncryptionError(
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	972 "Invalid signature (using PKC0S1 v1.5 and SHA-256)"
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	973 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	974
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	975 return actor_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	976
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	977 def getSignatureData(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	978 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	979 key_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	980 headers: Dict[str, str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	981 ) -> Tuple[Dict[str, str], Dict[str, str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	982 """Generate and return signature and corresponding headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	983
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	984 @param parsed_url: URL where the request is sent/has been received
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	985 @param key_id: ID of the key (URL linking to the data with public key)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	986 @param date: HTTP datetime string of signature generation
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	987 @param body: body of the HTTP request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	988 @param headers: headers to sign and their value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	989 default value will be used if not specified
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	990
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	991 @return: headers and signature data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	992 ``headers`` is an updated copy of ``headers`` arguments, with pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	993 removed, and ``Signature`` added.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	994 """
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	995 # headers must be lower case
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	996 l_headers: Dict[str, str] = {k.lower(): v for k, v in headers.items()}
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	997 to_sign = "\n".join(f"{k}: {v}" for k,v in l_headers.items())
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	998 signature = base64.b64encode(self.private_key.sign(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	999 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1000 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1001 padding.PKCS1v15(), # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1002 hashes.SHA256() # type: ignore
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1003 )).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1004 sign_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1005 "keyId": key_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1006 "Algorithm": "rsa-sha256",
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1007 "headers": " ".join(l_headers.keys()),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1008 "signature": signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1009 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1010 new_headers = {k: v for k,v in headers.items() if not k.startswith("(")}
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1011 new_headers["Signature"] = self.buildSignatureHeader(sign_data)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1012 return new_headers, sign_data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1013
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1014 async def convertAndPostItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1015 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1016 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1017 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1018 service: jid.JID,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1019 node: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1020 items: List[domish.Element],
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1021 subscribe_extra_nodes: bool = True,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1022 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1023 """Convert XMPP items to AP items and post them to actor inbox
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1024
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1025 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1026 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1027 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1028 @param node: (virtual) node corresponding where the item has been published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1029 @param subscribe_extra_nodes: if True, extra data nodes will be automatically
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1030 subscribed, that is comment nodes if present and attachments nodes.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1031 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1032 actor_id = await self.getAPActorIdFromAccount(ap_account)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1033 inbox = await self.getAPInboxFromId(actor_id)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1034 for item in items:
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1035 if item.name == "item":
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1036 cached_item = await self.host.memory.storage.searchPubsubItems({
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1037 "profiles": [self.client.profile],
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1038 "services": [service],
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1039 "nodes": [node],
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1040 "names": [item["id"]]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1041 })
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1042 is_new = not bool(cached_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1043 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1044 # event item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1045 event_data = self._events.event_elt_2_event_data(item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1046 try:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1047 author_jid = jid.JID(item["publisher"]).userhostJID()
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1048 except (KeyError, RuntimeWarning):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1049 root_elt = item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1050 while root_elt.parent is not None:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1051 root_elt = root_elt.parent
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1052 author_jid = jid.JID(root_elt["from"]).userhostJID()
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1053 if subscribe_extra_nodes and not self.isVirtualJID(author_jid):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1054 # we subscribe automatically to comment nodes if any
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1055 recipient_jid = self.getLocalJIDFromAccount(ap_account)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1056 recipient_client = self.client.getVirtualClient(recipient_jid)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1057 comments_data = event_data.get("comments")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1058 if comments_data:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1059 comment_service = jid.JID(comments_data["jid"])
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1060 comment_node = comments_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1061 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1062 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1063 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1064 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1065 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1066 recipient_client, service, node, event_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1067 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1068 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1069 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1070 f"no attachment node found for item {event_data['id']!r} "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1071 f"on {node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1072 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1073 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1074 event_data, author_jid, is_new=is_new
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1075 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1076 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1077 # blog item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1078 mb_data = await self._m.item2mbdata(client, item, service, node)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1079 author_jid = jid.JID(mb_data["author_jid"])
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1080 if subscribe_extra_nodes and not self.isVirtualJID(author_jid):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1081 # we subscribe automatically to comment nodes if any
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1082 recipient_jid = self.getLocalJIDFromAccount(ap_account)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1083 recipient_client = self.client.getVirtualClient(recipient_jid)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1084 for comment_data in mb_data.get("comments", []):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1085 comment_service = jid.JID(comment_data["service"])
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1086 if self.isVirtualJID(comment_service):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1087 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1088 f"ignoring virtual comment service: {comment_data}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1089 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1090 continue
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1091 comment_node = comment_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1092 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1093 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1094 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1095 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1096 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1097 recipient_client, service, node, mb_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1098 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1099 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1100 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1101 f"no attachment node found for item {mb_data['id']!r} on "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1102 f"{node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1103 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1104 ap_item = await self.mb_data_2_ap_item(client, mb_data, is_new=is_new)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1105
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	1106 url_actor = ap_item["actor"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1107 elif item.name == "retract":
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	1108 url_actor, ap_item = await self.apDeleteItem(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	1109 client.jid, node, item["id"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1110 )
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1111 else:
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1112 raise exceptions.InternalError(f"unexpected element: {item.toXml()}")
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1113 await self.signAndPost(inbox, url_actor, ap_item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1114
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1115 async def convertAndPostAttachments(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1116 self,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1117 client: SatXMPPEntity,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1118 ap_account: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1119 service: jid.JID,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1120 node: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1121 items: List[domish.Element],
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1122 publisher: Optional[jid.JID] = None
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1123 ) -> None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1124 """Convert XMPP item attachments to AP activities and post them to actor inbox
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1125
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1126 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1127 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1128 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1129 @param node: (virtual) node corresponding where the item has been published
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1130 subscribed, that is comment nodes if present and attachments nodes.
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1131 @param items: attachments items
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1132 @param publisher: publisher of the attachments item (it's NOT the PEP/Pubsub
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1133 service, it's the publisher of the item). To be filled only when the publisher
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1134 is known for sure, otherwise publisher will be determined either if
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1135 "publisher" attribute is set by pubsub service, or as a last resort, using
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1136 item's ID (which MUST be publisher bare JID according to pubsub-attachments
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1137 specification).
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1138 """
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1139 if len(items) != 1:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1140 log.warning(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1141 "we should get exactly one attachment item for an entity, got "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1142 f"{len(items)})"
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1143 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1144
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1145 actor_id = await self.getAPActorIdFromAccount(ap_account)
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1146 inbox = await self.getAPInboxFromId(actor_id)
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1147
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1148 item_elt = items[0]
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1149 item_id = item_elt["id"]
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1150
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1151 if publisher is None:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1152 item_pub_s = item_elt.getAttribute("publisher")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1153 publisher = jid.JID(item_pub_s) if item_pub_s else jid.JID(item_id)
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1154
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1155 if publisher.userhost() != item_id:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1156 log.warning(
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1157 "attachments item ID must be publisher's bare JID, ignoring: "
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1158 f"{item_elt.toXml()}"
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1159 )
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1160 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1161
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1162 if self.isVirtualJID(publisher):
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1163 log.debug(f"ignoring item coming from local virtual JID {publisher}")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1164 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1165
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1166 if publisher is not None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1167 item_elt["publisher"] = publisher.userhost()
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1168
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1169 item_service, item_node, item_id = self._pa.attachmentNode2Item(node)
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1170 item_account = await self.getAPAccountFromJidAndNode(item_service, item_node)
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1171 if self.isVirtualJID(item_service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1172 # it's a virtual JID mapping to an external AP actor, we can use the
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1173 # item_id directly
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1174 item_url = item_id
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1175 if not item_url.startswith("https:"):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1176 log.warning(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1177 "item ID of external AP actor is not an https link, ignoring: "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1178 f"{item_id!r}"
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1179 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1180 return
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1181 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1182 item_url = self.buildAPURL(TYPE_ITEM, item_account, item_id)
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1183
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1184 old_attachment_pubsub_items = await self.host.memory.storage.searchPubsubItems({
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1185 "profiles": [self.client.profile],
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1186 "services": [service],
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1187 "nodes": [node],
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1188 "names": [item_elt["id"]]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1189 })
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1190 if not old_attachment_pubsub_items:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1191 old_attachment = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1192 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1193 old_attachment_items = [i.data for i in old_attachment_pubsub_items]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1194 old_attachments = self._pa.items2attachmentData(client, old_attachment_items)
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1195 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1196 old_attachment = old_attachments[0]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1197 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1198 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1199 old_attachment = {}
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1200 publisher_account = await self.getAPAccountFromJidAndNode(
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1201 publisher,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1202 None
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1203 )
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1204 publisher_actor_id = self.buildAPURL(TYPE_ACTOR, publisher_account)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1205 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1206 attachments = self._pa.items2attachmentData(client, [item_elt])[0]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1207 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1208 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1209 attachments = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1210
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1211 # noticed
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1212 if "noticed" in attachments:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1213 if not "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1214 # new "noticed" attachment, we translate to "Like" activity
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1215 activity_id = self.buildAPURL("like", item_account, item_id)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1216 activity = self.createActivity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1217 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1218 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1219 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1220 activity["cc"] = [NS_AP_PUBLIC]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1221 await self.signAndPost(inbox, publisher_actor_id, activity)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1222 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1223 if "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1224 # "noticed" attachment has been removed, we undo the "Like" activity
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1225 activity_id = self.buildAPURL("like", item_account, item_id)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1226 activity = self.createActivity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1227 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1228 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1229 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1230 activity["cc"] = [NS_AP_PUBLIC]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1231 undo = self.createActivity("Undo", publisher_actor_id, activity)
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1232 await self.signAndPost(inbox, publisher_actor_id, undo)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1233
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1234 # reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1235 new_reactions = set(attachments.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1236 old_reactions = set(old_attachment.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1237 reactions_remove = old_reactions - new_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1238 reactions_add = new_reactions - old_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1239 for reactions, undo in ((reactions_remove, True), (reactions_add, False)):
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1240 for reaction in reactions:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1241 activity_id = self.buildAPURL(
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1242 "reaction", item_account, item_id, reaction.encode().hex()
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1243 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1244 reaction_activity = self.createActivity(
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1245 TYPE_REACTION, publisher_actor_id, item_url,
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1246 activity_id=activity_id
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1247 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1248 reaction_activity["content"] = reaction
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1249 reaction_activity["to"] = [ap_account]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1250 reaction_activity["cc"] = [NS_AP_PUBLIC]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1251 if undo:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1252 activy = self.createActivity(
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1253 "Undo", publisher_actor_id, reaction_activity
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1254 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1255 else:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1256 activy = reaction_activity
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1257 await self.signAndPost(inbox, publisher_actor_id, activy)
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1258
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1259 # RSVP
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1260 if "rsvp" in attachments:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1261 attending = attachments["rsvp"].get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1262 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1263 if attending != old_attending:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1264 activity_type = TYPE_JOIN if attending == "yes" else TYPE_LEAVE
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1265 activity_id = self.buildAPURL(activity_type.lower(), item_account, item_id)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1266 activity = self.createActivity(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1267 activity_type, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1268 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1269 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1270 activity["cc"] = [NS_AP_PUBLIC]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1271 await self.signAndPost(inbox, publisher_actor_id, activity)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1272 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1273 if "rsvp" in old_attachment:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1274 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1275 if old_attending == "yes":
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1276 activity_id = self.buildAPURL(TYPE_LEAVE.lower(), item_account, item_id)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1277 activity = self.createActivity(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1278 TYPE_LEAVE, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1279 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1280 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1281 activity["cc"] = [NS_AP_PUBLIC]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1282 await self.signAndPost(inbox, publisher_actor_id, activity)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1283
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1284 if service.user and self.isVirtualJID(service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1285 # the item is on a virtual service, we need to store it in cache
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1286 log.debug("storing attachments item in cache")
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1287 cached_node = await self.host.memory.storage.getPubsubNode(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1288 client, service, node, with_subscriptions=True, create=True
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1289 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1290 await self.host.memory.storage.cachePubsubItems(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1291 self.client,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1292 cached_node,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1293 [item_elt],
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1294 [attachments]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1295 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1296
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1297 async def signAndPost(self, url: str, actor_id: str, doc: dict) -> TReqResponse:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1298 """Sign a documentent and post it to AP server
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1299
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1300 @param url: AP server endpoint
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1301 @param actor_id: originating actor ID (URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1302 @param doc: document to send
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1303 """
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1304 if self.verbose:
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1305 __, actor_args = self.parseAPURL(actor_id)
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1306 actor_account = actor_args[0]
3881 2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1307 to_log = [
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1308 "",
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1309 f">>> {actor_account} is signing and posting to {url}:\n{pformat(doc)}"
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1310 ]
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1311
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1312 p_url = parse.urlparse(url)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1313 body = json.dumps(doc).encode()
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1314 digest_algo, digest_hash = self.getDigest(body)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1315 digest = f"{digest_algo}={digest_hash}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1316
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1317 headers = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1318 "(request-target)": f"post {p_url.path}",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1319 "Host": p_url.hostname,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1320 "Date": http.datetimeToString().decode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1321 "Digest": digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1322 }
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1323 headers["Content-Type"] = (
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1324 'application/activity+json'
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1325 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1326 headers, __ = self.getSignatureData(self.getKeyId(actor_id), headers)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1327
3881 2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1328 if self.verbose:
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1329 if self.verbose>=3:
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1330 h_to_log = "\n".join(f" {k}: {v}" for k,v in headers.items())
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1331 to_log.append(f" headers:\n{h_to_log}")
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1332 to_log.append("---")
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1333 log.info("\n".join(to_log))
2e4a0f6050bd component AP gateway: better verbose logging: Goffi <goffi@goffi.org> parents: 3880 diff changeset	1334
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1335 resp = await treq.post(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1336 url,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1337 body,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1338 headers=headers,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1339 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1340 if resp.code >= 300:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1341 text = await resp.text()
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	1342 log.warning(f"POST request to {url} failed [{resp.code}]: {text}")
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1343 elif self.verbose:
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	1344 log.info(f"==> response code: {resp.code}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1345 return resp
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1346
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1347 def _publishMessage(self, mess_data_s: str, service_s: str, profile: str):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1348 mess_data: dict = data_format.deserialise(mess_data_s) # type: ignore
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1349 service = jid.JID(service_s)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1350 client = self.host.getClient(profile)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1351 return defer.ensureDeferred(self.publishMessage(client, mess_data, service))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1352
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1353 @async_lru(maxsize=LRU_MAX_SIZE)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1354 async def getAPActorIdFromAccount(self, account: str) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1355 """Retrieve account ID from it's handle using WebFinger
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1356
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1357 Don't use this method to get local actor id from a local account derivated for
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1358 JID: in this case, the actor ID is retrieve with
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1359 ``self.buildAPURL(TYPE_ACTOR, ap_account)``
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1360
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1361 @param account: AP handle (user@domain.tld)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1362 @return: Actor ID (which is an URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1363 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1364 if account.count("@") != 1 or "/" in account:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1365 raise ValueError(f"Invalid account: {account!r}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1366 host = account.split("@")[1]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1367 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1368 finger_data = await treq.json_content(await treq.get(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1369 f"https://{host}/.well-known/webfinger?"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1370 f"resource=acct:{parse.quote_plus(account)}",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1371 ))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1372 except Exception as e:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1373 raise exceptions.DataError(f"Can't get webfinger data for {account!r}: {e}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1374 for link in finger_data.get("links", []):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1375 if (
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1376 link.get("type") == "application/activity+json"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1377 and link.get("rel") == "self"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1378 ):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1379 href = link.get("href", "").strip()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1380 if not href:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1381 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1382 f"Invalid webfinger data for {account:r}: missing href"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1383 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1384 break
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1385 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1386 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1387 f"No ActivityPub link found for {account!r}"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1388 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1389 return href
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1390
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1391 async def getAPActorDataFromAccount(self, account: str) -> dict:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1392 """Retrieve ActivityPub Actor data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1393
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1394 @param account: ActivityPub Actor identifier
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1395 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1396 href = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1397 return await self.apGet(href)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1398
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1399 async def getAPInboxFromId(self, actor_id: str, use_shared: bool = True) -> str:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1400 """Retrieve inbox of an actor_id
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1401
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1402 @param use_shared: if True, and a shared inbox exists, it will be used instead of
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1403 the user inbox
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1404 """
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1405 data = await self.getActorData(actor_id)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1406 if use_shared:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1407 try:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1408 return data["endpoints"]["sharedInbox"]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1409 except KeyError:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1410 pass
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1411 return data["inbox"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1412
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1413 @async_lru(maxsize=LRU_MAX_SIZE)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1414 async def getAPAccountFromId(self, actor_id: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1415 """Retrieve AP account from the ID URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1416
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1417 Works with external or local actor IDs.
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1418 @param actor_id: AP ID of the actor (URL to the actor data)
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1419 @return: AP handle
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1420 """
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1421 if self.isLocalURL(actor_id):
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1422 url_type, url_args = self.parseAPURL(actor_id)
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1423 if url_type != "actor" or not url_args:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1424 raise exceptions.DataError(
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1425 f"invalid local actor ID: {actor_id}"
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1426 )
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1427 account = url_args[0]
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1428 try:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1429 account_user, account_host = account.split('@')
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1430 except ValueError:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1431 raise exceptions.DataError(
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1432 f"invalid account from url: {actor_id}"
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1433 )
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1434 if not account_user or account_host != self.public_url:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1435 raise exceptions.DataError(
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1436 f"{account!r} is not a valid local account (from {actor_id})"
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1437 )
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1438 return account
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1439
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1440 url_parsed = parse.urlparse(actor_id)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1441 actor_data = await self.getActorData(actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1442 username = actor_data.get("preferredUsername")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1443 if not username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1444 raise exceptions.DataError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1445 'No "preferredUsername" field found, can\'t retrieve actor account'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1446 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1447 account = f"{username}@{url_parsed.hostname}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1448 # we try to retrieve the actor ID from the account to check it
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1449 found_id = await self.getAPActorIdFromAccount(account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1450 if found_id != actor_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1451 # cf. https://socialhub.activitypub.rocks/t/how-to-retrieve-user-server-tld-handle-from-actors-url/2196
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1452 msg = (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1453 f"Account ID found on WebFinger {found_id!r} doesn't match our actor ID "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1454 f"({actor_id!r}). This AP instance doesn't seems to use "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1455 '"preferredUsername" as we expect.'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1456 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1457 log.warning(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1458 raise exceptions.DataError(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1459 return account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1460
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1461 async def getAPItems(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1462 self,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1463 collection: dict,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1464 max_items: Optional[int] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1465 chronological_pagination: bool = True,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1466 after_id: Optional[str] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1467 start_index: Optional[int] = None,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1468 parser: Optional[Callable[[dict], Awaitable[domish.Element]]] = None,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1469 only_ids: bool = False,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1470 ) -> Tuple[List[domish.Element], rsm.RSMResponse]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1471 """Retrieve AP items and convert them to XMPP items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1472
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1473 @param account: AP account handle to get items from
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1474 @param max_items: maximum number of items to retrieve
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1475 retrieve all items by default
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1476 @param chronological_pagination: get pages in chronological order
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1477 AP use reversed chronological order for pagination, "first" page returns more
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1478 recent items. If "chronological_pagination" is True, "last" AP page will be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1479 retrieved first.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1480 @param after_id: if set, retrieve items starting from given ID
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1481 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1482 if ``after_id`` is not already in cache, we have to retrieve every page until
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1483 we find it.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1484 In most common cases, ``after_id`` should be in cache though (client usually
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1485 use known ID when in-order pagination is used).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1486 @param start_index: start retrieving items from the one with given index
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1487 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1488 all pages before the requested index will be retrieved to count items.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1489 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1490 if None, use default generic parser
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1491 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1492 Retrieving only item IDs avoid HTTP requests to retrieve items, it may be
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1493 sufficient in some use cases (e.g. when retrieving following/followers
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1494 collections)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1495 @return: XMPP Pubsub items and corresponding RSM Response
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1496 Items are always returned in chronological order in the result
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1497 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1498 if parser is None:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1499 parser = self.ap_item_2_mb_elt
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1500
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1501 rsm_resp: Dict[str, Union[bool, int]] = {}
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1502 try:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1503 count = collection["totalItems"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1504 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1505 log.warning(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1506 f'"totalItems" not found in collection {collection.get("id")}, '
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1507 "defaulting to 20"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1508 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1509 count = 20
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1510 else:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1511 log.info(f"{collection.get('id')} has {count} item(s)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1512
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1513 rsm_resp["count"] = count
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1514
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1515 if start_index is not None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1516 assert chronological_pagination and after_id is None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1517 if start_index >= count:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1518 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1519 elif start_index == 0:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1520 # this is the default behaviour
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1521 pass
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1522 elif start_index > 5000:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1523 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1524 "feature-not-implemented",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1525 text="Maximum limit for previous_index has been reached, this limit"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1526 "is set to avoid DoS"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1527 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1528 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1529 # we'll convert "start_index" to "after_id", thus we need the item just
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1530 # before "start_index"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1531 previous_index = start_index - 1
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1532 retrieved_items = 0
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1533 current_page = collection["last"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1534 while retrieved_items < count:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1535 page_data, items = await self.parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1536 current_page, parser, only_ids
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1537 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1538 if not items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1539 log.warning(f"found an empty AP page at {current_page}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1540 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1541 page_start_idx = retrieved_items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1542 retrieved_items += len(items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1543 if previous_index <= retrieved_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1544 after_id = items[previous_index - page_start_idx]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1545 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1546 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1547 current_page = page_data["prev"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1548 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1549 log.warning(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1550 f"missing previous page link at {current_page}: {page_data!r}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1551 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1552 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1553 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1554 "Error while retrieving previous page from AP service at "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1555 f"{current_page}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1556 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1557
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1558 init_page = "last" if chronological_pagination else "first"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1559 page = collection.get(init_page)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1560 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1561 raise exceptions.DataError(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1562 f"Initial page {init_page!r} not found for collection "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1563 f"{collection.get('id')})"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1564 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1565 items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1566 page_items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1567 retrieved_items = 0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1568 found_after_id = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1569
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1570 while retrieved_items < count:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1571 __, page_items = await self.parseAPPage(page, parser, only_ids)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1572 if not page_items:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1573 break
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1574 retrieved_items += len(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1575 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1576 # if we have an after_id, we ignore all items until the requested one is
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1577 # found
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1578 try:
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1579 limit_idx = [i["id"] for i in page_items].index(after_id)
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1580 except ValueError:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1581 # if "after_id" is not found, we don't add any item from this page
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1582 page_id = page.get("id") if isinstance(page, dict) else page
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1583 log.debug(f"{after_id!r} not found at {page_id}, skipping")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1584 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1585 found_after_id = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1586 if chronological_pagination:
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1587 start_index = retrieved_items - len(page_items) + limit_idx + 1
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1588 page_items = page_items[limit_idx+1:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1589 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1590 start_index = count - (retrieved_items - len(page_items) +
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1591 limit_idx + 1)
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1592 page_items = page_items[:limit_idx]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1593 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1594 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1595 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1596 if max_items is not None and len(items) >= max_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1597 if chronological_pagination:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1598 items = items[:max_items]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1599 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1600 items = items[-max_items:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1601 break
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1602 page = collection.get("prev" if chronological_pagination else "next")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1603 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1604 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1605
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1606 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1607 raise error.StanzaError("item-not-found")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1608
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1609 if items:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1610 if after_id is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1611 rsm_resp["index"] = 0 if chronological_pagination else count - len(items)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1612 if start_index is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1613 rsm_resp["index"] = start_index
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1614 elif after_id is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1615 log.warning("Can't determine index of first element")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1616 elif chronological_pagination:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1617 rsm_resp["index"] = 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1618 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1619 rsm_resp["index"] = count - len(items)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1620 rsm_resp.update({
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1621 "first": items[0]["id"],
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1622 "last": items[-1]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1623 })
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1624
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1625 return items, rsm.RSMResponse(**rsm_resp)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1626
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1627 async def ap_item_2_mb_data_and_elt(self, ap_item: dict) -> Tuple[dict, domish.Element]:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1628 """Convert AP item to parsed microblog data and corresponding item element"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1629 mb_data = await self.apItem2MBdata(ap_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1630 item_elt = await self._m.data2entry(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1631 self.client, mb_data, mb_data["id"], None, self._m.namespace
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1632 )
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1633 if "repeated" in mb_data["extra"]:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1634 item_elt["publisher"] = mb_data["extra"]["repeated"]["by"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1635 else:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1636 item_elt["publisher"] = mb_data["author_jid"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1637 return mb_data, item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1638
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1639 async def ap_item_2_mb_elt(self, ap_item: dict) -> domish.Element:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1640 """Convert AP item to XMPP item element"""
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1641 __, item_elt = await self.ap_item_2_mb_data_and_elt(ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1642 return item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1643
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1644 async def parseAPPage(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1645 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1646 page: Union[str, dict],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1647 parser: Callable[[dict], Awaitable[domish.Element]],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1648 only_ids: bool = False
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1649 ) -> Tuple[dict, List[domish.Element]]:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1650 """Convert AP objects from an AP page to XMPP items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1651
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1652 @param page: Can be either url linking and AP page, or the page data directly
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1653 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1654 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1655 @return: page data, pubsub items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1656 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1657 page_data = await self.apGetObject(page)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1658 if page_data is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1659 log.warning('No data found in collection')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1660 return {}, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1661 ap_items = await self.apGetList(page_data, "orderedItems", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1662 if ap_items is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1663 ap_items = await self.apGetList(page_data, "items", only_ids=only_ids)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1664 if not ap_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1665 log.warning(f'No item field found in collection: {page_data!r}')
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1666 return page_data, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1667 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1668 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1669 "Items are not ordered, this is not spec compliant"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1670 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1671 items = []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1672 # AP Collections are in antichronological order, but we expect chronological in
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1673 # Pubsub, thus we reverse it
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1674 for ap_item in reversed(ap_items):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1675 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1676 items.append(await parser(ap_item))
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1677 except (exceptions.DataError, NotImplementedError, error.StanzaError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1678 continue
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1679
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1680 return page_data, items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1681
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1682 async def getCommentsNodes(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1683 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1684 item_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1685 parent_id: Optional[str]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1686 ) -> Tuple[Optional[str], Optional[str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1687 """Get node where this item is and node to use for comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1688
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1689 if config option "comments_max_depth" is set, a common node will be used below the
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1690 given depth
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1691 @param item_id: ID of the reference item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1692 @param parent_id: ID of the parent item if any (the ID set in "inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1693 @return: a tuple with parent_node_id, comments_node_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1694 - parent_node_id is the ID of the node where reference item must be. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1695 returned when the root node (i.e. not a comments node) must be used.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1696 - comments_node_id: is the ID of the node to use for comments. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1697 returned when no comment node must be used (happens when we have reached
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1698 "comments_max_depth")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1699 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1700 if parent_id is None or not self.comments_max_depth:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1701 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1702 self._m.getCommentsNode(parent_id) if parent_id is not None else None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1703 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1704 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1705 parent_url = parent_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1706 parents = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1707 for __ in range(COMMENTS_MAX_PARENTS):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1708 parent_item = await self.apGet(parent_url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1709 parents.insert(0, parent_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1710 parent_url = parent_item.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1711 if parent_url is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1712 break
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1713 parent_limit = self.comments_max_depth-1
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1714 if len(parents) <= parent_limit:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1715 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1716 self._m.getCommentsNode(parents[-1]["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1717 self._m.getCommentsNode(item_id)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1718 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1719 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1720 last_level_item = parents[parent_limit]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1721 return (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1722 self._m.getCommentsNode(last_level_item["id"]),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1723 None
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1724 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1725
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1726 async def apItem2MBdata(self, ap_item: dict) -> dict:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1727 """Convert AP activity or object to microblog data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1728
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1729 @param ap_item: ActivityPub item to convert
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1730 Can be either an activity of an object
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1731 @return: AP Item's Object and microblog data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1732 @raise exceptions.DataError: something is invalid in the AP item
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	1733 @raise NotImplementedError: some AP data is not handled yet
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1734 @raise error.StanzaError: error while contacting the AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1735 """
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1736 is_activity = self.is_activity(ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1737 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1738 ap_object = await self.apGetObject(ap_item, "object")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1739 if not ap_object:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1740 log.warning(f'No "object" found in AP item {ap_item!r}')
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1741 raise exceptions.DataError
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1742 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1743 ap_object = ap_item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1744 item_id = ap_object.get("id")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1745 if not item_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1746 log.warning(f'No "id" found in AP item: {ap_object!r}')
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1747 raise exceptions.DataError
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1748 mb_data = {"id": item_id, "extra": {}}
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1749
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1750 # content
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1751 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1752 language, content_xhtml = ap_object["contentMap"].popitem()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1753 except (KeyError, AttributeError):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1754 try:
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1755 mb_data["content_xhtml"] = ap_object["content"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1756 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1757 log.warning(f"no content found:\n{ap_object!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1758 raise exceptions.DataError
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1759 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1760 mb_data["language"] = language
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1761 mb_data["content_xhtml"] = content_xhtml
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1762
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1763 mb_data["content"] = await self._t.convert(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1764 mb_data["content_xhtml"],
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1765 self._t.SYNTAX_XHTML,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1766 self._t.SYNTAX_TEXT,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1767 False,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1768 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1769
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1770 # author
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1771 if is_activity:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1772 authors = await self.apGetActors(ap_item, "actor")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1773 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1774 authors = await self.apGetActors(ap_object, "attributedTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1775 if len(authors) > 1:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1776 # we only keep first item as author
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1777 # TODO: handle multiple actors
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1778 log.warning("multiple actors are not managed")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1779
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1780 account = authors[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1781 author_jid = self.getLocalJIDFromAccount(account).full()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1782
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1783 mb_data["author"] = account.split("@", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1784 mb_data["author_jid"] = author_jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1785
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1786 # published/updated
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1787 for field in ("published", "updated"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1788 value = ap_object.get(field)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1789 if not value and field == "updated":
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1790 value = ap_object.get("published")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1791 if value:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1792 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1793 mb_data[field] = calendar.timegm(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1794 dateutil.parser.parse(str(value)).utctimetuple()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1795 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1796 except dateutil.parser.ParserError as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1797 log.warning(f"Can't parse {field!r} field: {e}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1798
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1799 # repeat
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1800 if "_repeated" in ap_item:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1801 mb_data["extra"]["repeated"] = ap_item["_repeated"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1802
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1803 # comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1804 in_reply_to = ap_object.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1805 __, comments_node = await self.getCommentsNodes(item_id, in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1806 if comments_node is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1807 comments_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1808 "service": author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1809 "node": comments_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1810 "uri": uri.buildXMPPUri(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1811 "pubsub",
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1812 path=author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1813 node=comments_node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1814 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1815 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1816 mb_data["comments"] = [comments_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1817
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1818 return mb_data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1819
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1820 async def getReplyToIdFromXMPPNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1821 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1822 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1823 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1824 parent_item: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1825 mb_data: dict
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1826 ) -> str:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1827 """Get URL to use for ``inReplyTo`` field in AP item.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1828
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1829 There is currently no way to know the parent service of a comment with XEP-0277.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1830 To work around that, we try to check if we have this item in the cache (we
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1831 should). If there is more that one item with this ID, we first try to find one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1832 with this author_jid. If nothing is found, we use ap_account to build `inReplyTo`.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1833
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1834 @param ap_account: AP account corresponding to the publication author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1835 @param parent_item: ID of the node where the publication this item is replying to
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1836 has been posted
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1837 @param mb_data: microblog data of the publication
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1838 @return: URL to use in ``inReplyTo`` field
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1839 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1840 # FIXME: propose a protoXEP to properly get parent item, node and service
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1841
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1842 found_items = await self.host.memory.storage.searchPubsubItems({
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1843 "profiles": [client.profile],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1844 "names": [parent_item]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1845 })
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1846 if not found_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1847 log.warning(f"parent item {parent_item!r} not found in cache")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1848 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1849 elif len(found_items) == 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1850 cached_node = found_items[0].node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1851 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1852 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1853 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1854 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1855 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1856 # we found several cached item with given ID, we check if there is one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1857 # corresponding to this author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1858 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1859 author = jid.JID(mb_data["author_jid"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1860 cached_item = next(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1861 i for i in found_items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1862 if jid.JID(i.data["publisher"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1863 == author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1864 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1865 except StopIteration:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1866 # no item corresponding to this author, we use ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1867 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1868 "Can't find a single cached item for parent item "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1869 f"{parent_item!r}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1870 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1871 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1872 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1873 cached_node = cached_item.node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1874 parent_ap_account = await self.getAPAccountFromJidAndNode(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1875 cached_node.service,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1876 cached_node.name
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1877 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1878
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1879 return self.buildAPURL(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1880 TYPE_ITEM, parent_ap_account, parent_item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1881 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1882
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1883 async def repeatedMB2APItem(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1884 self,
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1885 mb_data: dict
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1886 ) -> dict:
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1887 """Convert repeated blog item to suitable AP Announce activity
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1888
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1889 @param mb_data: microblog metadata of an item repeating an other blog post
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1890 @return: Announce activity linking to the repeated item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1891 """
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1892 repeated = mb_data["extra"]["repeated"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1893 repeater = jid.JID(repeated["by"])
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1894 repeater_account = await self.getAPAccountFromJidAndNode(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1895 repeater,
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1896 None
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1897 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1898 repeater_id = self.buildAPURL(TYPE_ACTOR, repeater_account)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1899 repeated_uri = repeated["uri"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1900
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1901 if not repeated_uri.startswith("xmpp:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1902 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1903 "Only xmpp: URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1904 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1905 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1906 raise NotImplementedError
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1907 parsed_url = uri.parseXMPPUri(repeated_uri)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1908 if parsed_url["type"] != "pubsub":
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1909 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1910 "Only pubsub URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1911 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1912 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1913 raise NotImplementedError
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1914 rep_service = jid.JID(parsed_url["path"])
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1915 rep_item = parsed_url["item"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1916 activity_id = self.buildAPURL("item", repeater.userhost(), mb_data["id"])
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1917
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1918 if self.isVirtualJID(rep_service):
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1919 # it's an AP actor linked through this gateway
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1920 # in this case we can simply use the item ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1921 if not rep_item.startswith("https:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1922 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1923 f"Was expecting an HTTPS url as item ID and got {rep_item!r}\n"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1924 f"{mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1925 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1926 announced_uri = rep_item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1927 repeated_account = self._e.unescape(rep_service.user)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1928 else:
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1929 # the repeated item is an XMPP publication, we build the corresponding ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1930 rep_node = parsed_url["node"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1931 repeated_account = await self.getAPAccountFromJidAndNode(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1932 rep_service, rep_node
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1933 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1934 announced_uri = self.buildAPURL("item", repeated_account, rep_item)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1935
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1936 announce = self.createActivity(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1937 "Announce", repeater_id, announced_uri, activity_id=activity_id
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1938 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1939 announce["to"] = [NS_AP_PUBLIC]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1940 announce["cc"] = [
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1941 self.buildAPURL(TYPE_FOLLOWERS, repeater_account),
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1942 await self.getAPActorIdFromAccount(repeated_account)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1943 ]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1944 return announce
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1945
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1946 async def mb_data_2_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1947 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1948 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1949 mb_data: dict,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1950 public: bool =True,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1951 is_new: bool = True,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1952 ) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1953 """Convert Libervia Microblog Data to ActivityPub item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1954
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1955 @param mb_data: microblog data (as used in plugin XEP-0277) to convert
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1956 If ``public`` is True, ``service`` and ``node`` keys must be set.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1957 If ``published`` is not set, current datetime will be used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1958 @param public: True if the message is not a private/direct one
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1959 if True, the AP Item will be marked as public, and AP followers of target AP
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1960 account (which retrieve from ``service``) will be put in ``cc``.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1961 ``inReplyTo`` will also be set if suitable
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1962 if False, no destinee will be set (i.e., no ``to`` or ``cc`` or public flag).
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1963 This is usually used for direct messages.
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1964 @param is_new: if True, the item is a new one (no instance has been found in
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1965 cache).
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1966 If True, a "Create" activity will be generated, otherwise an "Update" one will
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1967 be.
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1968 @return: Activity item
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1969 """
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1970 extra = mb_data.get("extra", {})
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1971 if "repeated" in extra:
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1972 return await self.repeatedMB2APItem(mb_data)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1973 if not mb_data.get("id"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1974 mb_data["id"] = shortuuid.uuid()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1975 if not mb_data.get("author_jid"):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1976 mb_data["author_jid"] = client.jid.userhost()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1977 ap_account = await self.getAPAccountFromJidAndNode(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1978 jid.JID(mb_data["author_jid"]),
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1979 None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1980 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1981 url_actor = self.buildAPURL(TYPE_ACTOR, ap_account)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1982 url_item = self.buildAPURL(TYPE_ITEM, ap_account, mb_data["id"])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1983 ap_object = {
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1984 "id": url_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1985 "type": "Note",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1986 "published": utils.xmpp_date(mb_data.get("published")),
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1987 "attributedTo": url_actor,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1988 "content": mb_data.get("content_xhtml") or mb_data["content"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1989 }
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1990
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1991 language = mb_data.get("language")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1992 if language:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1993 ap_object["contentMap"] = {language: ap_object["content"]}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1994
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1995 if public:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1996 ap_object["to"] = [NS_AP_PUBLIC]
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1997 if self.auto_mentions:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1998 for m in RE_MENTION.finditer(ap_object["content"]):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1999 mention = m.group()
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2000 mentioned = mention[1:]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2001 __, m_host = mentioned.split("@", 1)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2002 if m_host in (self.public_url, self.client.jid.host):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2003 # we ignore mention of local users, they should be sent as XMPP
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2004 # references
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2005 continue
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2006 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2007 mentioned_id = await self.getAPActorIdFromAccount(mentioned)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2008 except Exception as e:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2009 log.warning(f"Can't add mention to {mentioned!r}: {e}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2010 else:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2011 ap_object["to"].append(mentioned_id)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2012 ap_object.setdefault("tag", []).append({
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2013 "type": TYPE_MENTION,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2014 "href": mentioned_id,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2015 "name": mention,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2016 })
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2017 try:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2018 node = mb_data["node"]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2019 service = jid.JID(mb_data["service"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2020 except KeyError:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2021 # node and service must always be specified when this method is used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2022 raise exceptions.InternalError(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2023 "node or service is missing in mb_data"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2024 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2025 target_ap_account = await self.getAPAccountFromJidAndNode(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2026 service, node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2027 )
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	2028 if self.isVirtualJID(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2029 # service is a proxy JID for AP account
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2030 actor_data = await self.getAPActorDataFromAccount(target_ap_account)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2031 followers = actor_data.get("followers")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2032 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2033 # service is a real XMPP entity
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2034 followers = self.buildAPURL(TYPE_FOLLOWERS, target_ap_account)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2035 if followers:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2036 ap_object["cc"] = [followers]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2037 if self._m.isCommentNode(node):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2038 parent_item = self._m.getParentItem(node)
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	2039 if self.isVirtualJID(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2040 # the publication is on a virtual node (i.e. an XMPP node managed by
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2041 # this gateway and linking to an ActivityPub actor)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2042 ap_object["inReplyTo"] = parent_item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2043 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2044 # the publication is from a followed real XMPP node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2045 ap_object["inReplyTo"] = await self.getReplyToIdFromXMPPNode(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2046 client,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2047 ap_account,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2048 parent_item,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2049 mb_data
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2050 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2051
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	2052 return self.createActivity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2053 "Create" if is_new else "Update", url_actor, ap_object, activity_id=url_item
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	2054 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2055
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2056 async def publishMessage(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2057 self,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2058 client: SatXMPPEntity,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2059 mess_data: dict,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2060 service: jid.JID
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2061 ) -> None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2062 """Send an AP message
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2063
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2064 .. note::
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2065
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2066 This is a temporary method used for development only
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2067
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2068 @param mess_data: message data. Following keys must be set:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2069
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2070 ``node``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2071 identifier of message which is being replied (this will
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2072 correspond to pubsub node in the future)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2073
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2074 ``content_xhtml`` or ``content``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2075 message body (respectively in XHTML or plain text)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2076
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2077 @param service: JID corresponding to the AP actor.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2078 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2079 if not service.user:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2080 raise ValueError("service must have a local part")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2081 account = self._e.unescape(service.user)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2082 ap_actor_data = await self.getAPActorDataFromAccount(account)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2083
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2084 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2085 inbox_url = ap_actor_data["endpoints"]["sharedInbox"]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2086 except KeyError:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2087 raise exceptions.DataError("Can't get ActivityPub actor inbox")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2088
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2089 item_data = await self.mb_data_2_ap_item(client, mess_data)
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	2090 url_actor = item_data["actor"]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2091 resp = await self.signAndPost(inbox_url, url_actor, item_data)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2092
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2093 async def apDeleteItem(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2094 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2095 jid_: jid.JID,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2096 node: Optional[str],
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2097 item_id: str,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2098 public: bool = True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2099 ) -> Tuple[str, Dict[str, Any]]:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2100 """Build activity to delete an AP item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2101
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2102 @param jid_: JID of the entity deleting an item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2103 @param node: node where the item is deleted
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2104 None if it's microblog or a message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2105 @param item_id: ID of the item to delete
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2106 it's the Pubsub ID or message's origin ID
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2107 @param public: if True, the activity will be addressed to public namespace
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2108 @return: actor_id of the entity deleting the item, activity to send
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2109 """
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2110 if node is None:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2111 node = self._m.namespace
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2112
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2113 author_account = await self.getAPAccountFromJidAndNode(jid_, node)
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2114 author_actor_id = self.buildAPURL(TYPE_ACTOR, author_account)
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2115
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2116 items = await self.host.memory.storage.searchPubsubItems({
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2117 "profiles": [self.client.profile],
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2118 "services": [jid_],
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2119 "names": [item_id]
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2120 })
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2121 if not items:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2122 log.warning(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2123 f"Deleting an unknown item at service {jid_}, node {node} and id "
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2124 f"{item_id}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2125 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2126 else:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2127 try:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2128 mb_data = await self._m.item2mbdata(self.client, items[0].data, jid_, node)
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2129 if "repeated" in mb_data["extra"]:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2130 # we are deleting a repeated item, we must translate this to an
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2131 # "Undo" of the "Announce" activity instead of a "Delete" one
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2132 announce = await self.repeatedMB2APItem(mb_data)
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2133 undo = self.createActivity("Undo", author_actor_id, announce)
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2134 return author_actor_id, undo
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2135 except Exception as e:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2136 log.debug(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2137 f"Can't parse item, maybe it's not a blog item: {e}\n"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2138 f"{items[0].toXml()}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2139 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2140
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2141 url_item = self.buildAPURL(TYPE_ITEM, author_account, item_id)
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2142 ap_item = self.createActivity(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2143 "Delete",
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2144 author_actor_id,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2145 {
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2146 "id": url_item,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2147 "type": TYPE_TOMBSTONE
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2148 }
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2149 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2150 if public:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2151 ap_item["to"] = [NS_AP_PUBLIC]
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2152 return author_actor_id, ap_item
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2153
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2154 def _messageReceivedTrigger(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2155 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2156 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2157 message_elt: domish.Element,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2158 post_treat: defer.Deferred
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2159 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2160 """add the gateway workflow on post treatment"""
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2161 if not self.client:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2162 log.warning(f"no client set, ignoring message: {message_elt.toXml()}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2163 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2164 post_treat.addCallback(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2165 lambda mess_data: defer.ensureDeferred(self.onMessage(client, mess_data))
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2166 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2167 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2168
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2169 async def onMessage(self, client: SatXMPPEntity, mess_data: dict) -> dict:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2170 """Called once message has been parsed
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2171
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2172 this method handle the conversion to AP items and posting
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2173 """
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2174 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2175 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2176 if mess_data["type"] not in ("chat", "normal"):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2177 log.warning(f"ignoring message with unexpected type: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2178 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2179 if not self.isLocal(mess_data["from"]):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2180 log.warning(f"ignoring non local message: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2181 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2182 if not mess_data["to"].user:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2183 log.warning(
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2184 f"ignoring message addressed to gateway itself: {mess_data}"
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2185 )
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2186 return mess_data
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2187
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2188 actor_account = self._e.unescape(mess_data["to"].user)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2189 actor_id = await self.getAPActorIdFromAccount(actor_account)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	2190 inbox = await self.getAPInboxFromId(actor_id, use_shared=False)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2191
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2192 try:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2193 language, message = next(iter(mess_data["message"].items()))
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2194 except (KeyError, StopIteration):
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2195 log.warning(f"ignoring empty message: {mess_data}")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2196 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2197
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2198 mb_data = {
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2199 "content": message,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2200 }
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2201 if language:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2202 mb_data["language"] = language
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2203 origin_id = mess_data["extra"].get("origin_id")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2204 if origin_id:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2205 # we need to use origin ID when present to be able to retract the message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2206 mb_data["id"] = origin_id
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2207 client = self.client.getVirtualClient(mess_data["from"])
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2208 ap_item = await self.mb_data_2_ap_item(client, mb_data, public=False)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2209 ap_item["object"]["to"] = ap_item["to"] = [actor_id]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2210 await self.signAndPost(inbox, ap_item["actor"], ap_item)
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2211 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2212
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2213 async def _onMessageRetract(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2214 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2215 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2216 message_elt: domish.Element,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2217 retract_elt: domish.Element,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2218 fastened_elts
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2219 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2220 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2221 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2222 from_jid = jid.JID(message_elt["from"])
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2223 if not self.isLocal(from_jid):
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2224 log.debug(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2225 f"ignoring retract request from non local jid {from_jid}"
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2226 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2227 return False
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2228 to_jid = jid.JID(message_elt["to"])
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2229 if (to_jid.host != self.client.jid.full() or not to_jid.user):
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2230 # to_jid should be a virtual JID from this gateway
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2231 raise exceptions.InternalError(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2232 f"Invalid destinee's JID: {to_jid.full()}"
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2233 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2234 ap_account = self._e.unescape(to_jid.user)
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2235 actor_id = await self.getAPActorIdFromAccount(ap_account)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	2236 inbox = await self.getAPInboxFromId(actor_id, use_shared=False)
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2237 url_actor, ap_item = await self.apDeleteItem(
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2238 from_jid.userhostJID(), None, fastened_elts.id, public=False
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2239 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2240 resp = await self.signAndPost(inbox, url_actor, ap_item)
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2241 return False
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2242
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2243 async def _onReferenceReceived(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2244 self,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2245 client: SatXMPPEntity,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2246 message_elt: domish.Element,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2247 reference_data: Dict[str, Union[str, int]]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2248 ) -> bool:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2249 parsed_uri: dict = reference_data.get("parsed_uri")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2250 if not parsed_uri:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2251 log.warning(f"no parsed URI available in reference {reference_data}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2252 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2253
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2254 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2255 mentioned = jid.JID(parsed_uri["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2256 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2257 log.warning(f"invalid target: {reference_data['uri']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2258 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2259
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2260 if mentioned.host != self.client.jid.full() or not mentioned.user:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2261 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2262 f"ignoring mentioned user {mentioned}, it's not a JID mapping an AP "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2263 "account"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2264 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2265 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2266
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2267 ap_account = self._e.unescape(mentioned.user)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2268 actor_id = await self.getAPActorIdFromAccount(ap_account)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2269
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2270 parsed_anchor: dict = reference_data.get("parsed_anchor")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2271 if not parsed_anchor:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2272 log.warning(f"no XMPP anchor, ignoring reference {reference_data!r}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2273 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2274
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2275 if parsed_anchor["type"] != "pubsub":
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2276 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2277 f"ignoring reference with non pubsub anchor, this is not supported: "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2278 "{reference_data!r}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2279 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2280 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2281
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2282 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2283 pubsub_service = jid.JID(parsed_anchor["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2284 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2285 log.warning(f"invalid anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2286 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2287 pubsub_node = parsed_anchor.get("node")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2288 if not pubsub_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2289 log.warning(f"missing pubsub node in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2290 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2291 pubsub_item = parsed_anchor.get("item")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2292 if not pubsub_item:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2293 log.warning(f"missing pubsub item in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2294 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2295
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2296 cached_node = await self.host.memory.storage.getPubsubNode(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2297 client, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2298 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2299 if not cached_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2300 log.warning(f"Anchored node not found in cache: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2301 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2302
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2303 cached_items, __ = await self.host.memory.storage.getItems(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2304 cached_node, item_ids=[pubsub_item]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2305 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2306 if not cached_items:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2307 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2308 f"Anchored pubsub item not found in cache: {reference_data['anchor']}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2309 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2310 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2311
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2312 cached_item = cached_items[0]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2313
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2314 mb_data = await self._m.item2mbdata(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2315 client, cached_item.data, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2316 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2317 ap_item = await self.mb_data_2_ap_item(client, mb_data)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2318 ap_object = ap_item["object"]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2319 ap_object["to"] = [actor_id]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2320 ap_object.setdefault("tag", []).append({
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2321 "type": TYPE_MENTION,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2322 "href": actor_id,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2323 "name": ap_account,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2324 })
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2325
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	2326 inbox = await self.getAPInboxFromId(actor_id, use_shared=False)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2327
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2328 resp = await self.signAndPost(inbox, ap_item["actor"], ap_item)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2329
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2330 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2331
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2332 async def newReplyToXMPPItem(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2333 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2334 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2335 ap_item: dict,
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2336 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2337 mentions: List[Dict[str, str]],
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2338 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2339 """We got an AP item which is a reply to an XMPP item"""
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2340 in_reply_to = ap_item["inReplyTo"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2341 url_type, url_args = self.parseAPURL(in_reply_to)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2342 if url_type != "item":
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2343 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2344 "Ignoring AP item replying to an XMPP item with an unexpected URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2345 f"type({url_type!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2346 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2347 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2348 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2349 parent_item_account, parent_item_id = url_args[0].split("/", 1)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2350 except (IndexError, ValueError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2351 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2352 "Ignoring AP item replying to an XMPP item with invalid inReplyTo URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2353 f"({in_reply_to!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2354 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2355 return
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2356 parent_item_service, parent_item_node = await self.getJIDAndNode(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2357 parent_item_account
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2358 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2359 if parent_item_node is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2360 parent_item_node = self._m.namespace
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2361 items, __ = await self._p.getItems(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2362 client, parent_item_service, parent_item_node, item_ids=[parent_item_id]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2363 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2364 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2365 parent_item_elt = items[0]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2366 except IndexError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2367 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2368 f"Can't find parent item at {parent_item_service} (node "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2369 f"{parent_item_node!r})\n{pformat(ap_item)}")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2370 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2371 parent_item_parsed = await self._m.item2mbdata(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2372 client, parent_item_elt, parent_item_service, parent_item_node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2373 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2374 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2375 comment_service = jid.JID(parent_item_parsed["comments"][0]["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2376 comment_node = parent_item_parsed["comments"][0]["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2377 except (KeyError, IndexError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2378 # we don't have a comment node set for this item
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2379 from sat.tools.xml_tools import ppElt
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2380 log.info(f"{ppElt(parent_item_elt.toXml())}")
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2381 raise NotImplementedError()
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2382 else:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2383 __, item_elt = await self.ap_item_2_mb_data_and_elt(ap_item)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2384 await self._p.publish(client, comment_service, comment_node, [item_elt])
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2385 await self.notifyMentions(
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2386 targets, mentions, comment_service, comment_node, item_elt["id"]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2387 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2388
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2389 def getAPItemTargets(
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2390 self,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2391 item: Dict[str, Any]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2392 ) -> Tuple[bool, Dict[str, Set[str]], List[Dict[str, str]]]:
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2393 """Retrieve targets of an AP item, and indicate if it's a public one
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2394
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2395 @param item: AP object payload
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2396 @return: Are returned:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2397 - is_public flag, indicating if the item is world-readable
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2398 - a dict mapping target type to targets
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2399 """
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2400 targets: Dict[str, Set[str]] = {}
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2401 is_public = False
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2402 # TODO: handle "audience"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2403 for key in ("to", "bto", "cc", "bcc"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2404 values = item.get(key)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2405 if not values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2406 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2407 if isinstance(values, str):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2408 values = [values]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2409 for value in values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2410 if value in PUBLIC_TUPLE:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2411 is_public = True
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2412 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2413 if not value:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2414 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2415 if not self.isLocalURL(value):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2416 continue
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2417 target_type = self.parseAPURL(value)[0]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2418 if target_type != TYPE_ACTOR:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2419 log.debug(f"ignoring non actor type as a target: {href}")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2420 else:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2421 targets.setdefault(target_type, set()).add(value)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2422
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2423 mentions = []
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2424 tags = item.get("tag")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2425 if tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2426 for tag in tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2427 if tag.get("type") != TYPE_MENTION:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2428 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2429 href = tag.get("href")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2430 if not href:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2431 log.warning('Missing "href" field from mention object: {tag!r}')
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2432 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2433 if not self.isLocalURL(href):
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2434 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2435 uri_type = self.parseAPURL(href)[0]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2436 if uri_type != TYPE_ACTOR:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2437 log.debug(f"ignoring non actor URI as a target: {href}")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2438 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2439 mention = {"uri": href}
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2440 mentions.append(mention)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2441 name = tag.get("name")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2442 if name:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2443 mention["content"] = name
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2444
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2445 return is_public, targets, mentions
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2446
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2447 async def newAPItem(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2448 self,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2449 client: SatXMPPEntity,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2450 destinee: Optional[jid.JID],
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2451 node: str,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2452 item: dict,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2453 ) -> None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2454 """Analyse, cache and send notification for received AP item
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2455
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2456 @param destinee: jid of the destinee,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2457 @param node: XMPP pubsub node
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2458 @param item: AP object payload
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2459 """
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2460 is_public, targets, mentions = self.getAPItemTargets(item)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2461 if not is_public and targets.keys() == {TYPE_ACTOR}:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2462 # this is a direct message
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2463 await self.handleMessageAPItem(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2464 client, targets, mentions, destinee, item
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2465 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2466 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2467 await self.handlePubsubAPItem(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2468 client, targets, mentions, destinee, node, item, is_public
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2469 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2470
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2471 async def handleMessageAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2472 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2473 client: SatXMPPEntity,
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2474 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2475 mentions: List[Dict[str, str]],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2476 destinee: Optional[jid.JID],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2477 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2478 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2479 """Parse and deliver direct AP items translating to XMPP messages
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2480
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2481 @param targets: actors where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2482 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2483 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2484 """
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2485 targets_jids = {
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2486 await self.getJIDFromId(t)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2487 for t_set in targets.values()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2488 for t in t_set
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2489 }
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2490 if destinee is not None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2491 targets_jids.add(destinee)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2492 mb_data = await self.apItem2MBdata(item)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2493 defer_l = []
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2494 for target_jid in targets_jids:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2495 defer_l.append(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2496 client.sendMessage(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2497 target_jid,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2498 {'': mb_data.get("content", "")},
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2499 mb_data.get("title"),
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2500 extra={"origin_id": mb_data["id"]}
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2501 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2502 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2503 await defer.DeferredList(defer_l)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2504
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2505 async def notifyMentions(
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2506 self,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2507 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2508 mentions: List[Dict[str, str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2509 service: jid.JID,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2510 node: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2511 item_id: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2512 ) -> None:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2513 """Send mention notifications to recipients and mentioned entities
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2514
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2515 XEP-0372 (References) is used.
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2516
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2517 Mentions are also sent to recipients as they are primary audience (see
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2518 https://www.w3.org/TR/activitystreams-vocabulary/#microsyntaxes).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2519
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2520 """
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2521 anchor = uri.buildXMPPUri("pubsub", path=service.full(), node=node, item=item_id)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2522 seen = set()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2523 # we start with explicit mentions because mentions' content will be used in the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2524 # future to fill "begin" and "end" reference attributes (we can't do it at the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2525 # moment as there is no way to specify the XML element to use in the blog item).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2526 for mention in mentions:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2527 mentioned_jid = await self.getJIDFromId(mention["uri"])
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2528 self._refs.sendReference(
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2529 self.client,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2530 to_jid=mentioned_jid,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2531 anchor=anchor
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2532 )
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2533 seen.add(mentioned_jid)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2534
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2535 remaining = {
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2536 await self.getJIDFromId(t)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2537 for t_set in targets.values()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2538 for t in t_set
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2539 } - seen
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2540 for target in remaining:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2541 self._refs.sendReference(
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2542 self.client,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2543 to_jid=target,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2544 anchor=anchor
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2545 )
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2546
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2547 async def handlePubsubAPItem(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2548 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2549 client: SatXMPPEntity,
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2550 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2551 mentions: List[Dict[str, str]],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2552 destinee: Optional[jid.JID],
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2553 node: str,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2554 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2555 public: bool
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2556 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2557 """Analyse, cache and deliver AP items translating to Pubsub
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2558
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2559 @param targets: actors/collections where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2560 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2561 @param node: XMPP pubsub node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2562 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2563 @param public: True if the item is public
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2564 """
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2565 # XXX: "public" is not used for now
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2566 service = client.jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2567 in_reply_to = item.get("inReplyTo")
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2568
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2569 if in_reply_to and isinstance(in_reply_to, list):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2570 in_reply_to = in_reply_to[0]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2571 if in_reply_to and isinstance(in_reply_to, str):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2572 if self.isLocalURL(in_reply_to):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2573 # this is a reply to an XMPP item
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2574 await self.newReplyToXMPPItem(client, item, targets, mentions)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2575 return
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2576
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2577 # this item is a reply to an AP item, we use or create a corresponding node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2578 # for comments
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2579 parent_node, __ = await self.getCommentsNodes(item["id"], in_reply_to)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2580 node = parent_node or node
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2581 cached_node = await self.host.memory.storage.getPubsubNode(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	2582 client, service, node, with_subscriptions=True, create=True,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	2583 create_kwargs={"subscribed": True}
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2584 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2585 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2586 # it is a root item (i.e. not a reply to an other item)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2587 create = node == self._events.namespace
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2588 cached_node = await self.host.memory.storage.getPubsubNode(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2589 client, service, node, with_subscriptions=True, create=create
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2590 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2591 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2592 log.warning(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2593 f"Received item in unknown node {node!r} at {service}. This may be "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2594 f"due to a cache purge. We synchronise the node\n{item}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2595
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2596 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2597 return
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2598 if item.get("type") == TYPE_EVENT:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2599 data, item_elt = await self.ap_events.ap_item_2_event_data_and_elt(item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2600 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2601 data, item_elt = await self.ap_item_2_mb_data_and_elt(item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2602 await self.host.memory.storage.cachePubsubItems(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2603 client,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2604 cached_node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2605 [item_elt],
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2606 [data]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2607 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2608
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2609 for subscription in cached_node.subscriptions:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2610 if subscription.state != SubscriptionState.SUBSCRIBED:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2611 continue
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2612 self.pubsub_service.notifyPublish(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2613 service,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2614 node,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2615 [(subscription.subscriber, None, [item_elt])]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2616 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2617
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2618 await self.notifyMentions(targets, mentions, service, node, item_elt["id"])
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2619
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2620 async def newAPDeleteItem(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2621 self,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2622 client: SatXMPPEntity,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2623 destinee: Optional[jid.JID],
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2624 node: str,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2625 item: dict,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2626 ) -> None:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2627 """Analyse, cache and send notification for received AP item
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2628
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2629 @param destinee: jid of the destinee,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2630 @param node: XMPP pubsub node
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2631 @param activity: parent AP activity
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2632 @param item: AP object payload
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	2633 only the "id" field is used
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2634 """
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2635 item_id = item.get("id")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2636 if not item_id:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2637 raise exceptions.DataError('"id" attribute is missing in item')
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2638 if not item_id.startswith("http"):
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2639 raise exceptions.DataError(f"invalid id: {item_id!r}")
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2640 if self.isLocalURL(item_id):
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2641 raise ValueError("Local IDs should not be used")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2642
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2643 # we have no way to know if a deleted item is a direct one (thus a message) or one
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2644 # converted to pubsub. We check if the id is in message history to decide what to
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2645 # do.
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2646 history = await self.host.memory.storage.get(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2647 client,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2648 History,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2649 History.origin_id,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2650 item_id,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2651 (History.messages, History.subjects)
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2652 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2653
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2654 if history is not None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2655 # it's a direct message
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2656 if history.source_jid != client.jid:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2657 log.warning(
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	2658 f"retraction received from an entity ''{client.jid}) which is "
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2659 f"not the original sender of the message ({history.source_jid}), "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2660 "hack attemps?"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2661 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2662 raise exceptions.PermissionError("forbidden")
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2663
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2664 await self._r.retractByHistory(client, history)
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2665 else:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2666 # no history in cache with this ID, it's probably a pubsub item
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2667 cached_node = await self.host.memory.storage.getPubsubNode(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2668 client, client.jid, node, with_subscriptions=True
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2669 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2670 if cached_node is None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2671 log.warning(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2672 f"Received an item retract for node {node!r} at {client.jid} "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2673 "which is not cached"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2674 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2675 raise exceptions.NotFound
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2676 await self.host.memory.storage.deletePubsubItems(cached_node, [item_id])
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2677 # notifyRetract is expecting domish.Element instances
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2678 item_elt = domish.Element((None, "item"))
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2679 item_elt["id"] = item_id
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2680 for subscription in cached_node.subscriptions:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2681 if subscription.state != SubscriptionState.SUBSCRIBED:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2682 continue
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2683 self.pubsub_service.notifyRetract(
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2684 client.jid,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2685 node,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2686 [(subscription.subscriber, None, [item_elt])]
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2687 )

Mercurial > libervia-backend

annotate sat/plugins/plugin_comp_ap_gateway/__init__.py @ 3934:e345d93fb6e5

annotate sat/plugins/plugin_comp_ap_gateway/init.py @ 3934:e345d93fb6e5