libervia-backend: libervia/backend/plugins/plugin_comp_ap_gateway/__init_

annotate libervia/backend/plugins/plugin_comp_ap_gateway/init.py @ 4314:6a70fcd93a7a

plugin XEP-0131: Stanza Headers and Internet Metadata implementation: - SHIM is now supported and put in `msg_data["extra"]["headers"]`. - `Keywords` are converted from and to list of string in `msg_data["extra"]["keywords"]` field (if present in headers on message sending, values are merged). - Python minimal version upgraded to 3.11 due to use of `StrEnum`. rel 451

author	Goffi <goffi@goffi.org>
date	Sat, 28 Sep 2024 15:56:04 +0200
parents	0f953ce5f0a8
children

rev	line source
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia ActivityPub Gateway
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 import base64
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	20 import calendar
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	21 import hashlib
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	22 import json
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	23 from pathlib import Path
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	24 from pprint import pformat
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	25 import re
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	26 from typing import (
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	27 Any,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	28 Awaitable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	29 Callable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	30 Dict,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	31 List,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	32 Optional,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	33 Set,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	34 Tuple,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	35 Union,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	36 overload,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	37 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	38 from urllib import parse
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	39
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	40 from cryptography.exceptions import InvalidSignature
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	41 from cryptography.hazmat.primitives import serialization
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 from cryptography.hazmat.primitives import hashes
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	43 from cryptography.hazmat.primitives.asymmetric import rsa
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	44 from cryptography.hazmat.primitives.asymmetric import padding
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	45 import dateutil
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	46 import shortuuid
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	47 import treq
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 from treq.response import _Response as TReqResponse
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	49 from twisted.internet import defer, reactor, threads
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	50 from twisted.web import http
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	51 from twisted.words.protocols.jabber import error, jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	52 from twisted.words.xish import domish
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	53 from wokkel import pubsub, rsm
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	54
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	55 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	56 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	57 from libervia.backend.core.core_types import SatXMPPEntity
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	58 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	59 from libervia.backend.core.log import getLogger
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	60 from libervia.backend.memory import persistent
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	61 from libervia.backend.memory.sqla_mapping import History, SubscriptionState
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	62 from libervia.backend.tools import utils
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	63 from libervia.backend.tools.common import data_format, date_utils, tls, uri
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	64 from libervia.backend.tools.common.async_utils import async_lru
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	65
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	66 from .ad_hoc import APAdHocService
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	67 from .events import APEvents
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	68 from .constants import (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	69 ACTIVITY_OBJECT_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	70 ACTIVITY_TARGET_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	71 ACTIVITY_TYPES,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	72 ACTIVITY_TYPES_LOWER,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	73 COMMENTS_MAX_PARENTS,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	74 CONF_SECTION,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	75 IMPORT_NAME,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	76 LRU_MAX_SIZE,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	77 MEDIA_TYPE_AP,
3846 cc13efdd8360 component AP gateway: return item when `item` URL is used: Goffi <goffi@goffi.org> parents: 3845 diff changeset	78 NS_AP,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	79 NS_AP_PUBLIC,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	80 PUBLIC_TUPLE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	81 TYPE_ACTOR,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	82 TYPE_EVENT,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	83 TYPE_FOLLOWERS,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	84 TYPE_ITEM,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	85 TYPE_LIKE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	86 TYPE_MENTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	87 TYPE_REACTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	88 TYPE_TOMBSTONE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	89 TYPE_JOIN,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	90 TYPE_LEAVE,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	91 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	92 from .http_server import HTTPServer
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	93 from .pubsub_service import APPubsubService
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	94 from .regex import RE_MENTION
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	95
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	96
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	97 log = getLogger(__name__)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	98
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	99 IMPORT_NAME = "ap-gateway"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	100
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	101 PLUGIN_INFO = {
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	102 C.PI_NAME: "ActivityPub Gateway component",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	103 C.PI_IMPORT_NAME: IMPORT_NAME,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	104 C.PI_MODES: [C.PLUG_MODE_COMPONENT],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	105 C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	106 C.PI_PROTOCOLS: [],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	107 C.PI_DEPENDENCIES: [
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	108 "XEP-0050",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	109 "XEP-0054",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	110 "XEP-0060",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	111 "XEP-0084",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	112 "XEP-0106",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	113 "XEP-0277",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	114 "XEP-0292",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	115 "XEP-0329",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	116 "XEP-0372",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	117 "XEP-0424",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	118 "XEP-0465",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	119 "XEP-0470",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	120 "XEP-0447",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	121 "XEP-0471",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	122 "PUBSUB_CACHE",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	123 "TEXT_SYNTAXES",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	124 "IDENTITY",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	125 ],
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	126 C.PI_RECOMMENDATIONS: [],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	127 C.PI_MAIN: "APGateway",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	128 C.PI_HANDLER: C.BOOL_TRUE,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	129 C.PI_DESCRIPTION: _(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 "Gateway for bidirectional communication between XMPP and ActivityPub."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	131 ),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	132 }
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	133
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	134 HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	135 RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	136 RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	137 RE_ALLOWED_UNQUOTED = re.compile(r"^[a-zA-Z0-9_-]+$")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	138
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	139
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	140 class MissingLocalPartError(Exception):
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	141 """JID local part is missing"""
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	142
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	143
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	144 class APGateway:
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	145 IMPORT_NAME = IMPORT_NAME
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	146 # show data send or received through HTTP, used for debugging
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	147 # 1: log POST objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	148 # 2: log POST and GET objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	149 # 3: log POST and GET objects with HTTP headers for GET requests
4025 44abce96ac6b component AP gateway: revert forgotten verbose mode Goffi <goffi@goffi.org> parents: 4023 diff changeset	150 verbose = 0
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	151
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	152 def __init__(self, host):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	153 self.host = host
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	154 self.initialised = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	155 self.client = None
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	156 self.http_sign_get = True
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	157 self._p = host.plugins["XEP-0060"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	158 self._a = host.plugins["XEP-0084"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	159 self._e = host.plugins["XEP-0106"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	160 self._m = host.plugins["XEP-0277"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	161 self._v = host.plugins["XEP-0292"]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	162 self._refs = host.plugins["XEP-0372"]
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	163 self._r = host.plugins["XEP-0424"]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	164 self._sfs = host.plugins["XEP-0447"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	165 self._pps = host.plugins["XEP-0465"]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	166 self._pa = host.plugins["XEP-0470"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	167 self._c = host.plugins["PUBSUB_CACHE"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	168 self._t = host.plugins["TEXT_SYNTAXES"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	169 self._i = host.plugins["IDENTITY"]
4027 26c3e1bc7fb7 plugin XEP-0471: renamed "events" plugin to XEP-0471 now that there is a XEP Goffi <goffi@goffi.org> parents: 4025 diff changeset	170 self._events = host.plugins["XEP-0471"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	171 self._p.add_managed_node(
3850 4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	172 "",
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	173 items_cb=self._items_received,
3850 4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	174 # we want to be sure that the callbacks are launched before pubsub cache's
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	175 # one, as we need to inspect items before they are actually removed from cache
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	176 # or updated
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	177 priority=1000,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	178 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	179 self.pubsub_service = APPubsubService(self)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	180 self.ad_hoc = APAdHocService(self)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	181 self.ap_events = APEvents(self)
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	182 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	183 "message_received", self, self._message_received_trigger, priority=-1000
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	184 )
6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	185 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	186 "XEP-0424_retract_received", self, self._on_message_retract
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	187 )
6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	188 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	189 "XEP-0372_ref_received", self, self._on_reference_received
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	190 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	191
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	192 host.bridge.add_method(
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	193 "ap_send",
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	194 ".plugin",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	195 in_sign="sss",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	196 out_sign="",
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	197 method=self._publish_message,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	198 async_=True,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	199 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	200
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	201 def get_handler(self, __):
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	202 return self.pubsub_service
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	203
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	204 async def init(self, client):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	205 if self.initialised:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	206 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	207
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 self.initialised = True
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 log.info(_("ActivityPub Gateway initialization"))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	210
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	211 # RSA keys
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	212 stored_data = await self.host.memory.storage.get_privates(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	213 IMPORT_NAME, ["rsa_key"], profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	214 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	215 private_key_pem = stored_data.get("rsa_key")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	216 if private_key_pem is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	217 self.private_key = await threads.deferToThread(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	218 rsa.generate_private_key,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	219 public_exponent=65537,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	220 key_size=4096,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	221 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	222 private_key_pem = self.private_key.private_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	223 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 format=serialization.PrivateFormat.PKCS8,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	225 encryption_algorithm=serialization.NoEncryption(),
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 ).decode()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	227 await self.host.memory.storage.set_private_value(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 IMPORT_NAME, "rsa_key", private_key_pem, profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	229 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	230 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	231 self.private_key = serialization.load_pem_private_key(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	232 private_key_pem.encode(),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	233 password=None,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	234 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	235 self.public_key = self.private_key.public_key()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	236 self.public_key_pem = self.public_key.public_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	237 encoding=serialization.Encoding.PEM,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	238 format=serialization.PublicFormat.SubjectPublicKeyInfo,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	239 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	240
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	241 # params
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	242 # URL and port
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	243 self.public_url = self.host.memory.config_get(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	244 CONF_SECTION, "public_url"
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	245 ) or self.host.memory.config_get(CONF_SECTION, "xmpp_domain")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 if self.public_url is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	247 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	248 '"public_url" not set in configuration, this is mandatory to have'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	249 "ActivityPub Gateway running. Please set this option it to public facing "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	250 f"url in {CONF_SECTION!r} configuration section."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	251 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	252 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	253 if parse.urlparse(self.public_url).scheme:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	254 log.error(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	255 'Scheme must not be specified in "public_url", please remove it from '
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	256 '"public_url" configuration option. ActivityPub Gateway won\'t be run.'
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	257 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	258 return
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	259 self.http_port = int(self.host.memory.config_get(CONF_SECTION, "http_port", 8123))
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	260 connection_type = self.host.memory.config_get(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	261 CONF_SECTION, "http_connection_type", "https"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	262 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	263 if connection_type not in ("http", "https"):
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	264 raise exceptions.ConfigError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	265 'bad ap-gateay http_connection_type, you must use one of "http" or '
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	266 '"https"'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	267 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	268 self.http_sign_get = C.bool(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	269 self.host.memory.config_get(CONF_SECTION, "http_sign_get", C.BOOL_TRUE)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	270 )
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	271 self.max_items = int(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	272 self.host.memory.config_get(CONF_SECTION, "new_node_max_items", 50)
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	273 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	274 self.comments_max_depth = int(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	275 self.host.memory.config_get(CONF_SECTION, "comments_max_depth", 0)
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	276 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	277 self.ap_path = self.host.memory.config_get(CONF_SECTION, "ap_path", "_ap")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	278 self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	279 # True (default) if we provide gateway only to entities/services from our server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	280 self.local_only = C.bool(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	281 self.host.memory.config_get(CONF_SECTION, "local_only", C.BOOL_TRUE)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	282 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	283 # if True (default), mention will be parsed in non-private content coming from
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	284 # XMPP. This is necessary as XEP-0372 are coming separately from item where the
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	285 # mention is done, which is hard to impossible to translate to ActivityPub (where
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	286 # mention specified inside the item directly). See documentation for details.
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	287 self.auto_mentions = C.bool(
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	288 self.host.memory.config_get(CONF_SECTION, "auto_mentions", C.BOOL_TRUE)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	289 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	290
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	291 html_redirect: Dict[str, Union[str, dict]] = self.host.memory.config_get(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	292 CONF_SECTION, "html_redirect_dict", {}
3977 6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	293 )
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	294 self.html_redirect: Dict[str, List[dict]] = {}
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	295 for url_type, target in html_redirect.items():
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	296 if isinstance(target, str):
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	297 target = {"url": target}
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	298 elif not isinstance(target, dict):
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	299 raise exceptions.ConfigError(
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	300 f"html_redirect target must be a URL or a dict, not {target!r}"
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	301 )
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	302 filters = target.setdefault("filters", {})
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	303 if "url" not in target:
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	304 log.warning(f"invalid HTML redirection, missing target URL: {target}")
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	305 continue
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	306 # a slash in the url_type is a syntactic shortcut to have a node filter
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	307 if "/" in url_type:
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	308 url_type, node_filter = url_type.split("/", 1)
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	309 filters["node"] = node_filter
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	310 self.html_redirect.setdefault(url_type, []).append(target)
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	311
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	312 # HTTP server launch
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	313 self.server = HTTPServer(self)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	314 if connection_type == "http":
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	315 reactor.listenTCP(self.http_port, self.server)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	316 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	317 options = tls.get_options_from_config(self.host.memory.config, CONF_SECTION)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	318 tls.tls_options_check(options)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	319 context_factory = tls.get_tls_context_factory(options)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	320 reactor.listenSSL(self.http_port, self.server, context_factory)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	321
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	322 async def profile_connecting(self, client):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	323 self.client = client
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	324 client.sendHistory = True
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	325 client._ap_storage = persistent.LazyPersistentBinaryDict(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	326 IMPORT_NAME, client.profile
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	327 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	328 await self.init(client)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	329
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	330 def profile_connected(self, client):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	331 self.ad_hoc.init(client)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	332
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	333 async def _items_received(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	334 self, client: SatXMPPEntity, itemsEvent: pubsub.ItemsEvent
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	335 ) -> None:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	336 """Callback called when pubsub items are received
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	337
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	338 if the items are adressed to a JID corresponding to an AP actor, they are
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	339 converted to AP items and sent to the corresponding AP server.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	340
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	341 If comments nodes are linked, they are automatically subscribed to get new items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	342 from there too.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	343 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	344 if client != self.client:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	345 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	346 # we need recipient as JID and not gateway own JID to be able to use methods such
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	347 # as "subscribe"
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	348 client = self.client.get_virtual_client(itemsEvent.sender)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	349 recipient = itemsEvent.recipient
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	350 if not recipient.user:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	351 log.debug("ignoring items event without local part specified")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	352 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	353
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	354 ap_account = self._e.unescape(recipient.user)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	355
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	356 if self._pa.is_attachment_node(itemsEvent.nodeIdentifier):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	357 await self.convert_and_post_attachments(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	358 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	359 ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	360 itemsEvent.sender,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	361 itemsEvent.nodeIdentifier,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	362 itemsEvent.items,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	363 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	364 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	365 await self.convert_and_post_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	366 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	367 ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	368 itemsEvent.sender,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	369 itemsEvent.nodeIdentifier,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	370 itemsEvent.items,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	371 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	372
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	373 async def get_virtual_client(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	374 self, requestor_actor_id: str, actor_id: str
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	375 ) -> SatXMPPEntity:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	376 """Get client for this component with a specified jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	377
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	378 This is needed to perform operations with the virtual JID corresponding to the AP
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	379 actor instead of the JID of the gateway itself.
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	380 @param requestor_actor_id: originating actor ID (URL)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	381 @param actor_id: ID of the actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	382 @return: virtual client
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	383 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	384 local_jid = await self.get_jid_from_id(requestor_actor_id, actor_id)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	385 return self.client.get_virtual_client(local_jid)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	386
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	387 def is_activity(self, data: dict) -> bool:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	388 """Return True if the data has an activity type"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	389 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	390 return (data.get("type") or "").lower() in ACTIVITY_TYPES_LOWER
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	391 except (KeyError, TypeError):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	392 return False
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	393
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	394 async def ap_get(self, url: str, requestor_actor_id: str) -> dict:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	395 """Retrieve AP JSON from given URL with HTTP Signature
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	396
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	397 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	398 @param requestor_actor_id: originating actor ID (URL)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	399 @raise error.StanzaError: "service-unavailable" is sent when something went wrong
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	400 with AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	401 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	402 if self.http_sign_get:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	403 headers = self._generate_signed_headers(url, requestor_actor_id, method="get")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	404 else:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	405 headers = {}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	406 headers["Accept"] = MEDIA_TYPE_AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	407
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	408 resp = await treq.get(url, headers=headers)
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	409 if resp.code >= 300:
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	410 text = await resp.text()
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	411 if resp.code == 404:
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	412 raise exceptions.NotFound(f"Can't find resource at {url}")
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	413 else:
3977 6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	414 msg = f"HTTP error {resp.code} (url: {url}): {text}"
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	415 raise exceptions.ExternalRequestError(msg)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	416 try:
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	417 return await treq.json_content(resp)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	418 except Exception as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	419 raise error.StanzaError(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	420 "service-unavailable", text=f"Can't get AP data at {url}: {e}"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	421 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	422
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	423 async def ap_post(self, url: str, requestor_actor_id: str, doc: dict) -> TReqResponse:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	424 """Sign a document and post it to AP server
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	425
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	426 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	427 @param requestor_actor_id: originating actor ID (URL)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	428 @param doc: document to send
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	429 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	430 if self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	431 __, actor_args = self.parse_apurl(requestor_actor_id)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	432 actor_account = actor_args[0]
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	433 to_log = [
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	434 "",
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	435 f">>> {actor_account} is signing and posting to {url}:\n{pformat(doc)}",
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	436 ]
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	437
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	438 body = json.dumps(doc).encode()
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	439 headers = self._generate_signed_headers(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	440 url, requestor_actor_id, method="post", body=body
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	441 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	442 headers["Content-Type"] = MEDIA_TYPE_AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	443
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	444 if self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	445 if self.verbose >= 3:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	446 h_to_log = "\n".join(f" {k}: {v}" for k, v in headers.items())
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	447 to_log.append(f" headers:\n{h_to_log}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	448 to_log.append("---")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	449 log.info("\n".join(to_log))
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	450
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	451 resp = await treq.post(url, body, headers=headers)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	452 if resp.code >= 300:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	453 text = await resp.text()
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	454 log.warning(f"POST request to {url} failed [{resp.code}]: {text}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	455 elif self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	456 log.info(f"==> response code: {resp.code}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	457 return resp
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	458
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	459 def _generate_signed_headers(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	460 self, url: str, actor_id: str, method: str, body: bytes \| None = None
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	461 ) -> dict[str, str]:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	462 """Generate HTTP headers with signature for a given request
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	463
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	464 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	465 @param actor_id: originating actor ID (URL)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	466 @param method: HTTP method (e.g., 'get', 'post')
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	467 @param body: request body if any
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	468 @return: signed headers
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	469 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	470 p_url = parse.urlparse(url)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	471 headers = {
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	472 "(request-target)": f"{method} {p_url.path}",
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	473 "Host": p_url.hostname,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	474 "Date": http.datetimeToString().decode(),
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	475 }
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	476
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	477 if body:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	478 digest_algo, digest_hash = self.get_digest(body)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	479 headers["Digest"] = f"{digest_algo}={digest_hash}"
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	480
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	481 headers, __ = self.get_signature_data(self.get_key_id(actor_id), headers)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	482 return headers
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	483
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	484 @overload
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	485 async def ap_get_object(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	486 self, requestor_actor_id: str, data: dict, key: str
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	487 ) -> dict \| None: ...
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	488
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	489 @overload
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	490 async def ap_get_object(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	491 self, requestor_actor_id: str, data: Union[str, dict], key: None = None
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	492 ) -> dict: ...
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	493
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	494 async def ap_get_object(self, requestor_actor_id: str, data, key=None) -> dict \| None:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	495 """Retrieve an AP object, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	496
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	497 This method is to be used with attributes marked as "Functional" in
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	498 https://www.w3.org/TR/activitystreams-vocabulary
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	499 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	500 @param data: AP object where an other object is looked for, or the object itself
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	501 @param key: name of the object to look for, or None if data is the object directly
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	502 @return: found object if any
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	503 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	504 if key is not None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	505 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	506 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	507 value = data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	508 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	509 if key is None:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	510 raise ValueError("None can't be used with ap_get_object is key is None")
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	511 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	512 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	513 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	514 elif isinstance(value, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	515 if self.is_local_url(value):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	516 return await self.ap_get_local_object(value)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	517 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	518 return await self.ap_get(value, requestor_actor_id)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	519 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	520 raise NotImplementedError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	521 "was expecting a string or a dict, got {type(value)}: {value!r}}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	522 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	523
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	524 async def ap_get_local_object(self, url: str) -> dict:
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	525 """Retrieve or generate local object
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	526
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	527 for now, only handle XMPP items to convert to AP
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	528 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	529 url_type, url_args = self.parse_apurl(url)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	530 if url_type == TYPE_ITEM:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	531 try:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	532 account, item_id = url_args
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	533 except ValueError:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	534 raise ValueError(f"invalid URL: {url}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	535 author_jid, node = await self.get_jid_and_node(account)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	536 if node is None:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	537 node = self._m.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	538 cached_node = await self.host.memory.storage.get_pubsub_node(
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	539 self.client, author_jid, node
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	540 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	541 if not cached_node:
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	542 log.debug(f"node {node!r} at {author_jid} is not found in cache")
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	543 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	544 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	545 cached_items, __ = await self.host.memory.storage.get_items(
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	546 cached_node, item_ids=[item_id]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	547 )
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	548 if not cached_items:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	549 log.debug(
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	550 f"item {item_id!r} of {node!r} at {author_jid} is not found in "
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	551 "cache"
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	552 )
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	553 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	554 else:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	555 found_item = cached_items[0].data
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	556
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	557 if found_item is None:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	558 # the node is not in cache, we have to make a request to retrieve the item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	559 # If the node doesn't exist, get_items will raise a NotFound exception
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	560 found_items, __ = await self._p.get_items(
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	561 self.client, author_jid, node, item_ids=[item_id]
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	562 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	563 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	564 found_item = found_items[0]
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	565 except IndexError:
4014 4ef473116499 component AP gateway (http): handle properly NotFound error: Goffi <goffi@goffi.org> parents: 4013 diff changeset	566 raise exceptions.NotFound(f"requested item at {url} can't be found")
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	567
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	568 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	569 # this is an event
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	570 event_data = self._events.event_elt_2_event_data(found_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	571 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	572 event_data, author_jid
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	573 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	574 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	575 ap_item["object"]["@context"] = ap_item["@context"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	576 return ap_item["object"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	577 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	578 # this is a blog item
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	579 mb_data = await self._m.item_2_mb_data(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	580 self.client, found_item, author_jid, node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	581 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	582 ap_item = await self.mb_data_2_ap_item(self.client, mb_data)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	583 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	584 return ap_item["object"]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	585 else:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	586 raise NotImplementedError(
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	587 'only object from "item" URLs can be retrieved for now'
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	588 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	589
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	590 async def ap_get_list(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	591 self, requestor_actor_id: str, data: dict, key: str, only_ids: bool = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	592 ) -> Optional[List[Dict[str, Any]]]:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	593 """Retrieve a list of objects from AP data, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	594
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	595 This method is to be used with non functional vocabularies. Use ``ap_get_object``
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	596 otherwise.
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	597 If the value is a dictionary, it will be wrapped in a list
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	598 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	599 @param data: AP object where a list of objects is looked for
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	600 @param key: key of the list to look for
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	601 @param only_ids: if Trye, only items IDs are retrieved
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	602 @return: list of objects, or None if the key is not present
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	603 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	604 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	605 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	606 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	607 elif isinstance(value, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	608 if self.is_local_url(value):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	609 value = await self.ap_get_local_object(value)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	610 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	611 value = await self.ap_get(value, requestor_actor_id)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	612 if isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	613 return [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	614 if not isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	615 raise ValueError(f"A list was expected, got {type(value)}: {value!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	616 if only_ids:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	617 return [{"id": v["id"]} if isinstance(v, dict) else {"id": v} for v in value]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	618 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	619 return [await self.ap_get_object(requestor_actor_id, i) for i in value]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	620
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	621 async def ap_get_actors(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	622 self, requestor_actor_id: str, data: dict, key: str, as_account: bool = True
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	623 ) -> List[str]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	624 """Retrieve AP actors from data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	625
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	626 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	627 @param data: AP object containing a field with actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	628 @param key: field to use to retrieve actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	629 @param as_account: if True returns account handles, otherwise will return actor
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	630 IDs
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	631 @raise exceptions.DataError: there is not actor data or it is invalid
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	632 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	633 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	634 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	635 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	636 f"no actor associated to object {data.get('id')!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	637 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	638 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	639 actor_id = value.get("id")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	640 if actor_id is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	641 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	642 f"invalid actor associated to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	643 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	644 value = [actor_id]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	645 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	646 value = [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	647 elif isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	648 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	649 value = [a if isinstance(a, str) else a["id"] for a in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	650 except (TypeError, KeyError):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	651 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	652 f"invalid actors list to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	653 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	654 if not value:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	655 raise exceptions.DataError(f"list of actors is empty")
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	656 if as_account:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	657 return [
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	658 await self.get_ap_account_from_id(requestor_actor_id, actor_id)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	659 for actor_id in value
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	660 ]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	661 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	662 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	663
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	664 async def ap_get_sender_actor(
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	665 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	666 requestor_actor_id: str,
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	667 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	668 ) -> str:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	669 """Retrieve actor who sent data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	670
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	671 This is done by checking "actor" field first, then "attributedTo" field.
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	672 Only the first found actor is taken into account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	673 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	674 @param data: AP object
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	675 @return: actor id of the sender
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	676 @raise exceptions.NotFound: no actor has been found in data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	677 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	678 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	679 actors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	680 requestor_actor_id, data, "actor", as_account=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	681 )
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	682 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	683 actors = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	684 if not actors:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	685 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	686 actors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	687 requestor_actor_id, data, "attributedTo", as_account=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	688 )
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	689 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	690 raise exceptions.NotFound(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	691 'actor not specified in "actor" or "attributedTo"'
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	692 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	693 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	694 return actors[0]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	695 except IndexError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	696 raise exceptions.NotFound("list of actors is empty")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	697
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	698 def must_encode(self, text: str) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	699 """Indicate if a text must be period encoded"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	700 return (
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	701 not RE_ALLOWED_UNQUOTED.match(text) or text.startswith("___") or "---" in text
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	702 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	703
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	704 def period_encode(self, text: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	705 """Period encode a text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	706
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	707 see [get_jid_and_node] for reasons of period encoding
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	708 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	709 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	710 parse.quote(text, safe="")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	711 .replace("---", "%2d%2d%2d")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	712 .replace("___", "%5f%5f%5f")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	713 .replace(".", "%2e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	714 .replace("~", "%7e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	715 .replace("%", ".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	716 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	717
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	718 async def get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	719 self, jid_: jid.JID, node: Optional[str]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	720 ) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	721 """Construct AP account from JID and node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	722
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	723 The account construction will use escaping when necessary
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	724 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	725 if not node or node == self._m.namespace:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	726 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	727
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	728 if self.client is None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	729 raise exceptions.InternalError("Client is not set yet")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	730
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	731 if self.is_virtual_jid(jid_):
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	732 # this is an proxy JID to an AP Actor
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	733 return self._e.unescape(jid_.user)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	734
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	735 if node and not jid_.user and not self.must_encode(node):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	736 is_pubsub = await self.is_pubsub(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	737 # when we have a pubsub service, the user part can be used to set the node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	738 # this produces more user-friendly AP accounts
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	739 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	740 jid_.user = node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	741 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	742
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	743 is_local = self.is_local(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	744 user = jid_.user if is_local else jid_.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	745 if user is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	746 user = ""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	747 account_elts = []
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	748 if node and self.must_encode(node) or self.must_encode(user):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	749 account_elts = ["___"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	750 if node:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	751 node = self.period_encode(node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	752 user = self.period_encode(user)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	753
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	754 if not user:
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	755 raise MissingLocalPartError("there should be a local part")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	756
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	757 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	758 account_elts.extend((node, "---"))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	759
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	760 account_elts.extend(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	761 (user, "@", jid_.host if is_local else self.client.jid.userhost())
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	762 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	763 return "".join(account_elts)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	764
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	765 def is_local(self, jid_: jid.JID) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	766 """Returns True if jid_ use a domain or subdomain of gateway's host"""
4297 0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	767 # FIXME: kept for compatiblity, need to be removed in favor of
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	768 # "self.client.is_local".
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	769
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	770 return self.client.is_local(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	771
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	772 async def is_pubsub(self, jid_: jid.JID) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	773 """Indicate if a JID is a Pubsub service"""
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	774 host_disco = await self.host.get_disco_infos(self.client, jid_)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	775 return ("pubsub", "service") in host_disco.identities and not (
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	776 "pubsub",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	777 "pep",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	778 ) in host_disco.identities
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	779
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	780 async def get_jid_and_node(self, ap_account: str) -> tuple[jid.JID, str \| None]:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	781 """Decode raw AP account handle to get XMPP JID and Pubsub Node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	782
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	783 Username are case insensitive.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	784
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	785 By default, the username correspond to local username (i.e. username from
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	786 component's server).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	787
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	788 If local name's domain is a pubsub service (and not PEP), the username is taken as
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	789 a pubsub node.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	790
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	791 If ``---`` is present in username, the part before is used as pubsub node, and the
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	792 rest as a JID user part.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	793
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	794 If username starts with ``___``, characters are encoded using period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	795 (i.e. percent encoding where a ``.`` is used instead of ``%``).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	796
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	797 This horror is necessary due to limitation in some AP implementation (notably
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	798 Mastodon), cf. https://github.com/mastodon/mastodon/issues/17222
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	799
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	800 examples:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	801
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	802 ``toto@example.org`` => JID = toto@example.org, node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	803
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	804 ``___toto.40example.net@example.org`` => JID = toto@example.net (this one is a
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	805 non-local JID, and will work only if setings ``local_only`` is False), node = None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	806
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	807 ``toto@pubsub.example.org`` (with pubsub.example.org being a pubsub service) =>
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	808 JID = pubsub.example.org, node = toto
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	809
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	810 ``tata---toto@example.org`` => JID = toto@example.org, node = tata
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	811
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	812 ``___urn.3axmpp.3amicroblog.3a0@pubsub.example.org`` (with pubsub.example.org
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	813 being a pubsub service) ==> JID = pubsub.example.org, node = urn:xmpp:microblog:0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	814
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	815 @param ap_account: ActivityPub account handle (``username@domain.tld``)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	816 @return: service JID and pubsub node
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	817 if pubsub node is None, default microblog pubsub node (and possibly other
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	818 nodes that plugins may hanlde) will be used
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	819 @raise ValueError: invalid account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	820 @raise PermissionError: non local jid is used when gateway doesn't allow them
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	821 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	822 if ap_account.count("@") != 1:
4262 d366d90a71aa component AP Gateway: log invalid account in case of error. Goffi <goffi@goffi.org> parents: 4259 diff changeset	823 raise ValueError(f"Invalid AP account {ap_account!r}")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	824 if ap_account.startswith("___"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	825 encoded = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	826 ap_account = ap_account[3:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	827 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	828 encoded = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	829
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	830 username, domain = ap_account.split("@")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	831
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	832 if "---" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	833 node, username = username.rsplit("---", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	834 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	835 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	836
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	837 if encoded:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	838 username = parse.unquote(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	839 RE_PERIOD_ENC.sub(r"%\g<hex>", username), errors="strict"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	840 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	841 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	842 node = parse.unquote(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	843 RE_PERIOD_ENC.sub(r"%\g<hex>", node), errors="strict"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	844 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	845
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	846 if "@" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	847 username, domain = username.rsplit("@", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	848
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	849 if not node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	850 # we need to check host disco, because disco request to user may be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	851 # blocked for privacy reason (see
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	852 # https://xmpp.org/extensions/xep-0030.html#security)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	853 is_pubsub = await self.is_pubsub(jid.JID(domain))
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	854
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	855 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	856 # if the host is a pubsub service and not a PEP, we consider that username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	857 # is in fact the node name
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	858 node = username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	859 username = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	860
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	861 jid_s = f"{username}@{domain}" if username else domain
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	862 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	863 jid_ = jid.JID(jid_s)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	864 except RuntimeError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	865 raise ValueError(f"Invalid jid: {jid_s!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	866
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	867 if self.local_only and not self.is_local(jid_):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	868 raise exceptions.PermissionError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	869 "This gateway is configured to map only local entities and services"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	870 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	871
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	872 return jid_, node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	873
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	874 def get_local_jid_from_account(self, account: str) -> jid.JID:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	875 """Compute JID linking to an AP account
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	876
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	877 The local jid is computer by escaping AP actor handle and using it as local part
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	878 of JID, where domain part is this gateway own JID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	879 """
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	880 return jid.JID(None, (self._e.escape(account), self.client.jid.host, None))
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	881
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	882 async def get_jid_from_id(self, requestor_actor_id: str, actor_id: str) -> jid.JID:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	883 """Compute JID linking to an AP Actor ID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	884
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	885 The local jid is computer by escaping AP actor handle and using it as local part
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	886 of JID, where domain part is this gateway own JID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	887 If the actor_id comes from local server (checked with self.public_url), it means
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	888 that we have an XMPP entity, and the original JID is returned
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	889
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	890 @param requestor_actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	891 @param actor_id: ID of the actor to generate JID from.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	892 @return: generated JID.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	893 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	894 if self.is_local_url(actor_id):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	895 request_type, extra_args = self.parse_apurl(actor_id)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	896 if request_type != TYPE_ACTOR or len(extra_args) != 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	897 raise ValueError(f"invalid actor id: {actor_id!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	898 actor_jid, __ = await self.get_jid_and_node(extra_args[0])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	899 return actor_jid
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	900
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	901 account = await self.get_ap_account_from_id(requestor_actor_id, actor_id)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	902 return self.get_local_jid_from_account(account)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	903
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	904 def parse_apurl(self, url: str) -> tuple[str, list[str]]:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	905 """Parse an URL leading to an AP endpoint
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	906
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	907 @param url: URL to parse (schema is not mandatory)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	908 @return: endpoint type and extra arguments
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	909 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	910 path = parse.urlparse(url).path.lstrip("/")
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	911 type_, *extra_args = path[len(self.ap_path) :].lstrip("/").split("/")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	912 return type_, [parse.unquote(a) for a in extra_args]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	913
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	914 def build_apurl(self, type_: str, *args: str) -> str:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	915 """Build an AP endpoint URL
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	916
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	917 @param type_: type of AP endpoing
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	918 @param arg: endpoint dependant arguments
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	919 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	920 return parse.urljoin(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	921 self.base_ap_url,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	922 str(Path(type_).joinpath(*(parse.quote_plus(a, safe="@") for a in args))),
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	923 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	924
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	925 def is_local_url(self, url: str) -> bool:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	926 """Tells if an URL link to this component
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	927
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	928 ``public_url`` and ``ap_path`` are used to check the URL
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	929 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	930 return url.startswith(self.base_ap_url)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	931
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	932 def is_virtual_jid(self, jid_: jid.JID) -> bool:
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	933 """Tell if a JID is an AP actor mapped through this gateway"""
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	934 return jid_.host == self.client.jid.userhost()
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	935
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	936 def build_signature_header(self, values: Dict[str, str]) -> str:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	937 """Build key="<value>" signature header from signature data"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	938 fields = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	939 for key, value in values.items():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	940 if key not in ("(created)", "(expired)"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	941 if '"' in value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	942 raise NotImplementedError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	943 "string escaping is not implemented, double-quote can't be used "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	944 f"in {value!r}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	945 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	946 value = f'"{value}"'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	947 fields.append(f"{key}={value}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	948
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	949 return ",".join(fields)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	950
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	951 def get_digest(self, body: bytes, algo="SHA-256") -> Tuple[str, str]:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	952 """Get digest data to use in header and signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	953
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	954 @param body: body of the request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	955 @return: hash name and digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	956 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	957 if algo != "SHA-256":
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	958 raise NotImplementedError("only SHA-256 is implemented for now")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	959 return algo, base64.b64encode(hashlib.sha256(body).digest()).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	960
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	961 @async_lru(maxsize=LRU_MAX_SIZE)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	962 async def get_actor_data(self, requestor_actor_id: str, actor_id: str) -> dict:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	963 """Retrieve actor data with LRU cache"""
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	964 return await self.ap_get(actor_id, requestor_actor_id)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	965
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	966 @async_lru(maxsize=LRU_MAX_SIZE)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	967 async def get_actor_pub_key_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	968 self, requestor_actor_id: str, actor_id: str
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	969 ) -> Tuple[str, str, rsa.RSAPublicKey]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	970 """Retrieve Public Key data from actor ID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	971
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	972 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	973 @param actor_id: actor ID (url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	974 @return: key_id, owner and public_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	975 @raise KeyError: publicKey is missing from actor data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	976 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	977 actor_data = await self.get_actor_data(requestor_actor_id, actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	978 pub_key_data = actor_data["publicKey"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	979 key_id = pub_key_data["id"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	980 owner = pub_key_data["owner"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	981 pub_key_pem = pub_key_data["publicKeyPem"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	982 pub_key = serialization.load_pem_public_key(pub_key_pem.encode())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	983 return key_id, owner, pub_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	984
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	985 def create_activity(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	986 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	987 activity: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	988 actor_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	989 object_: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	990 target: Optional[Union[str, dict]] = None,
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	991 activity_id: Optional[str] = None,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	992 **kwargs,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	993 ) -> Dict[str, Any]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	994 """Generate base data for an activity
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	995
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	996 @param activity: one of ACTIVITY_TYPES
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	997 @param actor_id: AP actor ID of the sender
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	998 @param object_: content of "object" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	999 @param target: content of "target" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1000 @param activity_id: ID to use for the activity
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1001 if not set it will be automatically generated, but it is usually desirable to
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1002 set the ID manually so it can be retrieved (e.g. for Undo)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1003 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1004 if activity not in ACTIVITY_TYPES:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1005 raise exceptions.InternalError(f"invalid activity: {activity!r}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1006 if object_ is None and activity in ACTIVITY_OBJECT_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1007 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1008 f'"object_" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1009 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1010 if target is None and activity in ACTIVITY_TARGET_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1011 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1012 f'"target" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1013 )
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	1014 if activity_id is None:
4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	1015 activity_id = f"{actor_id}#{activity.lower()}_{shortuuid.uuid()}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1016 data: Dict[str, Any] = {
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1017 "@context": [NS_AP],
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1018 "actor": actor_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1019 "id": activity_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1020 "type": activity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1021 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1022 data.update(kwargs)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1023 if object_ is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1024 data["object"] = object_
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1025 if target is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1026 data["target"] = target
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1027
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1028 return data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1029
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1030 def get_key_id(self, actor_id: str) -> str:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1031 """Get local key ID from actor ID"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1032 return f"{actor_id}#main-key"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1033
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1034 async def check_signature(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1035 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1036 requestor_actor_id: str,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1037 signature: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1038 key_id: str,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1039 headers: Dict[str, str],
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1040 ) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1041 """Verify that signature matches given headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1042
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1043 see https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-06#section-3.1.2
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1044
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1045 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1046 @param signature: Base64 encoded signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1047 @param key_id: ID of the key used to sign the data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1048 @param headers: headers and their values, including pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1049 @return: id of the signing actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1050
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1051 @raise InvalidSignature: signature doesn't match headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1052 """
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1053 to_sign = "\n".join(f"{k.lower()}: {v}" for k, v in headers.items())
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1054 if key_id.startswith("acct:"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1055 actor = key_id[5:]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1056 actor_id = await self.get_ap_actor_id_from_account(actor)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1057 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1058 actor_id = key_id.split("#", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1059
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1060 pub_key_id, pub_key_owner, pub_key = await self.get_actor_pub_key_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1061 requestor_actor_id, actor_id
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1062 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1063 if pub_key_id != key_id or pub_key_owner != actor_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1064 raise exceptions.EncryptionError("Public Key mismatch")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1065
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1066 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1067 pub_key.verify(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1068 base64.b64decode(signature),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1069 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1070 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1071 padding.PKCS1v15(), # type: ignore
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1072 hashes.SHA256(), # type: ignore
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1073 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1074 except InvalidSignature:
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1075 raise exceptions.EncryptionError(
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1076 "Invalid signature (using PKC0S1 v1.5 and SHA-256)"
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1077 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1078
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1079 return actor_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1080
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1081 def get_signature_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1082 self, key_id: str, headers: Dict[str, str]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1083 ) -> Tuple[Dict[str, str], Dict[str, str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1084 """Generate and return signature and corresponding headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1085
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1086 @param parsed_url: URL where the request is sent/has been received
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1087 @param key_id: ID of the key (URL linking to the data with public key)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1088 @param date: HTTP datetime string of signature generation
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1089 @param body: body of the HTTP request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1090 @param headers: headers to sign and their value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1091 default value will be used if not specified
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1092
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1093 @return: headers and signature data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1094 ``headers`` is an updated copy of ``headers`` arguments, with pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1095 removed, and ``Signature`` added.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1096 """
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1097 # headers must be lower case
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1098 l_headers: Dict[str, str] = {k.lower(): v for k, v in headers.items()}
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1099 to_sign = "\n".join(f"{k}: {v}" for k, v in l_headers.items())
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1100 signature = base64.b64encode(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1101 self.private_key.sign(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1102 to_sign.encode(),
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1103 # we have to use PKCS1v15 padding to be compatible with Mastodon
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1104 padding.PKCS1v15(), # type: ignore
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1105 hashes.SHA256(), # type: ignore
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1106 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1107 ).decode()
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1108 sign_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1109 "keyId": key_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1110 "Algorithm": "rsa-sha256",
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1111 "headers": " ".join(l_headers.keys()),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1112 "signature": signature,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1113 }
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1114 new_headers = {k: v for k, v in headers.items() if not k.startswith("(")}
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1115 new_headers["Signature"] = self.build_signature_header(sign_data)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1116 return new_headers, sign_data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1117
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1118 async def convert_and_post_items(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1119 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1120 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1121 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1122 service: jid.JID,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1123 node: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1124 items: List[domish.Element],
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1125 subscribe_extra_nodes: bool = True,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1126 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1127 """Convert XMPP items to AP items and post them to actor inbox
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1128
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1129 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1130 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1131 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1132 @param node: (virtual) node corresponding where the item has been published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1133 @param subscribe_extra_nodes: if True, extra data nodes will be automatically
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1134 subscribed, that is comment nodes if present and attachments nodes.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1135 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1136 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1137 requestor_actor_id = self.build_apurl(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1138 TYPE_ACTOR, await self.get_ap_account_from_jid_and_node(service, node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1139 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1140 inbox = await self.get_ap_inbox_from_id(requestor_actor_id, actor_id)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1141 for item in items:
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1142 if item.name == "item":
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1143 cached_item = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1144 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1145 "profiles": [self.client.profile],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1146 "services": [service],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1147 "nodes": [node],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1148 "names": [item["id"]],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1149 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1150 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1151 is_new = not bool(cached_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1152 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1153 # event item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1154 event_data = self._events.event_elt_2_event_data(item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1155 try:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1156 author_jid = jid.JID(item["publisher"]).userhostJID()
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1157 except (KeyError, RuntimeWarning):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1158 root_elt = item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1159 while root_elt.parent is not None:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1160 root_elt = root_elt.parent
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1161 author_jid = jid.JID(root_elt["from"]).userhostJID()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1162 if subscribe_extra_nodes and not self.is_virtual_jid(author_jid):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1163 # we subscribe automatically to comment nodes if any
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1164 recipient_jid = self.get_local_jid_from_account(ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1165 recipient_client = self.client.get_virtual_client(recipient_jid)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1166 comments_data = event_data.get("comments")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1167 if comments_data:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1168 comment_service = jid.JID(comments_data["jid"])
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1169 comment_node = comments_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1170 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1171 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1172 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1173 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1174 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1175 recipient_client, service, node, event_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1176 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1177 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1178 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1179 f"no attachment node found for item {event_data['id']!r} "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1180 f"on {node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1181 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1182 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1183 event_data, author_jid, is_new=is_new
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1184 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1185 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1186 # blog item
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1187 mb_data = await self._m.item_2_mb_data(client, item, service, node)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1188 author_jid = jid.JID(mb_data["author_jid"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1189 if subscribe_extra_nodes and not self.is_virtual_jid(author_jid):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1190 # we subscribe automatically to comment nodes if any
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1191 recipient_jid = self.get_local_jid_from_account(ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1192 recipient_client = self.client.get_virtual_client(recipient_jid)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1193 for comment_data in mb_data.get("comments", []):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1194 comment_service = jid.JID(comment_data["service"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1195 if self.is_virtual_jid(comment_service):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1196 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1197 f"ignoring virtual comment service: {comment_data}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1198 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1199 continue
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1200 comment_node = comment_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1201 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1202 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1203 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1204 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1205 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1206 recipient_client, service, node, mb_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1207 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1208 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1209 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1210 f"no attachment node found for item {mb_data['id']!r} on "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1211 f"{node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1212 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1213 ap_item = await self.mb_data_2_ap_item(client, mb_data, is_new=is_new)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1214
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	1215 url_actor = ap_item["actor"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1216 elif item.name == "retract":
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1217 url_actor, ap_item = await self.ap_delete_item(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	1218 client.jid, node, item["id"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1219 )
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1220 else:
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1221 raise exceptions.InternalError(f"unexpected element: {item.toXml()}")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1222 await self.ap_post(inbox, url_actor, ap_item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1223
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1224 async def convert_and_post_attachments(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1225 self,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1226 client: SatXMPPEntity,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1227 ap_account: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1228 service: jid.JID,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1229 node: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1230 items: List[domish.Element],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1231 publisher: Optional[jid.JID] = None,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1232 ) -> None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1233 """Convert XMPP item attachments to AP activities and post them to actor inbox
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1234
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1235 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1236 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1237 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1238 @param node: (virtual) node corresponding where the item has been published
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1239 subscribed, that is comment nodes if present and attachments nodes.
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1240 @param items: attachments items
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1241 @param publisher: publisher of the attachments item (it's NOT the PEP/Pubsub
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1242 service, it's the publisher of the item). To be filled only when the publisher
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1243 is known for sure, otherwise publisher will be determined either if
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1244 "publisher" attribute is set by pubsub service, or as a last resort, using
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1245 item's ID (which MUST be publisher bare JID according to pubsub-attachments
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1246 specification).
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1247 """
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1248 if len(items) != 1:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1249 log.warning(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1250 "we should get exactly one attachment item for an entity, got "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1251 f"{len(items)})"
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1252 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1253
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1254 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1255 requestor_actor_id = self.build_apurl(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1256 TYPE_ACTOR, await self.get_ap_account_from_jid_and_node(service, node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1257 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1258 inbox = await self.get_ap_inbox_from_id(requestor_actor_id, actor_id)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1259
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1260 item_elt = items[0]
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1261 item_id = item_elt["id"]
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1262
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1263 if publisher is None:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1264 item_pub_s = item_elt.getAttribute("publisher")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1265 publisher = jid.JID(item_pub_s) if item_pub_s else jid.JID(item_id)
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1266
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1267 if publisher.userhost() != item_id:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1268 log.warning(
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1269 "attachments item ID must be publisher's bare JID, ignoring: "
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1270 f"{item_elt.toXml()}"
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1271 )
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1272 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1273
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1274 if self.is_virtual_jid(publisher):
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1275 log.debug(f"ignoring item coming from local virtual JID {publisher}")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1276 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1277
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1278 if publisher is not None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1279 item_elt["publisher"] = publisher.userhost()
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1280
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1281 item_service, item_node, item_id = self._pa.attachment_node_2_item(node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1282 item_account = await self.get_ap_account_from_jid_and_node(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1283 item_service, item_node
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1284 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1285 if self.is_virtual_jid(item_service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1286 # it's a virtual JID mapping to an external AP actor, we can use the
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1287 # item_id directly
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1288 item_url = item_id
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1289 if not item_url.startswith("https:"):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1290 log.warning(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1291 "item ID of external AP actor is not an https link, ignoring: "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1292 f"{item_id!r}"
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1293 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1294 return
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1295 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1296 item_url = self.build_apurl(TYPE_ITEM, item_account, item_id)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1297
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1298 old_attachment_pubsub_items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1299 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1300 "profiles": [self.client.profile],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1301 "services": [service],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1302 "nodes": [node],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1303 "names": [item_elt["id"]],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1304 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1305 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1306 if not old_attachment_pubsub_items:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1307 old_attachment = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1308 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1309 old_attachment_items = [i.data for i in old_attachment_pubsub_items]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1310 old_attachments = self._pa.items_2_attachment_data(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1311 client, old_attachment_items
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1312 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1313 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1314 old_attachment = old_attachments[0]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1315 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1316 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1317 old_attachment = {}
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1318 publisher_account = await self.get_ap_account_from_jid_and_node(publisher, None)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1319 publisher_actor_id = self.build_apurl(TYPE_ACTOR, publisher_account)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1320 try:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1321 attachments = self._pa.items_2_attachment_data(client, [item_elt])[0]
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1322 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1323 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1324 attachments = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1325
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1326 # noticed
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1327 if "noticed" in attachments:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1328 if not "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1329 # new "noticed" attachment, we translate to "Like" activity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1330 activity_id = self.build_apurl("like", item_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1331 activity = self.create_activity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1332 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1333 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1334 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1335 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1336 await self.ap_post(inbox, publisher_actor_id, activity)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1337 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1338 if "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1339 # "noticed" attachment has been removed, we undo the "Like" activity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1340 activity_id = self.build_apurl("like", item_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1341 activity = self.create_activity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1342 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1343 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1344 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1345 activity["cc"] = [NS_AP_PUBLIC]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1346 undo = self.create_activity("Undo", publisher_actor_id, activity)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1347 await self.ap_post(inbox, publisher_actor_id, undo)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1348
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1349 # reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1350 new_reactions = set(attachments.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1351 old_reactions = set(old_attachment.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1352 reactions_remove = old_reactions - new_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1353 reactions_add = new_reactions - old_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1354 for reactions, undo in ((reactions_remove, True), (reactions_add, False)):
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1355 for reaction in reactions:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1356 activity_id = self.build_apurl(
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1357 "reaction", item_account, item_id, reaction.encode().hex()
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1358 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1359 reaction_activity = self.create_activity(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1360 TYPE_REACTION, publisher_actor_id, item_url, activity_id=activity_id
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1361 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1362 reaction_activity["content"] = reaction
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1363 reaction_activity["to"] = [ap_account]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1364 reaction_activity["cc"] = [NS_AP_PUBLIC]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1365 if undo:
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1366 activy = self.create_activity(
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1367 "Undo", publisher_actor_id, reaction_activity
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1368 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1369 else:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1370 activy = reaction_activity
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1371 await self.ap_post(inbox, publisher_actor_id, activy)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1372
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1373 # RSVP
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1374 if "rsvp" in attachments:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1375 attending = attachments["rsvp"].get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1376 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1377 if attending != old_attending:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1378 activity_type = TYPE_JOIN if attending == "yes" else TYPE_LEAVE
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1379 activity_id = self.build_apurl(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1380 activity_type.lower(), item_account, item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1381 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1382 activity = self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1383 activity_type, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1384 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1385 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1386 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1387 await self.ap_post(inbox, publisher_actor_id, activity)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1388 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1389 if "rsvp" in old_attachment:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1390 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1391 if old_attending == "yes":
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1392 activity_id = self.build_apurl(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1393 TYPE_LEAVE.lower(), item_account, item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1394 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1395 activity = self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1396 TYPE_LEAVE, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1397 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1398 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1399 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1400 await self.ap_post(inbox, publisher_actor_id, activity)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1401
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1402 if service.user and self.is_virtual_jid(service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1403 # the item is on a virtual service, we need to store it in cache
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1404 log.debug("storing attachments item in cache")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1405 cached_node = await self.host.memory.storage.get_pubsub_node(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1406 client, service, node, with_subscriptions=True, create=True
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1407 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1408 await self.host.memory.storage.cache_pubsub_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1409 self.client, cached_node, [item_elt], [attachments]
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1410 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1411
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1412 def _publish_message(self, mess_data_s: str, service_s: str, profile: str):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1413 mess_data: dict = data_format.deserialise(mess_data_s) # type: ignore
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1414 service = jid.JID(service_s)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1415 client = self.host.get_client(profile)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1416 return defer.ensureDeferred(self.publish_message(client, mess_data, service))
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1417
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1418 @async_lru(maxsize=LRU_MAX_SIZE)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1419 async def get_ap_actor_id_from_account(self, account: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1420 """Retrieve account ID from it's handle using WebFinger
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1421
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1422 Don't use this method to get local actor id from a local account derivated for
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1423 JID: in this case, the actor ID is retrieve with
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1424 ``self.build_apurl(TYPE_ACTOR, ap_account)``
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1425
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1426 @param account: AP handle (user@domain.tld)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1427 @return: Actor ID (which is an URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1428 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1429 if account.count("@") != 1 or "/" in account:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1430 raise ValueError(f"Invalid account: {account!r}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1431 host = account.split("@")[1]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1432 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1433 finger_data = await treq.json_content(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1434 await treq.get(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1435 f"https://{host}/.well-known/webfinger?"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1436 f"resource=acct:{parse.quote_plus(account)}",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1437 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1438 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1439 except Exception as e:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1440 raise exceptions.DataError(f"Can't get webfinger data for {account!r}: {e}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1441 for link in finger_data.get("links", []):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1442 if (
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1443 link.get("type") == "application/activity+json"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1444 and link.get("rel") == "self"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1445 ):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1446 href = link.get("href", "").strip()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1447 if not href:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1448 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1449 f"Invalid webfinger data for {account:r}: missing href"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1450 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1451 break
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1452 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1453 raise ValueError(f"No ActivityPub link found for {account!r}")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1454 return href
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1455
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1456 async def get_ap_actor_data_from_account(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1457 self, requestor_actor_id: str, account: str
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1458 ) -> dict:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1459 """Retrieve ActivityPub Actor data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1460
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1461 @param account: ActivityPub Actor identifier
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1462 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1463 href = await self.get_ap_actor_id_from_account(account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1464 return await self.ap_get(href, requestor_actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1465
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1466 async def get_ap_inbox_from_id(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1467 self, requestor_actor_id: str, actor_id: str, use_shared: bool = True
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1468 ) -> str:
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1469 """Retrieve inbox of an actor_id
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1470
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1471 @param requestor_actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1472 @param actor_id: ID of the actor from whom Inbox must be retrieved.
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1473 @param use_shared: if True, and a shared inbox exists, it will be used instead of
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1474 the user inbox
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1475 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1476 data = await self.get_actor_data(requestor_actor_id, actor_id)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1477 if use_shared:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1478 try:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1479 return data["endpoints"]["sharedInbox"]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1480 except KeyError:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1481 pass
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1482 return data["inbox"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1483
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1484 @async_lru(maxsize=LRU_MAX_SIZE)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1485 async def get_ap_account_from_id(self, requestor_actor_id: str, actor_id: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1486 """Retrieve AP account from the ID URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1487
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1488 Works with external or local actor IDs.
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1489 @param requestor_actor_id: ID of the actor doing the request.
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1490 @param actor_id: AP ID of the actor (URL to the actor data)
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1491 @return: AP handle
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1492 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1493 if self.is_local_url(actor_id):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1494 url_type, url_args = self.parse_apurl(actor_id)
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1495 if url_type != "actor" or not url_args:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1496 raise exceptions.DataError(f"invalid local actor ID: {actor_id}")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1497 account = url_args[0]
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1498 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1499 account_user, account_host = account.split("@")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1500 except ValueError:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1501 raise exceptions.DataError(f"invalid account from url: {actor_id}")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1502 if not account_user or account_host != self.public_url:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1503 raise exceptions.DataError(
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1504 f"{account!r} is not a valid local account (from {actor_id})"
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1505 )
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1506 return account
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1507
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1508 url_parsed = parse.urlparse(actor_id)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1509 actor_data = await self.get_actor_data(requestor_actor_id, actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1510 username = actor_data.get("preferredUsername")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1511 if not username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1512 raise exceptions.DataError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1513 'No "preferredUsername" field found, can\'t retrieve actor account'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1514 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1515 account = f"{username}@{url_parsed.hostname}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1516 # we try to retrieve the actor ID from the account to check it
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1517 found_id = await self.get_ap_actor_id_from_account(account)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1518 if found_id != actor_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1519 # cf. https://socialhub.activitypub.rocks/t/how-to-retrieve-user-server-tld-handle-from-actors-url/2196
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1520 msg = (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1521 f"Account ID found on WebFinger {found_id!r} doesn't match our actor ID "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1522 f"({actor_id!r}). This AP instance doesn't seems to use "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1523 '"preferredUsername" as we expect.'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1524 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1525 log.warning(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1526 raise exceptions.DataError(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1527 return account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1528
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1529 async def get_ap_items(
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1530 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1531 actor_id: str,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1532 collection: dict,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1533 max_items: Optional[int] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1534 chronological_pagination: bool = True,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1535 after_id: Optional[str] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1536 start_index: Optional[int] = None,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1537 parser: Optional[Callable[[str, dict], Awaitable[domish.Element]]] = None,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1538 only_ids: bool = False,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1539 ) -> Tuple[List[domish.Element], rsm.RSMResponse]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1540 """Retrieve AP items and convert them to XMPP items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1541
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1542 @param actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1543 @param collection: AP collection data.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1544 Items will be retrieved from this collection.
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1545 @param max_items: maximum number of items to retrieve
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1546 retrieve all items by default
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1547 @param chronological_pagination: get pages in chronological order
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1548 AP use reversed chronological order for pagination, "first" page returns more
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1549 recent items. If "chronological_pagination" is True, "last" AP page will be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1550 retrieved first.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1551 @param after_id: if set, retrieve items starting from given ID
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1552 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1553 if ``after_id`` is not already in cache, we have to retrieve every page until
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1554 we find it.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1555 In most common cases, ``after_id`` should be in cache though (client usually
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1556 use known ID when in-order pagination is used).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1557 @param start_index: start retrieving items from the one with given index
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1558 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1559 all pages before the requested index will be retrieved to count items.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1560 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1561 if None, use default generic parser
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1562 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1563 Retrieving only item IDs avoid HTTP requests to retrieve items, it may be
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1564 sufficient in some use cases (e.g. when retrieving following/followers
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1565 collections)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1566 @return: XMPP Pubsub items and corresponding RSM Response
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1567 Items are always returned in chronological order in the result
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1568 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1569 if parser is None:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1570 parser = self.ap_item_2_mb_elt
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1571
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1572 rsm_resp: Dict[str, Union[bool, int]] = {}
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1573 try:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1574 count = collection["totalItems"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1575 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1576 log.warning(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1577 f'"totalItems" not found in collection {collection.get("id")}, '
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1578 "defaulting to 20"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1579 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1580 count = 20
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1581 else:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1582 log.info(f"{collection.get('id')} has {count} item(s)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1583
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1584 rsm_resp["count"] = count
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1585
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1586 if start_index is not None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1587 assert chronological_pagination and after_id is None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1588 if start_index >= count:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1589 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1590 elif start_index == 0:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1591 # this is the default behaviour
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1592 pass
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1593 elif start_index > 5000:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1594 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1595 "feature-not-implemented",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1596 text="Maximum limit for previous_index has been reached, this limit"
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1597 "is set to avoid DoS",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1598 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1599 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1600 # we'll convert "start_index" to "after_id", thus we need the item just
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1601 # before "start_index"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1602 previous_index = start_index - 1
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1603 retrieved_items = 0
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1604 current_page = collection["last"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1605 while retrieved_items < count:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1606 page_data, items = await self.parse_ap_page(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1607 actor_id, current_page, parser, only_ids
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1608 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1609 if not items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1610 log.warning(f"found an empty AP page at {current_page}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1611 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1612 page_start_idx = retrieved_items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1613 retrieved_items += len(items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1614 if previous_index <= retrieved_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1615 after_id = items[previous_index - page_start_idx]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1616 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1617 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1618 current_page = page_data["prev"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1619 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1620 log.warning(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1621 f"missing previous page link at {current_page}: {page_data!r}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1622 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1623 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1624 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1625 "Error while retrieving previous page from AP service at "
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1626 f"{current_page}",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1627 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1628
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1629 init_page = "last" if chronological_pagination else "first"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1630 page = collection.get(init_page)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1631 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1632 raise exceptions.DataError(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1633 f"Initial page {init_page!r} not found for collection "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1634 f"{collection.get('id')})"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1635 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1636 items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1637 page_items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1638 retrieved_items = 0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1639 found_after_id = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1640
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1641 while retrieved_items < count:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1642 __, page_items = await self.parse_ap_page(actor_id, page, parser, only_ids)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1643 if not page_items:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1644 break
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1645 retrieved_items += len(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1646 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1647 # if we have an after_id, we ignore all items until the requested one is
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1648 # found
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1649 try:
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1650 limit_idx = [i["id"] for i in page_items].index(after_id)
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1651 except ValueError:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1652 # if "after_id" is not found, we don't add any item from this page
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1653 page_id = page.get("id") if isinstance(page, dict) else page
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1654 log.debug(f"{after_id!r} not found at {page_id}, skipping")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1655 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1656 found_after_id = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1657 if chronological_pagination:
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1658 start_index = retrieved_items - len(page_items) + limit_idx + 1
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1659 page_items = page_items[limit_idx + 1 :]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1660 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1661 start_index = count - (
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1662 retrieved_items - len(page_items) + limit_idx + 1
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1663 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1664 page_items = page_items[:limit_idx]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1665 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1666 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1667 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1668 if max_items is not None and len(items) >= max_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1669 if chronological_pagination:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1670 items = items[:max_items]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1671 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1672 items = items[-max_items:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1673 break
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1674 page = collection.get("prev" if chronological_pagination else "next")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1675 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1676 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1677
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1678 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1679 raise error.StanzaError("item-not-found")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1680
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1681 if items:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1682 if after_id is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1683 rsm_resp["index"] = 0 if chronological_pagination else count - len(items)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1684 if start_index is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1685 rsm_resp["index"] = start_index
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1686 elif after_id is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1687 log.warning("Can't determine index of first element")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1688 elif chronological_pagination:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1689 rsm_resp["index"] = 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1690 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1691 rsm_resp["index"] = count - len(items)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1692 rsm_resp.update({"first": items[0]["id"], "last": items[-1]["id"]})
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1693
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1694 return items, rsm.RSMResponse(**rsm_resp)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1695
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1696 async def ap_item_2_mb_data_and_elt(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1697 self, requestor_actor_id: str, ap_item: dict
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1698 ) -> tuple[dict, domish.Element]:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1699 """Convert AP item to parsed microblog data and corresponding item element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1700
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1701 @param requestor_actor_id: ID of the actor requesting the conversion.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1702 @param ap_item: AP item to convert.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1703 @return: microblog and correspondign <item> element.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1704 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1705 mb_data = await self.ap_item_2_mb_data(requestor_actor_id, ap_item)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1706 item_elt = await self._m.mb_data_2_entry_elt(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1707 self.client, mb_data, mb_data["id"], None, self._m.namespace
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1708 )
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1709 if "repeated" in mb_data["extra"]:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1710 item_elt["publisher"] = mb_data["extra"]["repeated"]["by"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1711 else:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1712 item_elt["publisher"] = mb_data["author_jid"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1713 return mb_data, item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1714
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1715 async def ap_item_2_mb_elt(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1716 self, requestor_actor_id: str, ap_item: dict
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1717 ) -> domish.Element:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1718 """Convert AP item to XMPP item element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1719
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1720 @param requestor_actor_id: ID of the actor requesting the conversion.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1721 @param ap_item: AP item to convert.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1722 @return: <item> element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1723 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1724 __, item_elt = await self.ap_item_2_mb_data_and_elt(requestor_actor_id, ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1725 return item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1726
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1727 async def parse_ap_page(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1728 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1729 requestor_actor_id: str,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1730 page: Union[str, dict],
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1731 parser: Callable[[str, dict], Awaitable[domish.Element]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1732 only_ids: bool = False,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1733 ) -> Tuple[dict, List[domish.Element]]:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1734 """Convert AP objects from an AP page to XMPP items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1735
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1736 @param requestor_actor_id: ID of the actor doing the request.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1737 @param page: Can be either url linking and AP page, or the page data directly
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1738 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1739 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1740 @return: page data, pubsub items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1741 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1742 page_data = await self.ap_get_object(requestor_actor_id, page)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1743 if page_data is None:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1744 log.warning("No data found in collection")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1745 return {}, []
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1746 ap_items = await self.ap_get_list(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1747 requestor_actor_id, page_data, "orderedItems", only_ids=only_ids
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1748 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1749 if ap_items is None:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1750 ap_items = await self.ap_get_list(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1751 requestor_actor_id, page_data, "items", only_ids=only_ids
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1752 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1753 if not ap_items:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1754 log.warning(f"No item field found in collection: {page_data!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1755 return page_data, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1756 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1757 log.warning("Items are not ordered, this is not spec compliant")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1758 items = []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1759 # AP Collections are in antichronological order, but we expect chronological in
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1760 # Pubsub, thus we reverse it
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1761 for ap_item in reversed(ap_items):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1762 try:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1763 items.append(await parser(requestor_actor_id, ap_item))
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1764 except (exceptions.DataError, NotImplementedError, error.StanzaError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1765 continue
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1766
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1767 return page_data, items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1768
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1769 async def get_comments_nodes(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1770 self, requestor_actor_id: str, item_id: str, parent_id: Optional[str]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1771 ) -> Tuple[Optional[str], Optional[str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1772 """Get node where this item is and node to use for comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1773
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1774 if config option "comments_max_depth" is set, a common node will be used below the
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1775 given depth
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1776 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1777 @param item_id: ID of the reference item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1778 @param parent_id: ID of the parent item if any (the ID set in "inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1779 @return: a tuple with parent_node_id, comments_node_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1780 - parent_node_id is the ID of the node where reference item must be. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1781 returned when the root node (i.e. not a comments node) must be used.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1782 - comments_node_id: is the ID of the node to use for comments. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1783 returned when no comment node must be used (happens when we have reached
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1784 "comments_max_depth")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1785 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1786 if parent_id is None or not self.comments_max_depth:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1787 return (
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1788 self._m.get_comments_node(parent_id) if parent_id is not None else None,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1789 self._m.get_comments_node(item_id),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1790 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1791 parent_url = parent_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1792 parents = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1793 for __ in range(COMMENTS_MAX_PARENTS):
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1794 parent_item = await self.ap_get(parent_url, requestor_actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1795 parents.insert(0, parent_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1796 parent_url = parent_item.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1797 if parent_url is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1798 break
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1799 parent_limit = self.comments_max_depth - 1
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1800 if len(parents) <= parent_limit:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1801 return (
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1802 self._m.get_comments_node(parents[-1]["id"]),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1803 self._m.get_comments_node(item_id),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1804 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1805 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1806 last_level_item = parents[parent_limit]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1807 return (self._m.get_comments_node(last_level_item["id"]), None)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1808
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1809 async def ap_item_2_mb_data(self, requestor_actor_id: str, ap_item: dict) -> dict:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1810 """Convert AP activity or object to microblog data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1811
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1812 @param actor_id: ID of the actor doing the request.
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1813 @param ap_item: ActivityPub item to convert
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1814 Can be either an activity of an object
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1815 @return: AP Item's Object and microblog data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1816 @raise exceptions.DataError: something is invalid in the AP item
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	1817 @raise NotImplementedError: some AP data is not handled yet
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1818 @raise error.StanzaError: error while contacting the AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1819 """
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1820 is_activity = self.is_activity(ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1821 if is_activity:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1822 ap_object = await self.ap_get_object(requestor_actor_id, ap_item, "object")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1823 if not ap_object:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1824 log.warning(f'No "object" found in AP item {ap_item!r}')
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1825 raise exceptions.DataError
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1826 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1827 ap_object = ap_item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1828 item_id = ap_object.get("id")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1829 if not item_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1830 log.warning(f'No "id" found in AP item: {ap_object!r}')
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1831 raise exceptions.DataError
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1832 mb_data = {"id": item_id, "extra": {}}
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1833
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1834 # content
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1835 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1836 language, content_xhtml = ap_object["contentMap"].popitem()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1837 except (KeyError, AttributeError):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1838 try:
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1839 mb_data["content_xhtml"] = ap_object["content"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1840 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1841 log.warning(f"no content found:\n{ap_object!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1842 raise exceptions.DataError
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1843 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1844 mb_data["language"] = language
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1845 mb_data["content_xhtml"] = content_xhtml
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1846
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1847 mb_data["content"] = await self._t.convert(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1848 mb_data["content_xhtml"],
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1849 self._t.SYNTAX_XHTML,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1850 self._t.SYNTAX_TEXT,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1851 False,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1852 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1853
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1854 if "attachment" in ap_object:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1855 attachments = mb_data["extra"][C.KEY_ATTACHMENTS] = []
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1856 for ap_attachment in ap_object["attachment"]:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1857 try:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1858 url = ap_attachment["url"]
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1859 except KeyError:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1860 log.warning(
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1861 f'"url" missing in AP attachment, ignoring: {ap_attachment}'
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1862 )
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1863 continue
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1864
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1865 if not url.startswith("http"):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1866 log.warning(f"non HTTP URL in attachment, ignoring: {ap_attachment}")
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1867 continue
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1868 attachment = {"url": url}
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1869 for ap_key, key in (
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1870 ("mediaType", "media_type"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1871 # XXX: as weird as it seems, "name" is actually used for description
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1872 # in AP world
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1873 ("name", "desc"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1874 ):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1875 value = ap_attachment.get(ap_key)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1876 if value:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1877 attachment[key] = value
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1878 attachments.append(attachment)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1879
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1880 # author
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1881 if is_activity:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1882 authors = await self.ap_get_actors(requestor_actor_id, ap_item, "actor")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1883 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1884 authors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1885 requestor_actor_id, ap_object, "attributedTo"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1886 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1887 if len(authors) > 1:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1888 # we only keep first item as author
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1889 # TODO: handle multiple actors
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1890 log.warning("multiple actors are not managed")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1891
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1892 account = authors[0]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1893 author_jid = self.get_local_jid_from_account(account).full()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1894
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1895 mb_data["author"] = account.split("@", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1896 mb_data["author_jid"] = author_jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1897
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1898 # published/updated
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1899 for field in ("published", "updated"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1900 value = ap_object.get(field)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1901 if not value and field == "updated":
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1902 value = ap_object.get("published")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1903 if value:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1904 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1905 mb_data[field] = calendar.timegm(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1906 dateutil.parser.parse(str(value)).utctimetuple()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1907 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1908 except dateutil.parser.ParserError as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1909 log.warning(f"Can't parse {field!r} field: {e}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1910
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1911 # repeat
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1912 if "_repeated" in ap_item:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1913 mb_data["extra"]["repeated"] = ap_item["_repeated"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1914
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1915 # comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1916 in_reply_to = ap_object.get("inReplyTo")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1917 __, comments_node = await self.get_comments_nodes(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1918 requestor_actor_id, item_id, in_reply_to
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1919 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1920 if comments_node is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1921 comments_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1922 "service": author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1923 "node": comments_node,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1924 "uri": uri.build_xmpp_uri("pubsub", path=author_jid, node=comments_node),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1925 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1926 mb_data["comments"] = [comments_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1927
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1928 return mb_data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1929
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1930 async def get_reply_to_id_from_xmpp_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1931 self, client: SatXMPPEntity, ap_account: str, parent_item: str, mb_data: dict
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1932 ) -> str:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1933 """Get URL to use for ``inReplyTo`` field in AP item.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1934
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1935 There is currently no way to know the parent service of a comment with XEP-0277.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1936 To work around that, we try to check if we have this item in the cache (we
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1937 should). If there is more that one item with this ID, we first try to find one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1938 with this author_jid. If nothing is found, we use ap_account to build `inReplyTo`.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1939
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1940 @param ap_account: AP account corresponding to the publication author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1941 @param parent_item: ID of the node where the publication this item is replying to
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1942 has been posted
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1943 @param mb_data: microblog data of the publication
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1944 @return: URL to use in ``inReplyTo`` field
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1945 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1946 # FIXME: propose a protoXEP to properly get parent item, node and service
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1947
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1948 found_items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1949 {"profiles": [client.profile], "names": [parent_item]}
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1950 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1951 if not found_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1952 log.warning(f"parent item {parent_item!r} not found in cache")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1953 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1954 elif len(found_items) == 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1955 cached_node = found_items[0].node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1956 parent_ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1957 cached_node.service, cached_node.name
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1958 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1959 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1960 # we found several cached item with given ID, we check if there is one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1961 # corresponding to this author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1962 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1963 author = jid.JID(mb_data["author_jid"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1964 cached_item = next(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1965 i
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1966 for i in found_items
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1967 if jid.JID(i.data["publisher"]).userhostJID() == author
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1968 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1969 except StopIteration:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1970 # no item corresponding to this author, we use ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1971 log.warning(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1972 "Can't find a single cached item for parent item " f"{parent_item!r}"
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1973 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1974 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1975 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1976 cached_node = cached_item.node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1977 parent_ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1978 cached_node.service, cached_node.name
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1979 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1980
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1981 return self.build_apurl(TYPE_ITEM, parent_ap_account, parent_item)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1982
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1983 async def repeated_mb_2_ap_item(self, mb_data: dict) -> dict:
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1984 """Convert repeated blog item to suitable AP Announce activity
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1985
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1986 @param mb_data: microblog metadata of an item repeating an other blog post
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1987 @return: Announce activity linking to the repeated item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1988 """
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1989 repeated = mb_data["extra"]["repeated"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1990 repeater = jid.JID(repeated["by"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1991 repeater_account = await self.get_ap_account_from_jid_and_node(repeater, None)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1992 repeater_id = self.build_apurl(TYPE_ACTOR, repeater_account)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1993 repeated_uri = repeated["uri"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1994
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1995 if not repeated_uri.startswith("xmpp:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1996 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1997 "Only xmpp: URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1998 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1999 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2000 raise NotImplementedError
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2001 parsed_url = uri.parse_xmpp_uri(repeated_uri)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2002 if parsed_url["type"] != "pubsub":
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2003 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2004 "Only pubsub URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2005 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2006 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2007 raise NotImplementedError
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2008 rep_service = jid.JID(parsed_url["path"])
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2009 rep_item = parsed_url["item"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2010 activity_id = self.build_apurl("item", repeater.userhost(), mb_data["id"])
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2011
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2012 if self.is_virtual_jid(rep_service):
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2013 # it's an AP actor linked through this gateway
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2014 # in this case we can simply use the item ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2015 if not rep_item.startswith("https:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2016 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2017 f"Was expecting an HTTPS url as item ID and got {rep_item!r}\n"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2018 f"{mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2019 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2020 announced_uri = rep_item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2021 repeated_account = self._e.unescape(rep_service.user)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2022 else:
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2023 # the repeated item is an XMPP publication, we build the corresponding ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2024 rep_node = parsed_url["node"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2025 repeated_account = await self.get_ap_account_from_jid_and_node(
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2026 rep_service, rep_node
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2027 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2028 announced_uri = self.build_apurl("item", repeated_account, rep_item)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2029
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2030 announce = self.create_activity(
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2031 "Announce", repeater_id, announced_uri, activity_id=activity_id
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2032 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2033 announce["to"] = [NS_AP_PUBLIC]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2034 announce["cc"] = [
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2035 self.build_apurl(TYPE_FOLLOWERS, repeater_account),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2036 await self.get_ap_actor_id_from_account(repeated_account),
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2037 ]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2038 return announce
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2039
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2040 async def mb_data_2_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2041 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2042 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2043 mb_data: dict,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2044 public: bool = True,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2045 is_new: bool = True,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2046 ) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2047 """Convert Libervia Microblog Data to ActivityPub item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2048
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2049 @param mb_data: microblog data (as used in plugin XEP-0277) to convert
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2050 If ``public`` is True, ``service`` and ``node`` keys must be set.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2051 If ``published`` is not set, current datetime will be used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2052 @param public: True if the message is not a private/direct one
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2053 if True, the AP Item will be marked as public, and AP followers of target AP
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2054 account (which retrieve from ``service``) will be put in ``cc``.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2055 ``inReplyTo`` will also be set if suitable
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2056 if False, no destinee will be set (i.e., no ``to`` or ``cc`` or public flag).
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2057 This is usually used for direct messages.
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2058 @param is_new: if True, the item is a new one (no instance has been found in
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2059 cache).
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2060 If True, a "Create" activity will be generated, otherwise an "Update" one will
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2061 be.
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2062 @return: Activity item
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2063 """
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2064 extra = mb_data.get("extra", {})
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2065 if "repeated" in extra:
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2066 return await self.repeated_mb_2_ap_item(mb_data)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2067 if not mb_data.get("id"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2068 mb_data["id"] = shortuuid.uuid()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2069 if not mb_data.get("author_jid"):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2070 mb_data["author_jid"] = client.jid.userhost()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2071 ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2072 jid.JID(mb_data["author_jid"]), None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2073 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2074 url_actor = self.build_apurl(TYPE_ACTOR, ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2075 url_item = self.build_apurl(TYPE_ITEM, ap_account, mb_data["id"])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2076 ap_object = {
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2077 "id": url_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2078 "type": "Note",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2079 "published": utils.xmpp_date(mb_data.get("published")),
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2080 "attributedTo": url_actor,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2081 "content": mb_data.get("content_xhtml") or mb_data["content"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2082 }
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2083
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2084 language = mb_data.get("language")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2085 if language:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2086 ap_object["contentMap"] = {language: ap_object["content"]}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2087
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2088 attachments = extra.get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2089 if attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2090 ap_attachments = ap_object["attachment"] = []
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2091 for attachment in attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2092 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2093 url = next(s["url"] for s in attachment["sources"] if "url" in s)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2094 except (StopIteration, KeyError):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2095 log.warning(f"Ignoring attachment without URL: {attachment}")
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2096 continue
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2097 ap_attachment = {"url": url}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2098 for key, ap_key in (
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2099 ("media_type", "mediaType"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2100 # XXX: yes "name", cf. [ap_item_2_mb_data]
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2101 ("desc", "name"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2102 ):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2103 value = attachment.get(key)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2104 if value:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2105 ap_attachment[ap_key] = value
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2106 ap_attachments.append(ap_attachment)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2107
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2108 if public:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2109 ap_object["to"] = [NS_AP_PUBLIC]
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2110 if self.auto_mentions:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2111 for m in RE_MENTION.finditer(ap_object["content"]):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2112 mention = m.group()
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2113 mentioned = mention[1:]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2114 __, m_host = mentioned.split("@", 1)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2115 if m_host in (self.public_url, self.client.jid.host):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2116 # we ignore mention of local users, they should be sent as XMPP
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2117 # references
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2118 continue
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2119 try:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2120 mentioned_id = await self.get_ap_actor_id_from_account(mentioned)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2121 except Exception as e:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2122 log.warning(f"Can't add mention to {mentioned!r}: {e}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2123 else:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2124 ap_object["to"].append(mentioned_id)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2125 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2126 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2127 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2128 "href": mentioned_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2129 "name": mention,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2130 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2131 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2132 try:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2133 node = mb_data["node"]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2134 service = jid.JID(mb_data["service"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2135 except KeyError:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2136 # node and service must always be specified when this method is used
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2137 raise exceptions.InternalError("node or service is missing in mb_data")
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2138 try:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2139 target_ap_account = await self.get_ap_account_from_jid_and_node(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2140 service, node
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2141 )
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2142 except MissingLocalPartError:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2143 # we have no local part, this is probably a comment
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2144 try:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2145 service = jid.JID(mb_data["author_jid"])
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2146 if not mb_data["author_jid_verified"]:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2147 raise KeyError
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2148 except KeyError:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2149 raise exceptions.DataError(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2150 "Can't identify post author. Be sure to use a pubsub service "
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2151 "setting the 'publisher' attribute."
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2152 )
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2153 else:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2154 target_ap_account = await self.get_ap_account_from_jid_and_node(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2155 service, node
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2156 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2157 if self.is_virtual_jid(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2158 # service is a proxy JID for AP account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2159 actor_data = await self.get_ap_actor_data_from_account(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2160 url_actor, target_ap_account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2161 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2162 followers = actor_data.get("followers")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2163 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2164 # service is a real XMPP entity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2165 followers = self.build_apurl(TYPE_FOLLOWERS, target_ap_account)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2166 if followers:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2167 ap_object["cc"] = [followers]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2168 if self._m.is_comment_node(node):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2169 parent_item = self._m.get_parent_item(node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2170 if self.is_virtual_jid(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2171 # the publication is on a virtual node (i.e. an XMPP node managed by
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2172 # this gateway and linking to an ActivityPub actor)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2173 ap_object["inReplyTo"] = parent_item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2174 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2175 # the publication is from a followed real XMPP node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2176 ap_object["inReplyTo"] = await self.get_reply_to_id_from_xmpp_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2177 client, ap_account, parent_item, mb_data
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2178 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2179
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2180 return self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2181 "Create" if is_new else "Update", url_actor, ap_object, activity_id=url_item
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	2182 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2183
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2184 async def publish_message(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2185 self, client: SatXMPPEntity, mess_data: dict, service: jid.JID
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2186 ) -> None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2187 """Send an AP message
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2188
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2189 .. note::
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2190
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2191 This is a temporary method used for development only
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2192
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2193 @param mess_data: message data. Following keys must be set:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2194
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2195 ``node``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2196 identifier of message which is being replied (this will
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2197 correspond to pubsub node in the future)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2198
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2199 ``content_xhtml`` or ``content``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2200 message body (respectively in XHTML or plain text)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2201
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2202 @param service: JID corresponding to the AP actor.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2203 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2204 if not service.user:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2205 raise ValueError("service must have a local part")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2206 account = self._e.unescape(service.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2207 ap_actor_data = await self.get_ap_actor_data_from_account(account)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2208
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2209 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2210 inbox_url = ap_actor_data["endpoints"]["sharedInbox"]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2211 except KeyError:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2212 raise exceptions.DataError("Can't get ActivityPub actor inbox")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2213
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2214 item_data = await self.mb_data_2_ap_item(client, mess_data)
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	2215 url_actor = item_data["actor"]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2216 await self.ap_post(inbox_url, url_actor, item_data)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2217
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2218 async def ap_delete_item(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2219 self, jid_: jid.JID, node: Optional[str], item_id: str, public: bool = True
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2220 ) -> Tuple[str, Dict[str, Any]]:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2221 """Build activity to delete an AP item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2222
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2223 @param jid_: JID of the entity deleting an item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2224 @param node: node where the item is deleted
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2225 None if it's microblog or a message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2226 @param item_id: ID of the item to delete
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2227 it's the Pubsub ID or message's origin ID
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2228 @param public: if True, the activity will be addressed to public namespace
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2229 @return: actor_id of the entity deleting the item, activity to send
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2230 """
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2231 if node is None:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2232 node = self._m.namespace
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2233
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2234 author_account = await self.get_ap_account_from_jid_and_node(jid_, node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2235 author_actor_id = self.build_apurl(TYPE_ACTOR, author_account)
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2236
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2237 items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2238 {"profiles": [self.client.profile], "services": [jid_], "names": [item_id]}
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2239 )
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2240 if not items:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2241 log.warning(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2242 f"Deleting an unknown item at service {jid_}, node {node} and id "
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2243 f"{item_id}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2244 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2245 else:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2246 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2247 mb_data = await self._m.item_2_mb_data(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2248 self.client, items[0].data, jid_, node
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2249 )
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2250 if "repeated" in mb_data["extra"]:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2251 # we are deleting a repeated item, we must translate this to an
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2252 # "Undo" of the "Announce" activity instead of a "Delete" one
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2253 announce = await self.repeated_mb_2_ap_item(mb_data)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2254 undo = self.create_activity("Undo", author_actor_id, announce)
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2255 return author_actor_id, undo
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2256 except Exception as e:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2257 log.debug(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2258 f"Can't parse item, maybe it's not a blog item: {e}\n"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2259 f"{items[0].toXml()}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2260 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2261
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2262 url_item = self.build_apurl(TYPE_ITEM, author_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2263 ap_item = self.create_activity(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2264 "Delete", author_actor_id, {"id": url_item, "type": TYPE_TOMBSTONE}
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2265 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2266 if public:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2267 ap_item["to"] = [NS_AP_PUBLIC]
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2268 return author_actor_id, ap_item
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2269
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2270 def _message_received_trigger(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2271 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2272 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2273 message_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2274 post_treat: defer.Deferred,
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2275 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2276 """add the gateway workflow on post treatment"""
3979 996e0f84935e component AP gateway: log at debug level instead of warning when no client is set Goffi <goffi@goffi.org> parents: 3977 diff changeset	2277 if self.client is None:
996e0f84935e component AP gateway: log at debug level instead of warning when no client is set Goffi <goffi@goffi.org> parents: 3977 diff changeset	2278 log.debug(f"no client set, ignoring message: {message_elt.toXml()}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2279 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2280 post_treat.addCallback(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2281 lambda mess_data: defer.ensureDeferred(self.on_message(client, mess_data))
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2282 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2283 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2284
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2285 async def on_message(self, client: SatXMPPEntity, mess_data: dict) -> dict:
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2286 """Called once message has been parsed
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2287
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2288 this method handle the conversion to AP items and posting
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2289 """
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2290 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2291 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2292 if mess_data["type"] not in ("chat", "normal"):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2293 log.warning(f"ignoring message with unexpected type: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2294 return mess_data
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2295 if not self.is_local(mess_data["from"]):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2296 log.warning(f"ignoring non local message: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2297 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2298 if not mess_data["to"].user:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2299 log.warning(f"ignoring message addressed to gateway itself: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2300 return mess_data
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2301 requestor_actor_id = self.build_apurl(TYPE_ACTOR, mess_data["from"].userhost())
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2302
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2303 actor_account = self._e.unescape(mess_data["to"].user)
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2304 try:
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2305 actor_id = await self.get_ap_actor_id_from_account(actor_account)
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2306 except Exception as e:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2307 log.warning(f"Can't retrieve data on actor {actor_account}: {e}")
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2308 # TODO: send an error <message>
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2309 return mess_data
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2310 inbox = await self.get_ap_inbox_from_id(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2311 requestor_actor_id, actor_id, use_shared=False
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2312 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2313
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2314 try:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2315 language, message = next(iter(mess_data["message"].items()))
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2316 except (KeyError, StopIteration):
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2317 log.warning(f"ignoring empty message: {mess_data}")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2318 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2319
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2320 mb_data = {
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2321 "content": message,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2322 }
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2323 if language:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2324 mb_data["language"] = language
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2325 origin_id = mess_data["extra"].get("origin_id")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2326 if origin_id:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2327 # we need to use origin ID when present to be able to retract the message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2328 mb_data["id"] = origin_id
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2329 attachments = mess_data["extra"].get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2330 if attachments:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2331 mb_data["extra"] = {C.KEY_ATTACHMENTS: attachments}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2332
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2333 client = self.client.get_virtual_client(mess_data["from"])
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2334 ap_item = await self.mb_data_2_ap_item(client, mb_data, public=False)
4017 4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2335 ap_object = ap_item["object"]
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2336 ap_object["to"] = ap_item["to"] = [actor_id]
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2337 # we add a mention to direct message, otherwise peer is not notified in some AP
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2338 # implementations (notably Mastodon), and the message may be missed easily.
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2339 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2340 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2341 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2342 "href": actor_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2343 "name": f"@{actor_account}",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2344 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2345 )
4017 4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2346
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2347 try:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2348 await self.ap_post(inbox, ap_item["actor"], ap_item)
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2349 except Exception as e:
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2350 # TODO: send an error <message>
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2351 log.warning(f"Can't send message to {inbox}: {e}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2352 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2353
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2354 async def _on_message_retract(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2355 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2356 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2357 message_elt: domish.Element,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2358 retract_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2359 history: History,
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2360 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2361 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2362 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2363 from_jid = jid.JID(message_elt["from"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2364 if not self.is_local(from_jid):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2365 log.debug(f"ignoring retract request from non local jid {from_jid}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2366 return False
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2367 requestor_actor_id = self.build_apurl(TYPE_ACTOR, from_jid.userhost())
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2368 to_jid = jid.JID(message_elt["to"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2369 if to_jid.host != self.client.jid.full() or not to_jid.user:
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2370 # to_jid should be a virtual JID from this gateway
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2371 raise exceptions.InternalError(f"Invalid destinee's JID: {to_jid.full()}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2372 ap_account = self._e.unescape(to_jid.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2373 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2374 inbox = await self.get_ap_inbox_from_id(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2375 requestor_actor_id, actor_id, use_shared=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2376 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2377 url_actor, ap_item = await self.ap_delete_item(
4166 a1f7040b5a15 plugin XEP-0424: message retraction update: Goffi <goffi@goffi.org> parents: 4109 diff changeset	2378 from_jid.userhostJID(), None, retract_elt["id"], public=False
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2379 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2380 resp = await self.ap_post(inbox, url_actor, ap_item)
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2381 return False
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2382
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2383 async def _on_reference_received(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2384 self,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2385 client: SatXMPPEntity,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2386 message_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2387 reference_data: Dict[str, Union[str, int]],
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2388 ) -> bool:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2389 parsed_uri: dict = reference_data.get("parsed_uri")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2390 if not parsed_uri:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2391 log.warning(f"no parsed URI available in reference {reference_data}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2392 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2393
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2394 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2395 mentioned = jid.JID(parsed_uri["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2396 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2397 log.warning(f"invalid target: {reference_data['uri']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2398 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2399
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2400 if mentioned.host != self.client.jid.full() or not mentioned.user:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2401 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2402 f"ignoring mentioned user {mentioned}, it's not a JID mapping an AP "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2403 "account"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2404 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2405 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2406
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2407 ap_account = self._e.unescape(mentioned.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2408 actor_id = await self.get_ap_actor_id_from_account(ap_account)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2409
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2410 parsed_anchor: dict = reference_data.get("parsed_anchor")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2411 if not parsed_anchor:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2412 log.warning(f"no XMPP anchor, ignoring reference {reference_data!r}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2413 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2414
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2415 if parsed_anchor["type"] != "pubsub":
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2416 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2417 f"ignoring reference with non pubsub anchor, this is not supported: "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2418 "{reference_data!r}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2419 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2420 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2421
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2422 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2423 pubsub_service = jid.JID(parsed_anchor["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2424 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2425 log.warning(f"invalid anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2426 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2427 pubsub_node = parsed_anchor.get("node")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2428 if not pubsub_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2429 log.warning(f"missing pubsub node in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2430 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2431 pubsub_item = parsed_anchor.get("item")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2432 if not pubsub_item:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2433 log.warning(f"missing pubsub item in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2434 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2435
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2436 cached_node = await self.host.memory.storage.get_pubsub_node(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2437 client, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2438 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2439 if not cached_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2440 log.warning(f"Anchored node not found in cache: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2441 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2442
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2443 cached_items, __ = await self.host.memory.storage.get_items(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2444 cached_node, item_ids=[pubsub_item]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2445 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2446 if not cached_items:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2447 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2448 f"Anchored pubsub item not found in cache: {reference_data['anchor']}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2449 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2450 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2451
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2452 cached_item = cached_items[0]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2453
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2454 mb_data = await self._m.item_2_mb_data(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2455 client, cached_item.data, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2456 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2457 ap_item = await self.mb_data_2_ap_item(client, mb_data)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2458 ap_object = ap_item["object"]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2459 ap_object["to"] = [actor_id]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2460 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2461 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2462 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2463 "href": actor_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2464 "name": ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2465 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2466 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2467
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2468 requestor_actor_id = ap_item["actor"]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2469 inbox = await self.get_ap_inbox_from_id(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2470 requestor_actor_id, actor_id, use_shared=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2471 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2472
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2473 await self.ap_post(inbox, requestor_actor_id, ap_item)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2474
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2475 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2476
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2477 async def new_reply_to_xmpp_item(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2478 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2479 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2480 ap_item: dict,
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2481 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2482 mentions: List[Dict[str, str]],
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2483 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2484 """We got an AP item which is a reply to an XMPP item"""
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2485 in_reply_to = ap_item["inReplyTo"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2486 url_type, url_args = self.parse_apurl(in_reply_to)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2487 if url_type != "item":
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2488 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2489 "Ignoring AP item replying to an XMPP item with an unexpected URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2490 f"type({url_type!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2491 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2492 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2493 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2494 parent_item_account, parent_item_id = url_args[0], "/".join(url_args[1:])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2495 except (IndexError, ValueError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2496 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2497 "Ignoring AP item replying to an XMPP item with invalid inReplyTo URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2498 f"({in_reply_to!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2499 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2500 return
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2501 parent_item_service, parent_item_node = await self.get_jid_and_node(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2502 parent_item_account
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2503 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2504 if parent_item_node is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2505 parent_item_node = self._m.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2506 items, __ = await self._p.get_items(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2507 client, parent_item_service, parent_item_node, item_ids=[parent_item_id]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2508 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2509 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2510 parent_item_elt = items[0]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2511 except IndexError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2512 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2513 f"Can't find parent item at {parent_item_service} (node "
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2514 f"{parent_item_node!r})\n{pformat(ap_item)}"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2515 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2516 return
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2517 parent_item_parsed = await self._m.item_2_mb_data(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2518 client, parent_item_elt, parent_item_service, parent_item_node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2519 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2520 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2521 comment_service = jid.JID(parent_item_parsed["comments"][0]["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2522 comment_node = parent_item_parsed["comments"][0]["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2523 except (KeyError, IndexError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2524 # we don't have a comment node set for this item
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	2525 from libervia.backend.tools.xml_tools import pp_elt
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2526
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2527 log.info(f"{pp_elt(parent_item_elt.toXml())}")
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2528 raise NotImplementedError()
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2529 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2530 requestor_actor_id = self.build_apurl(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2531 TYPE_ACTOR,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2532 await self.get_ap_account_from_jid_and_node(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2533 comment_service, comment_node
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2534 ),
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2535 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2536 __, item_elt = await self.ap_item_2_mb_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2537 requestor_actor_id, ap_item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2538 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2539 await self._p.publish(client, comment_service, comment_node, [item_elt])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2540 await self.notify_mentions(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2541 targets, mentions, comment_service, comment_node, item_elt["id"]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2542 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2543
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2544 def get_ap_item_targets(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2545 self, item: Dict[str, Any]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2546 ) -> Tuple[bool, Dict[str, Set[str]], List[Dict[str, str]]]:
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2547 """Retrieve targets of an AP item, and indicate if it's a public one
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2548
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2549 @param item: AP object payload
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2550 @return: Are returned:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2551 - is_public flag, indicating if the item is world-readable
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2552 - a dict mapping target type to targets
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2553 """
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2554 targets: Dict[str, Set[str]] = {}
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2555 is_public = False
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2556 # TODO: handle "audience"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2557 for key in ("to", "bto", "cc", "bcc"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2558 values = item.get(key)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2559 if not values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2560 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2561 if isinstance(values, str):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2562 values = [values]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2563 for value in values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2564 if value in PUBLIC_TUPLE:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2565 is_public = True
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2566 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2567 if not value:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2568 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2569 if not self.is_local_url(value):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2570 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2571 target_type = self.parse_apurl(value)[0]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2572 if target_type != TYPE_ACTOR:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2573 log.debug(f"ignoring non actor type as a target: {value}")
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2574 else:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2575 targets.setdefault(target_type, set()).add(value)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2576
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2577 mentions = []
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2578 tags = item.get("tag")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2579 if tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2580 for tag in tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2581 if tag.get("type") != TYPE_MENTION:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2582 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2583 href = tag.get("href")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2584 if not href:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2585 log.warning('Missing "href" field from mention object: {tag!r}')
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2586 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2587 if not self.is_local_url(href):
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2588 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2589 uri_type = self.parse_apurl(href)[0]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2590 if uri_type != TYPE_ACTOR:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2591 log.debug(f"ignoring non actor URI as a target: {href}")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2592 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2593 mention = {"uri": href}
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2594 mentions.append(mention)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2595 name = tag.get("name")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2596 if name:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2597 mention["content"] = name
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2598
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2599 return is_public, targets, mentions
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2600
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2601 async def new_ap_item(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2602 self,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2603 client: SatXMPPEntity,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2604 destinee: Optional[jid.JID],
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2605 node: str,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2606 item: dict,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2607 ) -> None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2608 """Analyse, cache and send notification for received AP item
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2609
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2610 @param destinee: jid of the destinee,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2611 @param node: XMPP pubsub node
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2612 @param item: AP object payload
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2613 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2614 is_public, targets, mentions = self.get_ap_item_targets(item)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2615 if not is_public and targets.keys() == {TYPE_ACTOR}:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2616 # this is a direct message
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2617 await self.handle_message_ap_item(client, targets, mentions, destinee, item)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2618 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2619 await self.handle_pubsub_ap_item(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2620 client, targets, mentions, destinee, node, item, is_public
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2621 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2622
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2623 def get_requestor_actor_id_from_targets(self, targets: set[str]) -> str:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2624 """Find local actor to use as requestor_actor_id from request targets.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2625
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2626 A local actor must be used to sign HTTP request, notably HTTP GET request for AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2627 instance checking signature, such as Mastodon when set in "secure mode".
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2628
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2629 This method check a set of targets and use the first local one.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2630
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2631 If none match, a generic local actor is used.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2632
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2633 @param targets: set of actor IDs to which the current request is sent.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2634 @return: local actor ID to use as requestor_actor_id.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2635 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2636 try:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2637 return next(t for t in targets if self.is_local_url(t))
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2638 except StopIteration:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2639 log.warning(f"Can't find local target to use as requestor ID: {targets!r}")
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2640 return self.build_apurl(TYPE_ACTOR, f"libervia@{self.public_url}")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2641
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2642 async def handle_message_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2643 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2644 client: SatXMPPEntity,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2645 targets: dict[str, Set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2646 mentions: list[Dict[str, str]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2647 destinee: jid.JID \| None,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2648 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2649 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2650 """Parse and deliver direct AP items translating to XMPP messages
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2651
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2652 @param targets: actors where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2653 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2654 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2655 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2656 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2657 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2658 targets_jids = {
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2659 await self.get_jid_from_id(requestor_actor_id, url) for url in targets_urls
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2660 }
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2661 if destinee is not None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2662 targets_jids.add(destinee)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2663 mb_data = await self.ap_item_2_mb_data(requestor_actor_id, item)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2664 extra = {"origin_id": mb_data["id"]}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2665 attachments = mb_data["extra"].get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2666 if attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2667 extra[C.KEY_ATTACHMENTS] = attachments
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2668
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2669 defer_l = []
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2670 for target_jid in targets_jids:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2671 defer_l.append(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2672 client.sendMessage(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2673 target_jid,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2674 {"": mb_data.get("content", "")},
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2675 mb_data.get("title"),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2676 extra=extra,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2677 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2678 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2679 await defer.DeferredList(defer_l)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2680
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2681 async def notify_mentions(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2682 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2683 targets: dict[str, set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2684 mentions: list[dict[str, str]],
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2685 service: jid.JID,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2686 node: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2687 item_id: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2688 ) -> None:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2689 """Send mention notifications to recipients and mentioned entities
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2690
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2691 XEP-0372 (References) is used.
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2692
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2693 Mentions are also sent to recipients as they are primary audience (see
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2694 https://www.w3.org/TR/activitystreams-vocabulary/#microsyntaxes).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2695
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2696 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2697 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2698 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2699 anchor = uri.build_xmpp_uri(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2700 "pubsub", path=service.full(), node=node, item=item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2701 )
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2702 seen = set()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2703 # we start with explicit mentions because mentions' content will be used in the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2704 # future to fill "begin" and "end" reference attributes (we can't do it at the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2705 # moment as there is no way to specify the XML element to use in the blog item).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2706 for mention in mentions:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2707 mentioned_jid = await self.get_jid_from_id(requestor_actor_id, mention["uri"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2708 self._refs.send_reference(self.client, to_jid=mentioned_jid, anchor=anchor)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2709 seen.add(mentioned_jid)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2710
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2711 remaining = {
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2712 await self.get_jid_from_id(requestor_actor_id, t)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2713 for t_set in targets.values()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2714 for t in t_set
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2715 } - seen
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2716 for target in remaining:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2717 self._refs.send_reference(self.client, to_jid=target, anchor=anchor)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2718
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2719 async def handle_pubsub_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2720 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2721 client: SatXMPPEntity,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2722 targets: dict[str, set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2723 mentions: list[dict[str, str]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2724 destinee: jid.JID \| None,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2725 node: str,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2726 item: dict,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2727 public: bool,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2728 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2729 """Analyse, cache and deliver AP items translating to Pubsub
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2730
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2731 @param targets: actors/collections where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2732 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2733 @param node: XMPP pubsub node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2734 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2735 @param public: True if the item is public
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2736 """
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2737 # XXX: "public" is not used for now
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2738 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2739 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2740 service = client.jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2741 in_reply_to = item.get("inReplyTo")
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2742
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2743 if in_reply_to and isinstance(in_reply_to, list):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2744 in_reply_to = in_reply_to[0]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2745 if in_reply_to and isinstance(in_reply_to, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2746 if self.is_local_url(in_reply_to):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2747 # this is a reply to an XMPP item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2748 await self.new_reply_to_xmpp_item(client, item, targets, mentions)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2749 return
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2750
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2751 # this item is a reply to an AP item, we use or create a corresponding node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2752 # for comments
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2753 parent_node, __ = await self.get_comments_nodes(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2754 requestor_actor_id, item["id"], in_reply_to
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2755 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2756 node = parent_node or node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2757 cached_node = await self.host.memory.storage.get_pubsub_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2758 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2759 service,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2760 node,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2761 with_subscriptions=True,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2762 create=True,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2763 create_kwargs={"subscribed": True},
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2764 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2765 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2766 # it is a root item (i.e. not a reply to an other item)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2767 create = node == self._events.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2768 cached_node = await self.host.memory.storage.get_pubsub_node(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2769 client, service, node, with_subscriptions=True, create=create
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2770 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2771 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2772 log.warning(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2773 f"Received item in unknown node {node!r} at {service}. This may be "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2774 f"due to a cache purge. We synchronise the node\n{item}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2775 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2776 return
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2777 if item.get("type") == TYPE_EVENT:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2778 data, item_elt = await self.ap_events.ap_item_2_event_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2779 requestor_actor_id, item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2780 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2781 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2782 data, item_elt = await self.ap_item_2_mb_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2783 requestor_actor_id, item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2784 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2785 await self.host.memory.storage.cache_pubsub_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2786 client, cached_node, [item_elt], [data]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2787 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2788
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2789 for subscription in cached_node.subscriptions:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2790 if subscription.state != SubscriptionState.SUBSCRIBED:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2791 continue
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2792 self.pubsub_service.notifyPublish(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2793 service, node, [(subscription.subscriber, None, [item_elt])]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2794 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2795
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2796 await self.notify_mentions(targets, mentions, service, node, item_elt["id"])
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2797
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2798 async def new_ap_delete_item(
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2799 self,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2800 client: SatXMPPEntity,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2801 destinee: Optional[jid.JID],
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2802 node: str,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2803 item: dict,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2804 ) -> None:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2805 """Analyse, cache and send notification for received AP item
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2806
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2807 @param destinee: jid of the destinee,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2808 @param node: XMPP pubsub node
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2809 @param activity: parent AP activity
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2810 @param item: AP object payload
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	2811 only the "id" field is used
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2812 """
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2813 item_id = item.get("id")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2814 if not item_id:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2815 raise exceptions.DataError('"id" attribute is missing in item')
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2816 if not item_id.startswith("http"):
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2817 raise exceptions.DataError(f"invalid id: {item_id!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2818 if self.is_local_url(item_id):
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2819 raise ValueError("Local IDs should not be used")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2820
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2821 # we have no way to know if a deleted item is a direct one (thus a message) or one
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2822 # converted to pubsub. We check if the id is in message history to decide what to
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2823 # do.
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2824 history = await self.host.memory.storage.get(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2825 client,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2826 History,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2827 History.origin_id,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2828 item_id,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2829 (History.messages, History.subjects),
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2830 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2831
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2832 if history is not None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2833 # it's a direct message
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2834 if history.source_jid != client.jid:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2835 log.warning(
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	2836 f"retraction received from an entity ''{client.jid}) which is "
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2837 f"not the original sender of the message ({history.source_jid}), "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2838 "hack attemps?"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2839 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2840 raise exceptions.PermissionError("forbidden")
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2841
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2842 await self._r.retract_by_history(client, history)
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2843 else:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2844 # no history in cache with this ID, it's probably a pubsub item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2845 cached_node = await self.host.memory.storage.get_pubsub_node(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2846 client, client.jid, node, with_subscriptions=True
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2847 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2848 if cached_node is None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2849 log.warning(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2850 f"Received an item retract for node {node!r} at {client.jid} "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2851 "which is not cached"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2852 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2853 raise exceptions.NotFound
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2854 await self.host.memory.storage.delete_pubsub_items(cached_node, [item_id])
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2855 # notifyRetract is expecting domish.Element instances
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2856 item_elt = domish.Element((None, "item"))
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2857 item_elt["id"] = item_id
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2858 for subscription in cached_node.subscriptions:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2859 if subscription.state != SubscriptionState.SUBSCRIBED:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2860 continue
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2861 self.pubsub_service.notifyRetract(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2862 client.jid, node, [(subscription.subscriber, None, [item_elt])]
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2863 )

Mercurial > libervia-backend

annotate libervia/backend/plugins/plugin_comp_ap_gateway/__init__.py @ 4314:6a70fcd93a7a

annotate libervia/backend/plugins/plugin_comp_ap_gateway/init.py @ 4314:6a70fcd93a7a