libervia-backend: libervia/backend/plugins/plugin_comp_ap_gateway/__init_

annotate libervia/backend/plugins/plugin_comp_ap_gateway/init.py @ 4306:94e0968987cd

plugin XEP-0033: code modernisation, improve delivery, data validation: - Code has been rewritten using Pydantic models and `async` coroutines for data validation and cleaner element parsing/generation. - Delivery has been completely rewritten. It now works even if server doesn't support multicast, and send to local multicast service first. Delivering to local multicast service first is due to bad support of XEP-0033 in server (notably Prosody which has an incomplete implementation), and the current impossibility to detect if a sub-domain service handles fully multicast or only for local domains. This is a workaround to have a good balance between backward compatilibity and use of bandwith, and to make it work with the incoming email gateway implementation (the gateway will only deliver to entities of its own domain). - disco feature checking now uses `async` corountines. `host` implementation still use Deferred return values for compatibility with legacy code. rel 450

author	Goffi <goffi@goffi.org>
date	Thu, 26 Sep 2024 16:12:01 +0200
parents	0f953ce5f0a8
children

rev	line source
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia ActivityPub Gateway
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 import base64
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	20 import calendar
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	21 import hashlib
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	22 import json
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	23 from pathlib import Path
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	24 from pprint import pformat
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	25 import re
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	26 from typing import (
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	27 Any,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	28 Awaitable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	29 Callable,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	30 Dict,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	31 List,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	32 Optional,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	33 Set,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	34 Tuple,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	35 Union,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	36 overload,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	37 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	38 from urllib import parse
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	39
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	40 from cryptography.exceptions import InvalidSignature
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	41 from cryptography.hazmat.primitives import serialization
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 from cryptography.hazmat.primitives import hashes
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	43 from cryptography.hazmat.primitives.asymmetric import rsa
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	44 from cryptography.hazmat.primitives.asymmetric import padding
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	45 import dateutil
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	46 import shortuuid
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	47 import treq
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 from treq.response import _Response as TReqResponse
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	49 from twisted.internet import defer, reactor, threads
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	50 from twisted.web import http
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	51 from twisted.words.protocols.jabber import error, jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	52 from twisted.words.xish import domish
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	53 from wokkel import pubsub, rsm
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	54
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	55 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	56 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	57 from libervia.backend.core.core_types import SatXMPPEntity
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	58 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	59 from libervia.backend.core.log import getLogger
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	60 from libervia.backend.memory import persistent
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	61 from libervia.backend.memory.sqla_mapping import History, SubscriptionState
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	62 from libervia.backend.tools import utils
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	63 from libervia.backend.tools.common import data_format, date_utils, tls, uri
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	64 from libervia.backend.tools.common.async_utils import async_lru
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	65
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	66 from .ad_hoc import APAdHocService
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	67 from .events import APEvents
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	68 from .constants import (
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	69 ACTIVITY_OBJECT_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	70 ACTIVITY_TARGET_MANDATORY,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	71 ACTIVITY_TYPES,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	72 ACTIVITY_TYPES_LOWER,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	73 COMMENTS_MAX_PARENTS,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	74 CONF_SECTION,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	75 IMPORT_NAME,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	76 LRU_MAX_SIZE,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	77 MEDIA_TYPE_AP,
3846 cc13efdd8360 component AP gateway: return item when `item` URL is used: Goffi <goffi@goffi.org> parents: 3845 diff changeset	78 NS_AP,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	79 NS_AP_PUBLIC,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	80 PUBLIC_TUPLE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	81 TYPE_ACTOR,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	82 TYPE_EVENT,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	83 TYPE_FOLLOWERS,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	84 TYPE_ITEM,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	85 TYPE_LIKE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	86 TYPE_MENTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	87 TYPE_REACTION,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	88 TYPE_TOMBSTONE,
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	89 TYPE_JOIN,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	90 TYPE_LEAVE,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	91 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	92 from .http_server import HTTPServer
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	93 from .pubsub_service import APPubsubService
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	94 from .regex import RE_MENTION
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	95
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	96
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	97 log = getLogger(__name__)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	98
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	99 IMPORT_NAME = "ap-gateway"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	100
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	101 PLUGIN_INFO = {
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	102 C.PI_NAME: "ActivityPub Gateway component",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	103 C.PI_IMPORT_NAME: IMPORT_NAME,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	104 C.PI_MODES: [C.PLUG_MODE_COMPONENT],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	105 C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	106 C.PI_PROTOCOLS: [],
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	107 C.PI_DEPENDENCIES: [
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	108 "XEP-0050",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	109 "XEP-0054",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	110 "XEP-0060",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	111 "XEP-0084",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	112 "XEP-0106",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	113 "XEP-0277",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	114 "XEP-0292",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	115 "XEP-0329",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	116 "XEP-0372",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	117 "XEP-0424",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	118 "XEP-0465",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	119 "XEP-0470",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	120 "XEP-0447",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	121 "XEP-0471",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	122 "PUBSUB_CACHE",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	123 "TEXT_SYNTAXES",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	124 "IDENTITY",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	125 ],
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	126 C.PI_RECOMMENDATIONS: [],
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	127 C.PI_MAIN: "APGateway",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	128 C.PI_HANDLER: C.BOOL_TRUE,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	129 C.PI_DESCRIPTION: _(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 "Gateway for bidirectional communication between XMPP and ActivityPub."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	131 ),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	132 }
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	133
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	134 HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	135 RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	136 RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	137 RE_ALLOWED_UNQUOTED = re.compile(r"^[a-zA-Z0-9_-]+$")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	138
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	139
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	140 class MissingLocalPartError(Exception):
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	141 """JID local part is missing"""
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	142
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	143
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	144 class APGateway:
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	145 IMPORT_NAME = IMPORT_NAME
3847 aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	146 # show data send or received through HTTP, used for debugging
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	147 # 1: log POST objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	148 # 2: log POST and GET objects
aaa4e7815ba8 component AP gateway: new `verbose` attribute in AP gateway to activate debug logs: Goffi <goffi@goffi.org> parents: 3846 diff changeset	149 # 3: log POST and GET objects with HTTP headers for GET requests
4025 44abce96ac6b component AP gateway: revert forgotten verbose mode Goffi <goffi@goffi.org> parents: 4023 diff changeset	150 verbose = 0
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	151
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	152 def __init__(self, host):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	153 self.host = host
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	154 self.initialised = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	155 self.client = None
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	156 self.http_sign_get = True
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	157 self._p = host.plugins["XEP-0060"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	158 self._a = host.plugins["XEP-0084"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	159 self._e = host.plugins["XEP-0106"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	160 self._m = host.plugins["XEP-0277"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	161 self._v = host.plugins["XEP-0292"]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	162 self._refs = host.plugins["XEP-0372"]
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	163 self._r = host.plugins["XEP-0424"]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	164 self._sfs = host.plugins["XEP-0447"]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	165 self._pps = host.plugins["XEP-0465"]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	166 self._pa = host.plugins["XEP-0470"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	167 self._c = host.plugins["PUBSUB_CACHE"]
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	168 self._t = host.plugins["TEXT_SYNTAXES"]
6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	169 self._i = host.plugins["IDENTITY"]
4027 26c3e1bc7fb7 plugin XEP-0471: renamed "events" plugin to XEP-0471 now that there is a XEP Goffi <goffi@goffi.org> parents: 4025 diff changeset	170 self._events = host.plugins["XEP-0471"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	171 self._p.add_managed_node(
3850 4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	172 "",
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	173 items_cb=self._items_received,
3850 4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	174 # we want to be sure that the callbacks are launched before pubsub cache's
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	175 # one, as we need to inspect items before they are actually removed from cache
4479f6074bc8 component AP gateway: use the new `priority` argument of `addManagedNode`: Goffi <goffi@goffi.org> parents: 3847 diff changeset	176 # or updated
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	177 priority=1000,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	178 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	179 self.pubsub_service = APPubsubService(self)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	180 self.ad_hoc = APAdHocService(self)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	181 self.ap_events = APEvents(self)
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	182 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	183 "message_received", self, self._message_received_trigger, priority=-1000
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	184 )
6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	185 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	186 "XEP-0424_retract_received", self, self._on_message_retract
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	187 )
6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	188 host.trigger.add_with_check(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	189 "XEP-0372_ref_received", self, self._on_reference_received
4183 6784d07b99c8 plugin XEP-053, component AP gateway: use the new `trigger.add_with_check` method Goffi <goffi@goffi.org> parents: 4169 diff changeset	190 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	191
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	192 host.bridge.add_method(
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	193 "ap_send",
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	194 ".plugin",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	195 in_sign="sss",
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	196 out_sign="",
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	197 method=self._publish_message,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	198 async_=True,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	199 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	200
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	201 def get_handler(self, __):
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	202 return self.pubsub_service
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	203
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	204 async def init(self, client):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	205 if self.initialised:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	206 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	207
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 self.initialised = True
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 log.info(_("ActivityPub Gateway initialization"))
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	210
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	211 # RSA keys
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	212 stored_data = await self.host.memory.storage.get_privates(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	213 IMPORT_NAME, ["rsa_key"], profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	214 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	215 private_key_pem = stored_data.get("rsa_key")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	216 if private_key_pem is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	217 self.private_key = await threads.deferToThread(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	218 rsa.generate_private_key,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	219 public_exponent=65537,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	220 key_size=4096,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	221 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	222 private_key_pem = self.private_key.private_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	223 encoding=serialization.Encoding.PEM,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 format=serialization.PrivateFormat.PKCS8,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	225 encryption_algorithm=serialization.NoEncryption(),
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 ).decode()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	227 await self.host.memory.storage.set_private_value(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 IMPORT_NAME, "rsa_key", private_key_pem, profile=client.profile
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	229 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	230 else:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	231 self.private_key = serialization.load_pem_private_key(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	232 private_key_pem.encode(),
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	233 password=None,
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	234 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	235 self.public_key = self.private_key.public_key()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	236 self.public_key_pem = self.public_key.public_bytes(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	237 encoding=serialization.Encoding.PEM,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	238 format=serialization.PublicFormat.SubjectPublicKeyInfo,
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	239 ).decode()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	240
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	241 # params
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	242 # URL and port
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	243 self.public_url = self.host.memory.config_get(
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	244 CONF_SECTION, "public_url"
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	245 ) or self.host.memory.config_get(CONF_SECTION, "xmpp_domain")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 if self.public_url is None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	247 log.error(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	248 '"public_url" not set in configuration, this is mandatory to have'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	249 "ActivityPub Gateway running. Please set this option it to public facing "
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	250 f"url in {CONF_SECTION!r} configuration section."
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	251 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	252 return
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	253 if parse.urlparse(self.public_url).scheme:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	254 log.error(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	255 'Scheme must not be specified in "public_url", please remove it from '
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	256 '"public_url" configuration option. ActivityPub Gateway won\'t be run.'
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	257 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	258 return
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	259 self.http_port = int(self.host.memory.config_get(CONF_SECTION, "http_port", 8123))
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	260 connection_type = self.host.memory.config_get(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	261 CONF_SECTION, "http_connection_type", "https"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	262 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	263 if connection_type not in ("http", "https"):
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	264 raise exceptions.ConfigError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	265 'bad ap-gateay http_connection_type, you must use one of "http" or '
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	266 '"https"'
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	267 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	268 self.http_sign_get = C.bool(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	269 self.host.memory.config_get(CONF_SECTION, "http_sign_get", C.BOOL_TRUE)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	270 )
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	271 self.max_items = int(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	272 self.host.memory.config_get(CONF_SECTION, "new_node_max_items", 50)
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	273 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	274 self.comments_max_depth = int(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	275 self.host.memory.config_get(CONF_SECTION, "comments_max_depth", 0)
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	276 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	277 self.ap_path = self.host.memory.config_get(CONF_SECTION, "ap_path", "_ap")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	278 self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	279 # True (default) if we provide gateway only to entities/services from our server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	280 self.local_only = C.bool(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	281 self.host.memory.config_get(CONF_SECTION, "local_only", C.BOOL_TRUE)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	282 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	283 # if True (default), mention will be parsed in non-private content coming from
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	284 # XMPP. This is necessary as XEP-0372 are coming separately from item where the
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	285 # mention is done, which is hard to impossible to translate to ActivityPub (where
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	286 # mention specified inside the item directly). See documentation for details.
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	287 self.auto_mentions = C.bool(
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	288 self.host.memory.config_get(CONF_SECTION, "auto_mentions", C.BOOL_TRUE)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	289 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	290
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	291 html_redirect: Dict[str, Union[str, dict]] = self.host.memory.config_get(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	292 CONF_SECTION, "html_redirect_dict", {}
3977 6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	293 )
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	294 self.html_redirect: Dict[str, List[dict]] = {}
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	295 for url_type, target in html_redirect.items():
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	296 if isinstance(target, str):
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	297 target = {"url": target}
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	298 elif not isinstance(target, dict):
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	299 raise exceptions.ConfigError(
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	300 f"html_redirect target must be a URL or a dict, not {target!r}"
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	301 )
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	302 filters = target.setdefault("filters", {})
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	303 if "url" not in target:
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	304 log.warning(f"invalid HTML redirection, missing target URL: {target}")
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	305 continue
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	306 # a slash in the url_type is a syntactic shortcut to have a node filter
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	307 if "/" in url_type:
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	308 url_type, node_filter = url_type.split("/", 1)
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	309 filters["node"] = node_filter
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	310 self.html_redirect.setdefault(url_type, []).append(target)
6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	311
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	312 # HTTP server launch
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	313 self.server = HTTPServer(self)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	314 if connection_type == "http":
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	315 reactor.listenTCP(self.http_port, self.server)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	316 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	317 options = tls.get_options_from_config(self.host.memory.config, CONF_SECTION)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	318 tls.tls_options_check(options)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	319 context_factory = tls.get_tls_context_factory(options)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	320 reactor.listenSSL(self.http_port, self.server, context_factory)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	321
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	322 async def profile_connecting(self, client):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	323 self.client = client
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	324 client.sendHistory = True
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	325 client._ap_storage = persistent.LazyPersistentBinaryDict(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	326 IMPORT_NAME, client.profile
3824 6329ee6b6df4 component AP: convert AP identity data to XMPP: Goffi <goffi@goffi.org> parents: 3821 diff changeset	327 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	328 await self.init(client)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	329
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	330 def profile_connected(self, client):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	331 self.ad_hoc.init(client)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	332
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	333 async def _items_received(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	334 self, client: SatXMPPEntity, itemsEvent: pubsub.ItemsEvent
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	335 ) -> None:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	336 """Callback called when pubsub items are received
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	337
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	338 if the items are adressed to a JID corresponding to an AP actor, they are
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	339 converted to AP items and sent to the corresponding AP server.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	340
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	341 If comments nodes are linked, they are automatically subscribed to get new items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	342 from there too.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	343 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	344 if client != self.client:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	345 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	346 # we need recipient as JID and not gateway own JID to be able to use methods such
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	347 # as "subscribe"
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	348 client = self.client.get_virtual_client(itemsEvent.sender)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	349 recipient = itemsEvent.recipient
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	350 if not recipient.user:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	351 log.debug("ignoring items event without local part specified")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	352 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	353
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	354 ap_account = self._e.unescape(recipient.user)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	355
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	356 if self._pa.is_attachment_node(itemsEvent.nodeIdentifier):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	357 await self.convert_and_post_attachments(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	358 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	359 ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	360 itemsEvent.sender,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	361 itemsEvent.nodeIdentifier,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	362 itemsEvent.items,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	363 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	364 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	365 await self.convert_and_post_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	366 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	367 ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	368 itemsEvent.sender,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	369 itemsEvent.nodeIdentifier,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	370 itemsEvent.items,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	371 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	372
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	373 async def get_virtual_client(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	374 self, requestor_actor_id: str, actor_id: str
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	375 ) -> SatXMPPEntity:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	376 """Get client for this component with a specified jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	377
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	378 This is needed to perform operations with the virtual JID corresponding to the AP
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	379 actor instead of the JID of the gateway itself.
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	380 @param requestor_actor_id: originating actor ID (URL)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	381 @param actor_id: ID of the actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	382 @return: virtual client
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	383 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	384 local_jid = await self.get_jid_from_id(requestor_actor_id, actor_id)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	385 return self.client.get_virtual_client(local_jid)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	386
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	387 def is_activity(self, data: dict) -> bool:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	388 """Return True if the data has an activity type"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	389 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	390 return (data.get("type") or "").lower() in ACTIVITY_TYPES_LOWER
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	391 except (KeyError, TypeError):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	392 return False
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	393
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	394 async def ap_get(self, url: str, requestor_actor_id: str) -> dict:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	395 """Retrieve AP JSON from given URL with HTTP Signature
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	396
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	397 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	398 @param requestor_actor_id: originating actor ID (URL)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	399 @raise error.StanzaError: "service-unavailable" is sent when something went wrong
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	400 with AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	401 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	402 if self.http_sign_get:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	403 headers = self._generate_signed_headers(url, requestor_actor_id, method="get")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	404 else:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	405 headers = {}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	406 headers["Accept"] = MEDIA_TYPE_AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	407
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	408 resp = await treq.get(url, headers=headers)
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	409 if resp.code >= 300:
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	410 text = await resp.text()
03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	411 if resp.code == 404:
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	412 raise exceptions.NotFound(f"Can't find resource at {url}")
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	413 else:
3977 6fa4ca0c047e component AP gateway: HTML redirection: Goffi <goffi@goffi.org> parents: 3904 diff changeset	414 msg = f"HTTP error {resp.code} (url: {url}): {text}"
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	415 raise exceptions.ExternalRequestError(msg)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	416 try:
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	417 return await treq.json_content(resp)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	418 except Exception as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	419 raise error.StanzaError(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	420 "service-unavailable", text=f"Can't get AP data at {url}: {e}"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	421 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	422
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	423 async def ap_post(self, url: str, requestor_actor_id: str, doc: dict) -> TReqResponse:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	424 """Sign a document and post it to AP server
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	425
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	426 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	427 @param requestor_actor_id: originating actor ID (URL)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	428 @param doc: document to send
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	429 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	430 if self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	431 __, actor_args = self.parse_apurl(requestor_actor_id)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	432 actor_account = actor_args[0]
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	433 to_log = [
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	434 "",
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	435 f">>> {actor_account} is signing and posting to {url}:\n{pformat(doc)}",
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	436 ]
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	437
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	438 body = json.dumps(doc).encode()
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	439 headers = self._generate_signed_headers(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	440 url, requestor_actor_id, method="post", body=body
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	441 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	442 headers["Content-Type"] = MEDIA_TYPE_AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	443
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	444 if self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	445 if self.verbose >= 3:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	446 h_to_log = "\n".join(f" {k}: {v}" for k, v in headers.items())
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	447 to_log.append(f" headers:\n{h_to_log}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	448 to_log.append("---")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	449 log.info("\n".join(to_log))
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	450
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	451 resp = await treq.post(url, body, headers=headers)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	452 if resp.code >= 300:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	453 text = await resp.text()
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	454 log.warning(f"POST request to {url} failed [{resp.code}]: {text}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	455 elif self.verbose:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	456 log.info(f"==> response code: {resp.code}")
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	457 return resp
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	458
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	459 def _generate_signed_headers(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	460 self, url: str, actor_id: str, method: str, body: bytes \| None = None
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	461 ) -> dict[str, str]:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	462 """Generate HTTP headers with signature for a given request
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	463
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	464 @param url: AP server endpoint
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	465 @param actor_id: originating actor ID (URL)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	466 @param method: HTTP method (e.g., 'get', 'post')
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	467 @param body: request body if any
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	468 @return: signed headers
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	469 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	470 p_url = parse.urlparse(url)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	471 headers = {
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	472 "(request-target)": f"{method} {p_url.path}",
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	473 "Host": p_url.hostname,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	474 "Date": http.datetimeToString().decode(),
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	475 }
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	476
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	477 if body:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	478 digest_algo, digest_hash = self.get_digest(body)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	479 headers["Digest"] = f"{digest_algo}={digest_hash}"
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	480
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	481 headers, __ = self.get_signature_data(self.get_key_id(actor_id), headers)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	482 return headers
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	483
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	484 @overload
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	485 async def ap_get_object(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	486 self, requestor_actor_id: str, data: dict, key: str
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	487 ) -> dict \| None: ...
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	488
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	489 @overload
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	490 async def ap_get_object(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	491 self, requestor_actor_id: str, data: Union[str, dict], key: None = None
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	492 ) -> dict: ...
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	493
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	494 async def ap_get_object(self, requestor_actor_id: str, data, key=None) -> dict \| None:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	495 """Retrieve an AP object, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	496
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	497 This method is to be used with attributes marked as "Functional" in
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	498 https://www.w3.org/TR/activitystreams-vocabulary
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	499 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	500 @param data: AP object where an other object is looked for, or the object itself
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	501 @param key: name of the object to look for, or None if data is the object directly
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	502 @return: found object if any
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	503 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	504 if key is not None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	505 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	506 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	507 value = data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	508 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	509 if key is None:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	510 raise ValueError("None can't be used with ap_get_object is key is None")
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	511 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	512 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	513 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	514 elif isinstance(value, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	515 if self.is_local_url(value):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	516 return await self.ap_get_local_object(value)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	517 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	518 return await self.ap_get(value, requestor_actor_id)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	519 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	520 raise NotImplementedError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	521 "was expecting a string or a dict, got {type(value)}: {value!r}}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	522 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	523
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	524 async def ap_get_local_object(self, url: str) -> dict:
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	525 """Retrieve or generate local object
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	526
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	527 for now, only handle XMPP items to convert to AP
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	528 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	529 url_type, url_args = self.parse_apurl(url)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	530 if url_type == TYPE_ITEM:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	531 try:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	532 account, item_id = url_args
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	533 except ValueError:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	534 raise ValueError(f"invalid URL: {url}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	535 author_jid, node = await self.get_jid_and_node(account)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	536 if node is None:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	537 node = self._m.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	538 cached_node = await self.host.memory.storage.get_pubsub_node(
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	539 self.client, author_jid, node
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	540 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	541 if not cached_node:
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	542 log.debug(f"node {node!r} at {author_jid} is not found in cache")
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	543 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	544 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	545 cached_items, __ = await self.host.memory.storage.get_items(
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	546 cached_node, item_ids=[item_id]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	547 )
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	548 if not cached_items:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	549 log.debug(
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	550 f"item {item_id!r} of {node!r} at {author_jid} is not found in "
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	551 "cache"
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	552 )
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	553 found_item = None
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	554 else:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	555 found_item = cached_items[0].data
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	556
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	557 if found_item is None:
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	558 # the node is not in cache, we have to make a request to retrieve the item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	559 # If the node doesn't exist, get_items will raise a NotFound exception
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	560 found_items, __ = await self._p.get_items(
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	561 self.client, author_jid, node, item_ids=[item_id]
ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	562 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	563 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	564 found_item = found_items[0]
3880 03761f8ba8bb component AP gateway: raise exceptions on apGet + fix exceptions catching in apGetLocalObject: Goffi <goffi@goffi.org> parents: 3869 diff changeset	565 except IndexError:
4014 4ef473116499 component AP gateway (http): handle properly NotFound error: Goffi <goffi@goffi.org> parents: 4013 diff changeset	566 raise exceptions.NotFound(f"requested item at {url} can't be found")
3851 ef824b1091f3 component AP gateway: get items when not found in cache in `apGetLocalObject`: Goffi <goffi@goffi.org> parents: 3850 diff changeset	567
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	568 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	569 # this is an event
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	570 event_data = self._events.event_elt_2_event_data(found_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	571 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	572 event_data, author_jid
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	573 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	574 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	575 ap_item["object"]["@context"] = ap_item["@context"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	576 return ap_item["object"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	577 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	578 # this is a blog item
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	579 mb_data = await self._m.item_2_mb_data(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	580 self.client, found_item, author_jid, node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	581 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	582 ap_item = await self.mb_data_2_ap_item(self.client, mb_data)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	583 # the URL must return the object and not the activity
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	584 return ap_item["object"]
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	585 else:
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	586 raise NotImplementedError(
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	587 'only object from "item" URLs can be retrieved for now'
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	588 )
943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	589
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	590 async def ap_get_list(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	591 self, requestor_actor_id: str, data: dict, key: str, only_ids: bool = False
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	592 ) -> Optional[List[Dict[str, Any]]]:
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	593 """Retrieve a list of objects from AP data, dereferencing when necessary
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	594
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	595 This method is to be used with non functional vocabularies. Use ``ap_get_object``
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	596 otherwise.
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	597 If the value is a dictionary, it will be wrapped in a list
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	598 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	599 @param data: AP object where a list of objects is looked for
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	600 @param key: key of the list to look for
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	601 @param only_ids: if Trye, only items IDs are retrieved
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	602 @return: list of objects, or None if the key is not present
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	603 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	604 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	605 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	606 return None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	607 elif isinstance(value, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	608 if self.is_local_url(value):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	609 value = await self.ap_get_local_object(value)
3842 943901372eba component AP gateway: `apGetObject` and `apGetList` now work with local object: Goffi <goffi@goffi.org> parents: 3833 diff changeset	610 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	611 value = await self.ap_get(value, requestor_actor_id)
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	612 if isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	613 return [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	614 if not isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	615 raise ValueError(f"A list was expected, got {type(value)}: {value!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	616 if only_ids:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	617 return [{"id": v["id"]} if isinstance(v, dict) else {"id": v} for v in value]
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	618 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	619 return [await self.ap_get_object(requestor_actor_id, i) for i in value]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	620
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	621 async def ap_get_actors(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	622 self, requestor_actor_id: str, data: dict, key: str, as_account: bool = True
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	623 ) -> List[str]:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	624 """Retrieve AP actors from data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	625
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	626 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	627 @param data: AP object containing a field with actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	628 @param key: field to use to retrieve actors
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	629 @param as_account: if True returns account handles, otherwise will return actor
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	630 IDs
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	631 @raise exceptions.DataError: there is not actor data or it is invalid
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	632 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	633 value = data.get(key)
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	634 if value is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	635 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	636 f"no actor associated to object {data.get('id')!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	637 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	638 elif isinstance(value, dict):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	639 actor_id = value.get("id")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	640 if actor_id is None:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	641 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	642 f"invalid actor associated to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	643 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	644 value = [actor_id]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	645 elif isinstance(value, str):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	646 value = [value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	647 elif isinstance(value, list):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	648 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	649 value = [a if isinstance(a, str) else a["id"] for a in value]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	650 except (TypeError, KeyError):
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	651 raise exceptions.DataError(
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	652 f"invalid actors list to object {data.get('id')!r}: {value!r}"
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	653 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	654 if not value:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	655 raise exceptions.DataError(f"list of actors is empty")
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	656 if as_account:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	657 return [
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	658 await self.get_ap_account_from_id(requestor_actor_id, actor_id)
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	659 for actor_id in value
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	660 ]
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	661 else:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	662 return value
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	663
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	664 async def ap_get_sender_actor(
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	665 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	666 requestor_actor_id: str,
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	667 data: dict,
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	668 ) -> str:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	669 """Retrieve actor who sent data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	670
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	671 This is done by checking "actor" field first, then "attributedTo" field.
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	672 Only the first found actor is taken into account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	673 @param requestor_actor_id: ID of the actor doing the request.
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	674 @param data: AP object
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	675 @return: actor id of the sender
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	676 @raise exceptions.NotFound: no actor has been found in data
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	677 """
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	678 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	679 actors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	680 requestor_actor_id, data, "actor", as_account=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	681 )
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	682 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	683 actors = None
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	684 if not actors:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	685 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	686 actors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	687 requestor_actor_id, data, "attributedTo", as_account=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	688 )
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	689 except exceptions.DataError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	690 raise exceptions.NotFound(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	691 'actor not specified in "actor" or "attributedTo"'
3742 bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	692 )
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	693 try:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	694 return actors[0]
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	695 except IndexError:
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	696 raise exceptions.NotFound("list of actors is empty")
bf0505d41c09 comp AP: helper methods to get AP objects: Goffi <goffi@goffi.org> parents: 3729 diff changeset	697
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	698 def must_encode(self, text: str) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	699 """Indicate if a text must be period encoded"""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	700 return (
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	701 not RE_ALLOWED_UNQUOTED.match(text) or text.startswith("___") or "---" in text
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	702 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	703
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	704 def period_encode(self, text: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	705 """Period encode a text
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	706
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	707 see [get_jid_and_node] for reasons of period encoding
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	708 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	709 return (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	710 parse.quote(text, safe="")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	711 .replace("---", "%2d%2d%2d")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	712 .replace("___", "%5f%5f%5f")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	713 .replace(".", "%2e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	714 .replace("~", "%7e")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	715 .replace("%", ".")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	716 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	717
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	718 async def get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	719 self, jid_: jid.JID, node: Optional[str]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	720 ) -> str:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	721 """Construct AP account from JID and node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	722
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	723 The account construction will use escaping when necessary
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	724 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	725 if not node or node == self._m.namespace:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	726 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	727
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	728 if self.client is None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	729 raise exceptions.InternalError("Client is not set yet")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	730
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	731 if self.is_virtual_jid(jid_):
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	732 # this is an proxy JID to an AP Actor
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	733 return self._e.unescape(jid_.user)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	734
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	735 if node and not jid_.user and not self.must_encode(node):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	736 is_pubsub = await self.is_pubsub(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	737 # when we have a pubsub service, the user part can be used to set the node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	738 # this produces more user-friendly AP accounts
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	739 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	740 jid_.user = node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	741 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	742
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	743 is_local = self.is_local(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	744 user = jid_.user if is_local else jid_.userhost()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	745 if user is None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	746 user = ""
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	747 account_elts = []
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	748 if node and self.must_encode(node) or self.must_encode(user):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	749 account_elts = ["___"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	750 if node:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	751 node = self.period_encode(node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	752 user = self.period_encode(user)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	753
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	754 if not user:
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	755 raise MissingLocalPartError("there should be a local part")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	756
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	757 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	758 account_elts.extend((node, "---"))
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	759
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	760 account_elts.extend(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	761 (user, "@", jid_.host if is_local else self.client.jid.userhost())
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	762 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	763 return "".join(account_elts)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	764
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	765 def is_local(self, jid_: jid.JID) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	766 """Returns True if jid_ use a domain or subdomain of gateway's host"""
4297 0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	767 # FIXME: kept for compatiblity, need to be removed in favor of
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	768 # "self.client.is_local".
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	769
0f953ce5f0a8 core (xmpp): move `is_local` to `SatXMPPEntity` + type hints + some log level changes. Goffi <goffi@goffi.org> parents: 4270 diff changeset	770 return self.client.is_local(jid_)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	771
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	772 async def is_pubsub(self, jid_: jid.JID) -> bool:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	773 """Indicate if a JID is a Pubsub service"""
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	774 host_disco = await self.host.get_disco_infos(self.client, jid_)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	775 return ("pubsub", "service") in host_disco.identities and not (
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	776 "pubsub",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	777 "pep",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	778 ) in host_disco.identities
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	779
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	780 async def get_jid_and_node(self, ap_account: str) -> tuple[jid.JID, str \| None]:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	781 """Decode raw AP account handle to get XMPP JID and Pubsub Node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	782
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	783 Username are case insensitive.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	784
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	785 By default, the username correspond to local username (i.e. username from
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	786 component's server).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	787
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	788 If local name's domain is a pubsub service (and not PEP), the username is taken as
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	789 a pubsub node.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	790
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	791 If ``---`` is present in username, the part before is used as pubsub node, and the
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	792 rest as a JID user part.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	793
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	794 If username starts with ``___``, characters are encoded using period encoding
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	795 (i.e. percent encoding where a ``.`` is used instead of ``%``).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	796
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	797 This horror is necessary due to limitation in some AP implementation (notably
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	798 Mastodon), cf. https://github.com/mastodon/mastodon/issues/17222
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	799
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	800 examples:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	801
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	802 ``toto@example.org`` => JID = toto@example.org, node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	803
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	804 ``___toto.40example.net@example.org`` => JID = toto@example.net (this one is a
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	805 non-local JID, and will work only if setings ``local_only`` is False), node = None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	806
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	807 ``toto@pubsub.example.org`` (with pubsub.example.org being a pubsub service) =>
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	808 JID = pubsub.example.org, node = toto
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	809
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	810 ``tata---toto@example.org`` => JID = toto@example.org, node = tata
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	811
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	812 ``___urn.3axmpp.3amicroblog.3a0@pubsub.example.org`` (with pubsub.example.org
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	813 being a pubsub service) ==> JID = pubsub.example.org, node = urn:xmpp:microblog:0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	814
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	815 @param ap_account: ActivityPub account handle (``username@domain.tld``)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	816 @return: service JID and pubsub node
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	817 if pubsub node is None, default microblog pubsub node (and possibly other
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	818 nodes that plugins may hanlde) will be used
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	819 @raise ValueError: invalid account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	820 @raise PermissionError: non local jid is used when gateway doesn't allow them
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	821 """
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	822 if ap_account.count("@") != 1:
4262 d366d90a71aa component AP Gateway: log invalid account in case of error. Goffi <goffi@goffi.org> parents: 4259 diff changeset	823 raise ValueError(f"Invalid AP account {ap_account!r}")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	824 if ap_account.startswith("___"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	825 encoded = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	826 ap_account = ap_account[3:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	827 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	828 encoded = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	829
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	830 username, domain = ap_account.split("@")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	831
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	832 if "---" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	833 node, username = username.rsplit("---", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	834 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	835 node = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	836
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	837 if encoded:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	838 username = parse.unquote(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	839 RE_PERIOD_ENC.sub(r"%\g<hex>", username), errors="strict"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	840 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	841 if node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	842 node = parse.unquote(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	843 RE_PERIOD_ENC.sub(r"%\g<hex>", node), errors="strict"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	844 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	845
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	846 if "@" in username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	847 username, domain = username.rsplit("@", 1)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	848
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	849 if not node:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	850 # we need to check host disco, because disco request to user may be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	851 # blocked for privacy reason (see
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	852 # https://xmpp.org/extensions/xep-0030.html#security)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	853 is_pubsub = await self.is_pubsub(jid.JID(domain))
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	854
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	855 if is_pubsub:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	856 # if the host is a pubsub service and not a PEP, we consider that username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	857 # is in fact the node name
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	858 node = username
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	859 username = None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	860
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	861 jid_s = f"{username}@{domain}" if username else domain
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	862 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	863 jid_ = jid.JID(jid_s)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	864 except RuntimeError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	865 raise ValueError(f"Invalid jid: {jid_s!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	866
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	867 if self.local_only and not self.is_local(jid_):
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	868 raise exceptions.PermissionError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	869 "This gateway is configured to map only local entities and services"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	870 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	871
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	872 return jid_, node
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	873
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	874 def get_local_jid_from_account(self, account: str) -> jid.JID:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	875 """Compute JID linking to an AP account
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	876
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	877 The local jid is computer by escaping AP actor handle and using it as local part
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	878 of JID, where domain part is this gateway own JID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	879 """
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	880 return jid.JID(None, (self._e.escape(account), self.client.jid.host, None))
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	881
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	882 async def get_jid_from_id(self, requestor_actor_id: str, actor_id: str) -> jid.JID:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	883 """Compute JID linking to an AP Actor ID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	884
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	885 The local jid is computer by escaping AP actor handle and using it as local part
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	886 of JID, where domain part is this gateway own JID
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	887 If the actor_id comes from local server (checked with self.public_url), it means
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	888 that we have an XMPP entity, and the original JID is returned
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	889
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	890 @param requestor_actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	891 @param actor_id: ID of the actor to generate JID from.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	892 @return: generated JID.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	893 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	894 if self.is_local_url(actor_id):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	895 request_type, extra_args = self.parse_apurl(actor_id)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	896 if request_type != TYPE_ACTOR or len(extra_args) != 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	897 raise ValueError(f"invalid actor id: {actor_id!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	898 actor_jid, __ = await self.get_jid_and_node(extra_args[0])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	899 return actor_jid
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	900
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	901 account = await self.get_ap_account_from_id(requestor_actor_id, actor_id)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	902 return self.get_local_jid_from_account(account)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	903
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	904 def parse_apurl(self, url: str) -> tuple[str, list[str]]:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	905 """Parse an URL leading to an AP endpoint
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	906
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	907 @param url: URL to parse (schema is not mandatory)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	908 @return: endpoint type and extra arguments
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	909 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	910 path = parse.urlparse(url).path.lstrip("/")
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	911 type_, *extra_args = path[len(self.ap_path) :].lstrip("/").split("/")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	912 return type_, [parse.unquote(a) for a in extra_args]
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	913
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	914 def build_apurl(self, type_: str, *args: str) -> str:
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	915 """Build an AP endpoint URL
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	916
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	917 @param type_: type of AP endpoing
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	918 @param arg: endpoint dependant arguments
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	919 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	920 return parse.urljoin(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	921 self.base_ap_url,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	922 str(Path(type_).joinpath(*(parse.quote_plus(a, safe="@") for a in args))),
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	923 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	924
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	925 def is_local_url(self, url: str) -> bool:
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	926 """Tells if an URL link to this component
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	927
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	928 ``public_url`` and ``ap_path`` are used to check the URL
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	929 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	930 return url.startswith(self.base_ap_url)
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	931
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	932 def is_virtual_jid(self, jid_: jid.JID) -> bool:
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	933 """Tell if a JID is an AP actor mapped through this gateway"""
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	934 return jid_.host == self.client.jid.userhost()
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	935
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	936 def build_signature_header(self, values: Dict[str, str]) -> str:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	937 """Build key="<value>" signature header from signature data"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	938 fields = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	939 for key, value in values.items():
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	940 if key not in ("(created)", "(expired)"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	941 if '"' in value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	942 raise NotImplementedError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	943 "string escaping is not implemented, double-quote can't be used "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	944 f"in {value!r}"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	945 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	946 value = f'"{value}"'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	947 fields.append(f"{key}={value}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	948
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	949 return ",".join(fields)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	950
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	951 def get_digest(self, body: bytes, algo="SHA-256") -> Tuple[str, str]:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	952 """Get digest data to use in header and signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	953
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	954 @param body: body of the request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	955 @return: hash name and digest
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	956 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	957 if algo != "SHA-256":
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	958 raise NotImplementedError("only SHA-256 is implemented for now")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	959 return algo, base64.b64encode(hashlib.sha256(body).digest()).decode()
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	960
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	961 @async_lru(maxsize=LRU_MAX_SIZE)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	962 async def get_actor_data(self, requestor_actor_id: str, actor_id: str) -> dict:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	963 """Retrieve actor data with LRU cache"""
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	964 return await self.ap_get(actor_id, requestor_actor_id)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	965
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	966 @async_lru(maxsize=LRU_MAX_SIZE)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	967 async def get_actor_pub_key_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	968 self, requestor_actor_id: str, actor_id: str
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	969 ) -> Tuple[str, str, rsa.RSAPublicKey]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	970 """Retrieve Public Key data from actor ID
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	971
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	972 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	973 @param actor_id: actor ID (url)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	974 @return: key_id, owner and public_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	975 @raise KeyError: publicKey is missing from actor data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	976 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	977 actor_data = await self.get_actor_data(requestor_actor_id, actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	978 pub_key_data = actor_data["publicKey"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	979 key_id = pub_key_data["id"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	980 owner = pub_key_data["owner"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	981 pub_key_pem = pub_key_data["publicKeyPem"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	982 pub_key = serialization.load_pem_public_key(pub_key_pem.encode())
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	983 return key_id, owner, pub_key
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	984
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	985 def create_activity(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	986 self,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	987 activity: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	988 actor_id: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	989 object_: Optional[Union[str, dict]] = None,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	990 target: Optional[Union[str, dict]] = None,
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	991 activity_id: Optional[str] = None,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	992 **kwargs,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	993 ) -> Dict[str, Any]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	994 """Generate base data for an activity
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	995
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	996 @param activity: one of ACTIVITY_TYPES
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	997 @param actor_id: AP actor ID of the sender
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	998 @param object_: content of "object" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	999 @param target: content of "target" field
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1000 @param activity_id: ID to use for the activity
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1001 if not set it will be automatically generated, but it is usually desirable to
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1002 set the ID manually so it can be retrieved (e.g. for Undo)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1003 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1004 if activity not in ACTIVITY_TYPES:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1005 raise exceptions.InternalError(f"invalid activity: {activity!r}")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1006 if object_ is None and activity in ACTIVITY_OBJECT_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1007 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1008 f'"object_" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1009 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1010 if target is None and activity in ACTIVITY_TARGET_MANDATORY:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1011 raise exceptions.InternalError(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1012 f'"target" is mandatory for activity {activity!r}'
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1013 )
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	1014 if activity_id is None:
4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	1015 activity_id = f"{actor_id}#{activity.lower()}_{shortuuid.uuid()}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1016 data: Dict[str, Any] = {
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1017 "@context": [NS_AP],
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1018 "actor": actor_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1019 "id": activity_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1020 "type": activity,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1021 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1022 data.update(kwargs)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1023 if object_ is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1024 data["object"] = object_
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1025 if target is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1026 data["target"] = target
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1027
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1028 return data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1029
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1030 def get_key_id(self, actor_id: str) -> str:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1031 """Get local key ID from actor ID"""
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1032 return f"{actor_id}#main-key"
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1033
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1034 async def check_signature(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1035 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1036 requestor_actor_id: str,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1037 signature: str,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1038 key_id: str,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1039 headers: Dict[str, str],
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1040 ) -> str:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1041 """Verify that signature matches given headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1042
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1043 see https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-06#section-3.1.2
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1044
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1045 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1046 @param signature: Base64 encoded signature
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1047 @param key_id: ID of the key used to sign the data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1048 @param headers: headers and their values, including pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1049 @return: id of the signing actor
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1050
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1051 @raise InvalidSignature: signature doesn't match headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1052 """
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1053 to_sign = "\n".join(f"{k.lower()}: {v}" for k, v in headers.items())
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1054 if key_id.startswith("acct:"):
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1055 actor = key_id[5:]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1056 actor_id = await self.get_ap_actor_id_from_account(actor)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1057 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1058 actor_id = key_id.split("#", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1059
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1060 pub_key_id, pub_key_owner, pub_key = await self.get_actor_pub_key_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1061 requestor_actor_id, actor_id
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1062 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1063 if pub_key_id != key_id or pub_key_owner != actor_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1064 raise exceptions.EncryptionError("Public Key mismatch")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1065
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1066 try:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1067 pub_key.verify(
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1068 base64.b64decode(signature),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1069 to_sign.encode(),
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1070 # we have to use PKCS1v15 padding to be compatible with Mastodon
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1071 padding.PKCS1v15(), # type: ignore
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1072 hashes.SHA256(), # type: ignore
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1073 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1074 except InvalidSignature:
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1075 raise exceptions.EncryptionError(
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1076 "Invalid signature (using PKC0S1 v1.5 and SHA-256)"
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1077 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1078
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1079 return actor_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1080
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1081 def get_signature_data(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1082 self, key_id: str, headers: Dict[str, str]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1083 ) -> Tuple[Dict[str, str], Dict[str, str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1084 """Generate and return signature and corresponding headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1085
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1086 @param parsed_url: URL where the request is sent/has been received
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1087 @param key_id: ID of the key (URL linking to the data with public key)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1088 @param date: HTTP datetime string of signature generation
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1089 @param body: body of the HTTP request
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1090 @param headers: headers to sign and their value:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1091 default value will be used if not specified
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1092
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1093 @return: headers and signature data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1094 ``headers`` is an updated copy of ``headers`` arguments, with pseudo-headers
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1095 removed, and ``Signature`` added.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1096 """
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1097 # headers must be lower case
6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1098 l_headers: Dict[str, str] = {k.lower(): v for k, v in headers.items()}
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1099 to_sign = "\n".join(f"{k}: {v}" for k, v in l_headers.items())
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1100 signature = base64.b64encode(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1101 self.private_key.sign(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1102 to_sign.encode(),
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1103 # we have to use PKCS1v15 padding to be compatible with Mastodon
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1104 padding.PKCS1v15(), # type: ignore
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1105 hashes.SHA256(), # type: ignore
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1106 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1107 ).decode()
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1108 sign_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1109 "keyId": key_id,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1110 "Algorithm": "rsa-sha256",
3883 6da749bbf320 component AP gateway: fix headers case in signature: Goffi <goffi@goffi.org> parents: 3881 diff changeset	1111 "headers": " ".join(l_headers.keys()),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1112 "signature": signature,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1113 }
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1114 new_headers = {k: v for k, v in headers.items() if not k.startswith("(")}
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1115 new_headers["Signature"] = self.build_signature_header(sign_data)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1116 return new_headers, sign_data
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1117
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1118 async def convert_and_post_items(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1119 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1120 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1121 ap_account: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1122 service: jid.JID,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1123 node: str,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1124 items: List[domish.Element],
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1125 subscribe_extra_nodes: bool = True,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1126 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1127 """Convert XMPP items to AP items and post them to actor inbox
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1128
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1129 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1130 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1131 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1132 @param node: (virtual) node corresponding where the item has been published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1133 @param subscribe_extra_nodes: if True, extra data nodes will be automatically
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1134 subscribed, that is comment nodes if present and attachments nodes.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1135 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1136 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1137 requestor_actor_id = self.build_apurl(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1138 TYPE_ACTOR, await self.get_ap_account_from_jid_and_node(service, node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1139 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1140 inbox = await self.get_ap_inbox_from_id(requestor_actor_id, actor_id)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1141 for item in items:
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1142 if item.name == "item":
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1143 cached_item = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1144 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1145 "profiles": [self.client.profile],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1146 "services": [service],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1147 "nodes": [node],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1148 "names": [item["id"]],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1149 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1150 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1151 is_new = not bool(cached_item)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1152 if node.startswith(self._events.namespace):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1153 # event item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1154 event_data = self._events.event_elt_2_event_data(item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1155 try:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1156 author_jid = jid.JID(item["publisher"]).userhostJID()
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1157 except (KeyError, RuntimeWarning):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1158 root_elt = item
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1159 while root_elt.parent is not None:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1160 root_elt = root_elt.parent
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1161 author_jid = jid.JID(root_elt["from"]).userhostJID()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1162 if subscribe_extra_nodes and not self.is_virtual_jid(author_jid):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1163 # we subscribe automatically to comment nodes if any
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1164 recipient_jid = self.get_local_jid_from_account(ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1165 recipient_client = self.client.get_virtual_client(recipient_jid)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1166 comments_data = event_data.get("comments")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1167 if comments_data:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1168 comment_service = jid.JID(comments_data["jid"])
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1169 comment_node = comments_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1170 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1171 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1172 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1173 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1174 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1175 recipient_client, service, node, event_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1176 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1177 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1178 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1179 f"no attachment node found for item {event_data['id']!r} "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1180 f"on {node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1181 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1182 ap_item = await self.ap_events.event_data_2_ap_item(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1183 event_data, author_jid, is_new=is_new
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1184 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1185 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1186 # blog item
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1187 mb_data = await self._m.item_2_mb_data(client, item, service, node)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1188 author_jid = jid.JID(mb_data["author_jid"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1189 if subscribe_extra_nodes and not self.is_virtual_jid(author_jid):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1190 # we subscribe automatically to comment nodes if any
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1191 recipient_jid = self.get_local_jid_from_account(ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1192 recipient_client = self.client.get_virtual_client(recipient_jid)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1193 for comment_data in mb_data.get("comments", []):
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1194 comment_service = jid.JID(comment_data["service"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1195 if self.is_virtual_jid(comment_service):
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1196 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1197 f"ignoring virtual comment service: {comment_data}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1198 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1199 continue
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1200 comment_node = comment_data["node"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1201 await self._p.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1202 recipient_client, comment_service, comment_node
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1203 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1204 try:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1205 await self._pa.subscribe(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1206 recipient_client, service, node, mb_data["id"]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1207 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1208 except exceptions.NotFound:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1209 log.debug(
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1210 f"no attachment node found for item {mb_data['id']!r} on "
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1211 f"{node!r} at {service}"
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1212 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1213 ap_item = await self.mb_data_2_ap_item(client, mb_data, is_new=is_new)
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1214
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	1215 url_actor = ap_item["actor"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1216 elif item.name == "retract":
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1217 url_actor, ap_item = await self.ap_delete_item(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	1218 client.jid, node, item["id"]
3792 865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1219 )
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1220 else:
865167c34b82 comp AP gateway: convert pubsub item retractation to AP `Delete` activity: Goffi <goffi@goffi.org> parents: 3784 diff changeset	1221 raise exceptions.InternalError(f"unexpected element: {item.toXml()}")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1222 await self.ap_post(inbox, url_actor, ap_item)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1223
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1224 async def convert_and_post_attachments(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1225 self,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1226 client: SatXMPPEntity,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1227 ap_account: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1228 service: jid.JID,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1229 node: str,
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1230 items: List[domish.Element],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1231 publisher: Optional[jid.JID] = None,
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1232 ) -> None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1233 """Convert XMPP item attachments to AP activities and post them to actor inbox
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1234
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1235 @param ap_account: account of ActivityPub actor receiving the item
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1236 @param service: JID of the (virtual) pubsub service where the item has been
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1237 published
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1238 @param node: (virtual) node corresponding where the item has been published
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1239 subscribed, that is comment nodes if present and attachments nodes.
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1240 @param items: attachments items
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1241 @param publisher: publisher of the attachments item (it's NOT the PEP/Pubsub
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1242 service, it's the publisher of the item). To be filled only when the publisher
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1243 is known for sure, otherwise publisher will be determined either if
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1244 "publisher" attribute is set by pubsub service, or as a last resort, using
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1245 item's ID (which MUST be publisher bare JID according to pubsub-attachments
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1246 specification).
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1247 """
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1248 if len(items) != 1:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1249 log.warning(
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1250 "we should get exactly one attachment item for an entity, got "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1251 f"{len(items)})"
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1252 )
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1253
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1254 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1255 requestor_actor_id = self.build_apurl(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1256 TYPE_ACTOR, await self.get_ap_account_from_jid_and_node(service, node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1257 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1258 inbox = await self.get_ap_inbox_from_id(requestor_actor_id, actor_id)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1259
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1260 item_elt = items[0]
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1261 item_id = item_elt["id"]
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1262
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1263 if publisher is None:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1264 item_pub_s = item_elt.getAttribute("publisher")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1265 publisher = jid.JID(item_pub_s) if item_pub_s else jid.JID(item_id)
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1266
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1267 if publisher.userhost() != item_id:
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1268 log.warning(
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1269 "attachments item ID must be publisher's bare JID, ignoring: "
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1270 f"{item_elt.toXml()}"
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1271 )
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1272 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1273
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1274 if self.is_virtual_jid(publisher):
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1275 log.debug(f"ignoring item coming from local virtual JID {publisher}")
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1276 return
c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1277
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1278 if publisher is not None:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1279 item_elt["publisher"] = publisher.userhost()
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1280
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1281 item_service, item_node, item_id = self._pa.attachment_node_2_item(node)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1282 item_account = await self.get_ap_account_from_jid_and_node(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1283 item_service, item_node
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1284 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1285 if self.is_virtual_jid(item_service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1286 # it's a virtual JID mapping to an external AP actor, we can use the
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1287 # item_id directly
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1288 item_url = item_id
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1289 if not item_url.startswith("https:"):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1290 log.warning(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1291 "item ID of external AP actor is not an https link, ignoring: "
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1292 f"{item_id!r}"
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1293 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1294 return
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1295 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1296 item_url = self.build_apurl(TYPE_ITEM, item_account, item_id)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1297
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1298 old_attachment_pubsub_items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1299 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1300 "profiles": [self.client.profile],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1301 "services": [service],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1302 "nodes": [node],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1303 "names": [item_elt["id"]],
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1304 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1305 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1306 if not old_attachment_pubsub_items:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1307 old_attachment = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1308 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1309 old_attachment_items = [i.data for i in old_attachment_pubsub_items]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1310 old_attachments = self._pa.items_2_attachment_data(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1311 client, old_attachment_items
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1312 )
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1313 try:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1314 old_attachment = old_attachments[0]
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1315 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1316 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1317 old_attachment = {}
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1318 publisher_account = await self.get_ap_account_from_jid_and_node(publisher, None)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1319 publisher_actor_id = self.build_apurl(TYPE_ACTOR, publisher_account)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1320 try:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1321 attachments = self._pa.items_2_attachment_data(client, [item_elt])[0]
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1322 except IndexError:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1323 # no known element was present in attachments
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1324 attachments = {}
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1325
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1326 # noticed
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1327 if "noticed" in attachments:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1328 if not "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1329 # new "noticed" attachment, we translate to "Like" activity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1330 activity_id = self.build_apurl("like", item_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1331 activity = self.create_activity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1332 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1333 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1334 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1335 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1336 await self.ap_post(inbox, publisher_actor_id, activity)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1337 else:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1338 if "noticed" in old_attachment:
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1339 # "noticed" attachment has been removed, we undo the "Like" activity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1340 activity_id = self.build_apurl("like", item_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1341 activity = self.create_activity(
3869 c0bcbcf5b4b7 component AP gateway: handle `Like` and `Undo`/`Like` activities: Goffi <goffi@goffi.org> parents: 3865 diff changeset	1342 TYPE_LIKE, publisher_actor_id, item_url, activity_id=activity_id
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1343 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1344 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1345 activity["cc"] = [NS_AP_PUBLIC]
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1346 undo = self.create_activity("Undo", publisher_actor_id, activity)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1347 await self.ap_post(inbox, publisher_actor_id, undo)
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1348
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1349 # reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1350 new_reactions = set(attachments.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1351 old_reactions = set(old_attachment.get("reactions", {}).get("reactions", []))
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1352 reactions_remove = old_reactions - new_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1353 reactions_add = new_reactions - old_reactions
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1354 for reactions, undo in ((reactions_remove, True), (reactions_add, False)):
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1355 for reaction in reactions:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1356 activity_id = self.build_apurl(
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1357 "reaction", item_account, item_id, reaction.encode().hex()
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1358 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1359 reaction_activity = self.create_activity(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1360 TYPE_REACTION, publisher_actor_id, item_url, activity_id=activity_id
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1361 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1362 reaction_activity["content"] = reaction
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1363 reaction_activity["to"] = [ap_account]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1364 reaction_activity["cc"] = [NS_AP_PUBLIC]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1365 if undo:
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1366 activy = self.create_activity(
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1367 "Undo", publisher_actor_id, reaction_activity
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1368 )
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1369 else:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1370 activy = reaction_activity
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1371 await self.ap_post(inbox, publisher_actor_id, activy)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1372
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1373 # RSVP
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1374 if "rsvp" in attachments:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1375 attending = attachments["rsvp"].get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1376 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1377 if attending != old_attending:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1378 activity_type = TYPE_JOIN if attending == "yes" else TYPE_LEAVE
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1379 activity_id = self.build_apurl(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1380 activity_type.lower(), item_account, item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1381 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1382 activity = self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1383 activity_type, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1384 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1385 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1386 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1387 await self.ap_post(inbox, publisher_actor_id, activity)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1388 else:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1389 if "rsvp" in old_attachment:
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1390 old_attending = old_attachment.get("rsvp", {}).get("attending", "no")
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1391 if old_attending == "yes":
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1392 activity_id = self.build_apurl(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1393 TYPE_LEAVE.lower(), item_account, item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1394 )
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1395 activity = self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1396 TYPE_LEAVE, publisher_actor_id, item_url, activity_id=activity_id
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1397 )
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1398 activity["to"] = [ap_account]
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1399 activity["cc"] = [NS_AP_PUBLIC]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1400 await self.ap_post(inbox, publisher_actor_id, activity)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1401
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1402 if service.user and self.is_virtual_jid(service):
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1403 # the item is on a virtual service, we need to store it in cache
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1404 log.debug("storing attachments item in cache")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1405 cached_node = await self.host.memory.storage.get_pubsub_node(
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1406 client, service, node, with_subscriptions=True, create=True
59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1407 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1408 await self.host.memory.storage.cache_pubsub_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1409 self.client, cached_node, [item_elt], [attachments]
3865 59fbb66b2923 component AP gateway: handle XMPP attachments -> AP likes conversion: Goffi <goffi@goffi.org> parents: 3856 diff changeset	1410 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1411
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1412 def _publish_message(self, mess_data_s: str, service_s: str, profile: str):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1413 mess_data: dict = data_format.deserialise(mess_data_s) # type: ignore
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1414 service = jid.JID(service_s)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1415 client = self.host.get_client(profile)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1416 return defer.ensureDeferred(self.publish_message(client, mess_data, service))
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1417
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1418 @async_lru(maxsize=LRU_MAX_SIZE)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1419 async def get_ap_actor_id_from_account(self, account: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1420 """Retrieve account ID from it's handle using WebFinger
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1421
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1422 Don't use this method to get local actor id from a local account derivated for
2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1423 JID: in this case, the actor ID is retrieve with
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1424 ``self.build_apurl(TYPE_ACTOR, ap_account)``
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	1425
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1426 @param account: AP handle (user@domain.tld)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1427 @return: Actor ID (which is an URL)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1428 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1429 if account.count("@") != 1 or "/" in account:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1430 raise ValueError(f"Invalid account: {account!r}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1431 host = account.split("@")[1]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1432 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1433 finger_data = await treq.json_content(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1434 await treq.get(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1435 f"https://{host}/.well-known/webfinger?"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1436 f"resource=acct:{parse.quote_plus(account)}",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1437 )
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1438 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1439 except Exception as e:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	1440 raise exceptions.DataError(f"Can't get webfinger data for {account!r}: {e}")
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1441 for link in finger_data.get("links", []):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1442 if (
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1443 link.get("type") == "application/activity+json"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1444 and link.get("rel") == "self"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1445 ):
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1446 href = link.get("href", "").strip()
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1447 if not href:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1448 raise ValueError(
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1449 f"Invalid webfinger data for {account:r}: missing href"
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1450 )
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1451 break
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1452 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1453 raise ValueError(f"No ActivityPub link found for {account!r}")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1454 return href
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1455
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1456 async def get_ap_actor_data_from_account(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1457 self, requestor_actor_id: str, account: str
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1458 ) -> dict:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1459 """Retrieve ActivityPub Actor data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1460
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1461 @param account: ActivityPub Actor identifier
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1462 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1463 href = await self.get_ap_actor_id_from_account(account)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1464 return await self.ap_get(href, requestor_actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1465
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1466 async def get_ap_inbox_from_id(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1467 self, requestor_actor_id: str, actor_id: str, use_shared: bool = True
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1468 ) -> str:
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1469 """Retrieve inbox of an actor_id
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1470
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1471 @param requestor_actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1472 @param actor_id: ID of the actor from whom Inbox must be retrieved.
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1473 @param use_shared: if True, and a shared inbox exists, it will be used instead of
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1474 the user inbox
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1475 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1476 data = await self.get_actor_data(requestor_actor_id, actor_id)
3888 aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1477 if use_shared:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1478 try:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1479 return data["endpoints"]["sharedInbox"]
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1480 except KeyError:
aa7197b67c26 component AP gateway: AP <=> XMPP reactions conversions: Goffi <goffi@goffi.org> parents: 3883 diff changeset	1481 pass
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1482 return data["inbox"]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1483
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1484 @async_lru(maxsize=LRU_MAX_SIZE)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1485 async def get_ap_account_from_id(self, requestor_actor_id: str, actor_id: str) -> str:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1486 """Retrieve AP account from the ID URL
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1487
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1488 Works with external or local actor IDs.
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1489 @param requestor_actor_id: ID of the actor doing the request.
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1490 @param actor_id: AP ID of the actor (URL to the actor data)
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1491 @return: AP handle
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1492 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1493 if self.is_local_url(actor_id):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1494 url_type, url_args = self.parse_apurl(actor_id)
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1495 if url_type != "actor" or not url_args:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1496 raise exceptions.DataError(f"invalid local actor ID: {actor_id}")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1497 account = url_args[0]
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1498 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1499 account_user, account_host = account.split("@")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1500 except ValueError:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1501 raise exceptions.DataError(f"invalid account from url: {actor_id}")
3843 17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1502 if not account_user or account_host != self.public_url:
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1503 raise exceptions.DataError(
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1504 f"{account!r} is not a valid local account (from {actor_id})"
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1505 )
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1506 return account
17c757bd74bc component AP gateway: `getAPAccountFromId` now works with local IDs: Goffi <goffi@goffi.org> parents: 3842 diff changeset	1507
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1508 url_parsed = parse.urlparse(actor_id)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1509 actor_data = await self.get_actor_data(requestor_actor_id, actor_id)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1510 username = actor_data.get("preferredUsername")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1511 if not username:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1512 raise exceptions.DataError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1513 'No "preferredUsername" field found, can\'t retrieve actor account'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1514 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1515 account = f"{username}@{url_parsed.hostname}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1516 # we try to retrieve the actor ID from the account to check it
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1517 found_id = await self.get_ap_actor_id_from_account(account)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1518 if found_id != actor_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1519 # cf. https://socialhub.activitypub.rocks/t/how-to-retrieve-user-server-tld-handle-from-actors-url/2196
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1520 msg = (
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1521 f"Account ID found on WebFinger {found_id!r} doesn't match our actor ID "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1522 f"({actor_id!r}). This AP instance doesn't seems to use "
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1523 '"preferredUsername" as we expect.'
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1524 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1525 log.warning(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1526 raise exceptions.DataError(msg)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1527 return account
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1528
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1529 async def get_ap_items(
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1530 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1531 actor_id: str,
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1532 collection: dict,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1533 max_items: Optional[int] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1534 chronological_pagination: bool = True,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1535 after_id: Optional[str] = None,
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1536 start_index: Optional[int] = None,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1537 parser: Optional[Callable[[str, dict], Awaitable[domish.Element]]] = None,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1538 only_ids: bool = False,
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1539 ) -> Tuple[List[domish.Element], rsm.RSMResponse]:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1540 """Retrieve AP items and convert them to XMPP items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1541
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1542 @param actor_id: ID of the actor doing the request.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1543 @param collection: AP collection data.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1544 Items will be retrieved from this collection.
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1545 @param max_items: maximum number of items to retrieve
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1546 retrieve all items by default
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1547 @param chronological_pagination: get pages in chronological order
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1548 AP use reversed chronological order for pagination, "first" page returns more
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1549 recent items. If "chronological_pagination" is True, "last" AP page will be
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1550 retrieved first.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1551 @param after_id: if set, retrieve items starting from given ID
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1552 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1553 if ``after_id`` is not already in cache, we have to retrieve every page until
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1554 we find it.
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1555 In most common cases, ``after_id`` should be in cache though (client usually
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1556 use known ID when in-order pagination is used).
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1557 @param start_index: start retrieving items from the one with given index
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1558 Due to ActivityStream Collection Paging limitations, this is inefficient and
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1559 all pages before the requested index will be retrieved to count items.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1560 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1561 if None, use default generic parser
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1562 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1563 Retrieving only item IDs avoid HTTP requests to retrieve items, it may be
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1564 sufficient in some use cases (e.g. when retrieving following/followers
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1565 collections)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1566 @return: XMPP Pubsub items and corresponding RSM Response
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1567 Items are always returned in chronological order in the result
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1568 """
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1569 if parser is None:
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1570 parser = self.ap_item_2_mb_elt
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1571
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1572 rsm_resp: Dict[str, Union[bool, int]] = {}
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	1573 try:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1574 count = collection["totalItems"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1575 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1576 log.warning(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1577 f'"totalItems" not found in collection {collection.get("id")}, '
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1578 "defaulting to 20"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1579 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1580 count = 20
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1581 else:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1582 log.info(f"{collection.get('id')} has {count} item(s)")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1583
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1584 rsm_resp["count"] = count
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1585
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1586 if start_index is not None:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1587 assert chronological_pagination and after_id is None
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1588 if start_index >= count:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1589 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1590 elif start_index == 0:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1591 # this is the default behaviour
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1592 pass
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1593 elif start_index > 5000:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1594 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1595 "feature-not-implemented",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1596 text="Maximum limit for previous_index has been reached, this limit"
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1597 "is set to avoid DoS",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1598 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1599 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1600 # we'll convert "start_index" to "after_id", thus we need the item just
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1601 # before "start_index"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1602 previous_index = start_index - 1
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1603 retrieved_items = 0
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1604 current_page = collection["last"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1605 while retrieved_items < count:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1606 page_data, items = await self.parse_ap_page(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1607 actor_id, current_page, parser, only_ids
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1608 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1609 if not items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1610 log.warning(f"found an empty AP page at {current_page}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1611 return [], rsm_resp
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1612 page_start_idx = retrieved_items
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1613 retrieved_items += len(items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1614 if previous_index <= retrieved_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1615 after_id = items[previous_index - page_start_idx]["id"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1616 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1617 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1618 current_page = page_data["prev"]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1619 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1620 log.warning(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1621 f"missing previous page link at {current_page}: {page_data!r}"
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1622 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1623 raise error.StanzaError(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1624 "service-unavailable",
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1625 "Error while retrieving previous page from AP service at "
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1626 f"{current_page}",
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1627 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1628
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1629 init_page = "last" if chronological_pagination else "first"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1630 page = collection.get(init_page)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1631 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1632 raise exceptions.DataError(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1633 f"Initial page {init_page!r} not found for collection "
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1634 f"{collection.get('id')})"
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1635 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1636 items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1637 page_items = []
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1638 retrieved_items = 0
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1639 found_after_id = False
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1640
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1641 while retrieved_items < count:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1642 __, page_items = await self.parse_ap_page(actor_id, page, parser, only_ids)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1643 if not page_items:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1644 break
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1645 retrieved_items += len(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1646 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1647 # if we have an after_id, we ignore all items until the requested one is
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1648 # found
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1649 try:
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1650 limit_idx = [i["id"] for i in page_items].index(after_id)
86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1651 except ValueError:
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1652 # if "after_id" is not found, we don't add any item from this page
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1653 page_id = page.get("id") if isinstance(page, dict) else page
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1654 log.debug(f"{after_id!r} not found at {page_id}, skipping")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1655 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1656 found_after_id = True
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1657 if chronological_pagination:
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1658 start_index = retrieved_items - len(page_items) + limit_idx + 1
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1659 page_items = page_items[limit_idx + 1 :]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1660 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1661 start_index = count - (
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1662 retrieved_items - len(page_items) + limit_idx + 1
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1663 )
3729 86eea17cafa7 component AP gateway: split plugin in several files: Goffi <goffi@goffi.org> parents: 3728 diff changeset	1664 page_items = page_items[:limit_idx]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1665 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1666 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1667 items.extend(page_items)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1668 if max_items is not None and len(items) >= max_items:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1669 if chronological_pagination:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1670 items = items[:max_items]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1671 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1672 items = items[-max_items:]
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1673 break
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1674 page = collection.get("prev" if chronological_pagination else "next")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1675 if not page:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1676 break
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1677
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1678 if after_id is not None and not found_after_id:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1679 raise error.StanzaError("item-not-found")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1680
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1681 if items:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1682 if after_id is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1683 rsm_resp["index"] = 0 if chronological_pagination else count - len(items)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1684 if start_index is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1685 rsm_resp["index"] = start_index
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1686 elif after_id is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1687 log.warning("Can't determine index of first element")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1688 elif chronological_pagination:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1689 rsm_resp["index"] = 0
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1690 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1691 rsm_resp["index"] = count - len(items)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1692 rsm_resp.update({"first": items[0]["id"], "last": items[-1]["id"]})
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1693
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1694 return items, rsm.RSMResponse(**rsm_resp)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1695
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1696 async def ap_item_2_mb_data_and_elt(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1697 self, requestor_actor_id: str, ap_item: dict
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1698 ) -> tuple[dict, domish.Element]:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1699 """Convert AP item to parsed microblog data and corresponding item element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1700
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1701 @param requestor_actor_id: ID of the actor requesting the conversion.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1702 @param ap_item: AP item to convert.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1703 @return: microblog and correspondign <item> element.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1704 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1705 mb_data = await self.ap_item_2_mb_data(requestor_actor_id, ap_item)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1706 item_elt = await self._m.mb_data_2_entry_elt(
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1707 self.client, mb_data, mb_data["id"], None, self._m.namespace
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1708 )
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1709 if "repeated" in mb_data["extra"]:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1710 item_elt["publisher"] = mb_data["extra"]["repeated"]["by"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1711 else:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1712 item_elt["publisher"] = mb_data["author_jid"]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1713 return mb_data, item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1714
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1715 async def ap_item_2_mb_elt(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1716 self, requestor_actor_id: str, ap_item: dict
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1717 ) -> domish.Element:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1718 """Convert AP item to XMPP item element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1719
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1720 @param requestor_actor_id: ID of the actor requesting the conversion.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1721 @param ap_item: AP item to convert.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1722 @return: <item> element
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1723 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1724 __, item_elt = await self.ap_item_2_mb_data_and_elt(requestor_actor_id, ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1725 return item_elt
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1726
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1727 async def parse_ap_page(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1728 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1729 requestor_actor_id: str,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1730 page: Union[str, dict],
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1731 parser: Callable[[str, dict], Awaitable[domish.Element]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1732 only_ids: bool = False,
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1733 ) -> Tuple[dict, List[domish.Element]]:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1734 """Convert AP objects from an AP page to XMPP items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1735
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1736 @param requestor_actor_id: ID of the actor doing the request.
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1737 @param page: Can be either url linking and AP page, or the page data directly
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1738 @param parser: method to use to parse AP items and get XMPP item elements
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1739 @param only_ids: if True, only retrieve items IDs
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1740 @return: page data, pubsub items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1741 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1742 page_data = await self.ap_get_object(requestor_actor_id, page)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1743 if page_data is None:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1744 log.warning("No data found in collection")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1745 return {}, []
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1746 ap_items = await self.ap_get_list(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1747 requestor_actor_id, page_data, "orderedItems", only_ids=only_ids
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1748 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1749 if ap_items is None:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1750 ap_items = await self.ap_get_list(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1751 requestor_actor_id, page_data, "items", only_ids=only_ids
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1752 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1753 if not ap_items:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1754 log.warning(f"No item field found in collection: {page_data!r}")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1755 return page_data, []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1756 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1757 log.warning("Items are not ordered, this is not spec compliant")
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1758 items = []
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1759 # AP Collections are in antichronological order, but we expect chronological in
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1760 # Pubsub, thus we reverse it
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1761 for ap_item in reversed(ap_items):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1762 try:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1763 items.append(await parser(requestor_actor_id, ap_item))
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1764 except (exceptions.DataError, NotImplementedError, error.StanzaError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1765 continue
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1766
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1767 return page_data, items
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1768
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1769 async def get_comments_nodes(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1770 self, requestor_actor_id: str, item_id: str, parent_id: Optional[str]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1771 ) -> Tuple[Optional[str], Optional[str]]:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1772 """Get node where this item is and node to use for comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1773
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1774 if config option "comments_max_depth" is set, a common node will be used below the
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1775 given depth
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1776 @param requestor_actor_id: ID of the actor doing the request.
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1777 @param item_id: ID of the reference item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1778 @param parent_id: ID of the parent item if any (the ID set in "inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1779 @return: a tuple with parent_node_id, comments_node_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1780 - parent_node_id is the ID of the node where reference item must be. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1781 returned when the root node (i.e. not a comments node) must be used.
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1782 - comments_node_id: is the ID of the node to use for comments. None is
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1783 returned when no comment node must be used (happens when we have reached
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1784 "comments_max_depth")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1785 """
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1786 if parent_id is None or not self.comments_max_depth:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1787 return (
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1788 self._m.get_comments_node(parent_id) if parent_id is not None else None,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1789 self._m.get_comments_node(item_id),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1790 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1791 parent_url = parent_id
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1792 parents = []
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1793 for __ in range(COMMENTS_MAX_PARENTS):
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1794 parent_item = await self.ap_get(parent_url, requestor_actor_id)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1795 parents.insert(0, parent_item)
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1796 parent_url = parent_item.get("inReplyTo")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1797 if parent_url is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1798 break
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1799 parent_limit = self.comments_max_depth - 1
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1800 if len(parents) <= parent_limit:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1801 return (
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1802 self._m.get_comments_node(parents[-1]["id"]),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1803 self._m.get_comments_node(item_id),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1804 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1805 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1806 last_level_item = parents[parent_limit]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1807 return (self._m.get_comments_node(last_level_item["id"]), None)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1808
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1809 async def ap_item_2_mb_data(self, requestor_actor_id: str, ap_item: dict) -> dict:
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1810 """Convert AP activity or object to microblog data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1811
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1812 @param actor_id: ID of the actor doing the request.
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1813 @param ap_item: ActivityPub item to convert
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1814 Can be either an activity of an object
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1815 @return: AP Item's Object and microblog data
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1816 @raise exceptions.DataError: something is invalid in the AP item
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	1817 @raise NotImplementedError: some AP data is not handled yet
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1818 @raise error.StanzaError: error while contacting the AP server
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1819 """
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	1820 is_activity = self.is_activity(ap_item)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1821 if is_activity:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1822 ap_object = await self.ap_get_object(requestor_actor_id, ap_item, "object")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1823 if not ap_object:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1824 log.warning(f'No "object" found in AP item {ap_item!r}')
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1825 raise exceptions.DataError
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1826 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1827 ap_object = ap_item
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1828 item_id = ap_object.get("id")
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1829 if not item_id:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1830 log.warning(f'No "id" found in AP item: {ap_object!r}')
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1831 raise exceptions.DataError
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1832 mb_data = {"id": item_id, "extra": {}}
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1833
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1834 # content
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1835 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1836 language, content_xhtml = ap_object["contentMap"].popitem()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1837 except (KeyError, AttributeError):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1838 try:
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1839 mb_data["content_xhtml"] = ap_object["content"]
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1840 except KeyError:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1841 log.warning(f"no content found:\n{ap_object!r}")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1842 raise exceptions.DataError
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1843 else:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1844 mb_data["language"] = language
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1845 mb_data["content_xhtml"] = content_xhtml
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1846
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1847 mb_data["content"] = await self._t.convert(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1848 mb_data["content_xhtml"],
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1849 self._t.SYNTAX_XHTML,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1850 self._t.SYNTAX_TEXT,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1851 False,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	1852 )
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1853
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1854 if "attachment" in ap_object:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1855 attachments = mb_data["extra"][C.KEY_ATTACHMENTS] = []
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1856 for ap_attachment in ap_object["attachment"]:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1857 try:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1858 url = ap_attachment["url"]
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1859 except KeyError:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1860 log.warning(
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1861 f'"url" missing in AP attachment, ignoring: {ap_attachment}'
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1862 )
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1863 continue
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1864
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1865 if not url.startswith("http"):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1866 log.warning(f"non HTTP URL in attachment, ignoring: {ap_attachment}")
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1867 continue
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1868 attachment = {"url": url}
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1869 for ap_key, key in (
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1870 ("mediaType", "media_type"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1871 # XXX: as weird as it seems, "name" is actually used for description
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1872 # in AP world
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1873 ("name", "desc"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1874 ):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1875 value = ap_attachment.get(ap_key)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1876 if value:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1877 attachment[key] = value
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1878 attachments.append(attachment)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	1879
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1880 # author
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1881 if is_activity:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1882 authors = await self.ap_get_actors(requestor_actor_id, ap_item, "actor")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1883 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1884 authors = await self.ap_get_actors(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1885 requestor_actor_id, ap_object, "attributedTo"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1886 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1887 if len(authors) > 1:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1888 # we only keep first item as author
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1889 # TODO: handle multiple actors
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1890 log.warning("multiple actors are not managed")
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1891
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1892 account = authors[0]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1893 author_jid = self.get_local_jid_from_account(account).full()
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1894
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1895 mb_data["author"] = account.split("@", 1)[0]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1896 mb_data["author_jid"] = author_jid
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1897
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1898 # published/updated
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1899 for field in ("published", "updated"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1900 value = ap_object.get(field)
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1901 if not value and field == "updated":
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1902 value = ap_object.get("published")
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1903 if value:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1904 try:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1905 mb_data[field] = calendar.timegm(
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1906 dateutil.parser.parse(str(value)).utctimetuple()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1907 )
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1908 except dateutil.parser.ParserError as e:
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1909 log.warning(f"Can't parse {field!r} field: {e}")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1910
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1911 # repeat
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1912 if "_repeated" in ap_item:
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1913 mb_data["extra"]["repeated"] = ap_item["_repeated"]
65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	1914
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1915 # comments
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1916 in_reply_to = ap_object.get("inReplyTo")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1917 __, comments_node = await self.get_comments_nodes(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1918 requestor_actor_id, item_id, in_reply_to
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	1919 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1920 if comments_node is not None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1921 comments_data = {
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1922 "service": author_jid,
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1923 "node": comments_node,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1924 "uri": uri.build_xmpp_uri("pubsub", path=author_jid, node=comments_node),
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1925 }
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1926 mb_data["comments"] = [comments_data]
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1927
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	1928 return mb_data
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	1929
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1930 async def get_reply_to_id_from_xmpp_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1931 self, client: SatXMPPEntity, ap_account: str, parent_item: str, mb_data: dict
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1932 ) -> str:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1933 """Get URL to use for ``inReplyTo`` field in AP item.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1934
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1935 There is currently no way to know the parent service of a comment with XEP-0277.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1936 To work around that, we try to check if we have this item in the cache (we
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1937 should). If there is more that one item with this ID, we first try to find one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1938 with this author_jid. If nothing is found, we use ap_account to build `inReplyTo`.
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1939
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1940 @param ap_account: AP account corresponding to the publication author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1941 @param parent_item: ID of the node where the publication this item is replying to
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1942 has been posted
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1943 @param mb_data: microblog data of the publication
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1944 @return: URL to use in ``inReplyTo`` field
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1945 """
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1946 # FIXME: propose a protoXEP to properly get parent item, node and service
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1947
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1948 found_items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1949 {"profiles": [client.profile], "names": [parent_item]}
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1950 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1951 if not found_items:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1952 log.warning(f"parent item {parent_item!r} not found in cache")
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1953 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1954 elif len(found_items) == 1:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1955 cached_node = found_items[0].node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1956 parent_ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1957 cached_node.service, cached_node.name
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1958 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1959 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1960 # we found several cached item with given ID, we check if there is one
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1961 # corresponding to this author
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1962 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1963 author = jid.JID(mb_data["author_jid"]).userhostJID()
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1964 cached_item = next(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1965 i
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1966 for i in found_items
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1967 if jid.JID(i.data["publisher"]).userhostJID() == author
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1968 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1969 except StopIteration:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1970 # no item corresponding to this author, we use ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1971 log.warning(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1972 "Can't find a single cached item for parent item " f"{parent_item!r}"
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1973 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1974 parent_ap_account = ap_account
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1975 else:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1976 cached_node = cached_item.node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1977 parent_ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1978 cached_node.service, cached_node.name
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1979 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1980
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1981 return self.build_apurl(TYPE_ITEM, parent_ap_account, parent_item)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	1982
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1983 async def repeated_mb_2_ap_item(self, mb_data: dict) -> dict:
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1984 """Convert repeated blog item to suitable AP Announce activity
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1985
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1986 @param mb_data: microblog metadata of an item repeating an other blog post
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1987 @return: Announce activity linking to the repeated item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1988 """
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1989 repeated = mb_data["extra"]["repeated"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1990 repeater = jid.JID(repeated["by"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	1991 repeater_account = await self.get_ap_account_from_jid_and_node(repeater, None)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	1992 repeater_id = self.build_apurl(TYPE_ACTOR, repeater_account)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1993 repeated_uri = repeated["uri"]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1994
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1995 if not repeated_uri.startswith("xmpp:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1996 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1997 "Only xmpp: URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1998 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	1999 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2000 raise NotImplementedError
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2001 parsed_url = uri.parse_xmpp_uri(repeated_uri)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2002 if parsed_url["type"] != "pubsub":
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2003 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2004 "Only pubsub URL are handled for repeated item at the moment, ignoring "
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2005 f"item {mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2006 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2007 raise NotImplementedError
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2008 rep_service = jid.JID(parsed_url["path"])
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2009 rep_item = parsed_url["item"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2010 activity_id = self.build_apurl("item", repeater.userhost(), mb_data["id"])
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2011
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2012 if self.is_virtual_jid(rep_service):
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2013 # it's an AP actor linked through this gateway
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2014 # in this case we can simply use the item ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2015 if not rep_item.startswith("https:"):
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2016 log.warning(
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2017 f"Was expecting an HTTPS url as item ID and got {rep_item!r}\n"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2018 f"{mb_data}"
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2019 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2020 announced_uri = rep_item
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2021 repeated_account = self._e.unescape(rep_service.user)
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2022 else:
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2023 # the repeated item is an XMPP publication, we build the corresponding ID
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2024 rep_node = parsed_url["node"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2025 repeated_account = await self.get_ap_account_from_jid_and_node(
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2026 rep_service, rep_node
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2027 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2028 announced_uri = self.build_apurl("item", repeated_account, rep_item)
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2029
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2030 announce = self.create_activity(
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2031 "Announce", repeater_id, announced_uri, activity_id=activity_id
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2032 )
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2033 announce["to"] = [NS_AP_PUBLIC]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2034 announce["cc"] = [
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2035 self.build_apurl(TYPE_FOLLOWERS, repeater_account),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2036 await self.get_ap_actor_id_from_account(repeated_account),
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2037 ]
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2038 return announce
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2039
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2040 async def mb_data_2_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2041 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2042 client: SatXMPPEntity,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2043 mb_data: dict,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2044 public: bool = True,
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2045 is_new: bool = True,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2046 ) -> dict:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2047 """Convert Libervia Microblog Data to ActivityPub item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2048
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2049 @param mb_data: microblog data (as used in plugin XEP-0277) to convert
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2050 If ``public`` is True, ``service`` and ``node`` keys must be set.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2051 If ``published`` is not set, current datetime will be used
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2052 @param public: True if the message is not a private/direct one
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2053 if True, the AP Item will be marked as public, and AP followers of target AP
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2054 account (which retrieve from ``service``) will be put in ``cc``.
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2055 ``inReplyTo`` will also be set if suitable
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2056 if False, no destinee will be set (i.e., no ``to`` or ``cc`` or public flag).
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2057 This is usually used for direct messages.
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2058 @param is_new: if True, the item is a new one (no instance has been found in
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2059 cache).
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2060 If True, a "Create" activity will be generated, otherwise an "Update" one will
0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2061 be.
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2062 @return: Activity item
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2063 """
3855 54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2064 extra = mb_data.get("extra", {})
54305ebf5b94 component AP gateway: "repeat" to "Announce" conversion: Goffi <goffi@goffi.org> parents: 3853 diff changeset	2065 if "repeated" in extra:
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2066 return await self.repeated_mb_2_ap_item(mb_data)
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2067 if not mb_data.get("id"):
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2068 mb_data["id"] = shortuuid.uuid()
b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2069 if not mb_data.get("author_jid"):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2070 mb_data["author_jid"] = client.jid.userhost()
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2071 ap_account = await self.get_ap_account_from_jid_and_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2072 jid.JID(mb_data["author_jid"]), None
3728 b15644cae50d component AP gateway: JID/node ⟺ AP outbox conversion: Goffi <goffi@goffi.org> parents: 3684 diff changeset	2073 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2074 url_actor = self.build_apurl(TYPE_ACTOR, ap_account)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2075 url_item = self.build_apurl(TYPE_ITEM, ap_account, mb_data["id"])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2076 ap_object = {
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2077 "id": url_item,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2078 "type": "Note",
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2079 "published": utils.xmpp_date(mb_data.get("published")),
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2080 "attributedTo": url_actor,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2081 "content": mb_data.get("content_xhtml") or mb_data["content"],
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2082 }
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2083
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2084 language = mb_data.get("language")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2085 if language:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2086 ap_object["contentMap"] = {language: ap_object["content"]}
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2087
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2088 attachments = extra.get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2089 if attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2090 ap_attachments = ap_object["attachment"] = []
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2091 for attachment in attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2092 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2093 url = next(s["url"] for s in attachment["sources"] if "url" in s)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2094 except (StopIteration, KeyError):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2095 log.warning(f"Ignoring attachment without URL: {attachment}")
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2096 continue
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2097 ap_attachment = {"url": url}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2098 for key, ap_key in (
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2099 ("media_type", "mediaType"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2100 # XXX: yes "name", cf. [ap_item_2_mb_data]
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2101 ("desc", "name"),
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2102 ):
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2103 value = attachment.get(key)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2104 if value:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2105 ap_attachment[ap_key] = value
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2106 ap_attachments.append(ap_attachment)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2107
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2108 if public:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2109 ap_object["to"] = [NS_AP_PUBLIC]
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2110 if self.auto_mentions:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2111 for m in RE_MENTION.finditer(ap_object["content"]):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2112 mention = m.group()
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2113 mentioned = mention[1:]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2114 __, m_host = mentioned.split("@", 1)
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2115 if m_host in (self.public_url, self.client.jid.host):
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2116 # we ignore mention of local users, they should be sent as XMPP
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2117 # references
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2118 continue
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2119 try:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2120 mentioned_id = await self.get_ap_actor_id_from_account(mentioned)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2121 except Exception as e:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2122 log.warning(f"Can't add mention to {mentioned!r}: {e}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2123 else:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2124 ap_object["to"].append(mentioned_id)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2125 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2126 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2127 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2128 "href": mentioned_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2129 "name": mention,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2130 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2131 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2132 try:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2133 node = mb_data["node"]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2134 service = jid.JID(mb_data["service"])
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2135 except KeyError:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2136 # node and service must always be specified when this method is used
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2137 raise exceptions.InternalError("node or service is missing in mb_data")
4094 c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2138 try:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2139 target_ap_account = await self.get_ap_account_from_jid_and_node(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2140 service, node
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2141 )
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2142 except MissingLocalPartError:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2143 # we have no local part, this is probably a comment
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2144 try:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2145 service = jid.JID(mb_data["author_jid"])
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2146 if not mb_data["author_jid_verified"]:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2147 raise KeyError
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2148 except KeyError:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2149 raise exceptions.DataError(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2150 "Can't identify post author. Be sure to use a pubsub service "
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2151 "setting the 'publisher' attribute."
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2152 )
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2153 else:
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2154 target_ap_account = await self.get_ap_account_from_jid_and_node(
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2155 service, node
c3b68fdc2de7 component AP gateway: fix handling of XMPP comments authors: Goffi <goffi@goffi.org> parents: 4071 diff changeset	2156 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2157 if self.is_virtual_jid(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2158 # service is a proxy JID for AP account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2159 actor_data = await self.get_ap_actor_data_from_account(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2160 url_actor, target_ap_account
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2161 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2162 followers = actor_data.get("followers")
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2163 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2164 # service is a real XMPP entity
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2165 followers = self.build_apurl(TYPE_FOLLOWERS, target_ap_account)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2166 if followers:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2167 ap_object["cc"] = [followers]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2168 if self._m.is_comment_node(node):
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2169 parent_item = self._m.get_parent_item(node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2170 if self.is_virtual_jid(service):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2171 # the publication is on a virtual node (i.e. an XMPP node managed by
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2172 # this gateway and linking to an ActivityPub actor)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2173 ap_object["inReplyTo"] = parent_item
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2174 else:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2175 # the publication is from a followed real XMPP node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2176 ap_object["inReplyTo"] = await self.get_reply_to_id_from_xmpp_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2177 client, ap_account, parent_item, mb_data
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2178 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2179
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2180 return self.create_activity(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2181 "Create" if is_new else "Update", url_actor, ap_object, activity_id=url_item
3845 4f9d4650eab5 component AP gateway: use `createActivity` in `mbdata2APitem` Goffi <goffi@goffi.org> parents: 3844 diff changeset	2182 )
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2183
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2184 async def publish_message(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2185 self, client: SatXMPPEntity, mess_data: dict, service: jid.JID
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2186 ) -> None:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2187 """Send an AP message
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2188
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2189 .. note::
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2190
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2191 This is a temporary method used for development only
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2192
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2193 @param mess_data: message data. Following keys must be set:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2194
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2195 ``node``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2196 identifier of message which is being replied (this will
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2197 correspond to pubsub node in the future)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2198
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2199 ``content_xhtml`` or ``content``
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2200 message body (respectively in XHTML or plain text)
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2201
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2202 @param service: JID corresponding to the AP actor.
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2203 """
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2204 if not service.user:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2205 raise ValueError("service must have a local part")
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2206 account = self._e.unescape(service.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2207 ap_actor_data = await self.get_ap_actor_data_from_account(account)
3682 7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2208
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2209 try:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2210 inbox_url = ap_actor_data["endpoints"]["sharedInbox"]
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2211 except KeyError:
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2212 raise exceptions.DataError("Can't get ActivityPub actor inbox")
7c990aaa49d3 comp AP Gateway: ActivityPub Component first draft: Goffi <goffi@goffi.org> parents: diff changeset	2213
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2214 item_data = await self.mb_data_2_ap_item(client, mess_data)
3852 384ad98ea9fe component AP gateway: we need to use `actor` and not `attributedTo` for activities: Goffi <goffi@goffi.org> parents: 3851 diff changeset	2215 url_actor = item_data["actor"]
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2216 await self.ap_post(inbox_url, url_actor, item_data)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2217
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2218 async def ap_delete_item(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2219 self, jid_: jid.JID, node: Optional[str], item_id: str, public: bool = True
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2220 ) -> Tuple[str, Dict[str, Any]]:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2221 """Build activity to delete an AP item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2222
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2223 @param jid_: JID of the entity deleting an item
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2224 @param node: node where the item is deleted
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2225 None if it's microblog or a message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2226 @param item_id: ID of the item to delete
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2227 it's the Pubsub ID or message's origin ID
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2228 @param public: if True, the activity will be addressed to public namespace
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2229 @return: actor_id of the entity deleting the item, activity to send
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2230 """
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2231 if node is None:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2232 node = self._m.namespace
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2233
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2234 author_account = await self.get_ap_account_from_jid_and_node(jid_, node)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2235 author_actor_id = self.build_apurl(TYPE_ACTOR, author_account)
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2236
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2237 items = await self.host.memory.storage.search_pubsub_items(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2238 {"profiles": [self.client.profile], "services": [jid_], "names": [item_id]}
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2239 )
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2240 if not items:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2241 log.warning(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2242 f"Deleting an unknown item at service {jid_}, node {node} and id "
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2243 f"{item_id}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2244 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2245 else:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2246 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2247 mb_data = await self._m.item_2_mb_data(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2248 self.client, items[0].data, jid_, node
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2249 )
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2250 if "repeated" in mb_data["extra"]:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2251 # we are deleting a repeated item, we must translate this to an
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2252 # "Undo" of the "Announce" activity instead of a "Delete" one
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2253 announce = await self.repeated_mb_2_ap_item(mb_data)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2254 undo = self.create_activity("Undo", author_actor_id, announce)
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2255 return author_actor_id, undo
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2256 except Exception as e:
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2257 log.debug(
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2258 f"Can't parse item, maybe it's not a blog item: {e}\n"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2259 f"{items[0].toXml()}"
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2260 )
bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2261
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2262 url_item = self.build_apurl(TYPE_ITEM, author_account, item_id)
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2263 ap_item = self.create_activity(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2264 "Delete", author_actor_id, {"id": url_item, "type": TYPE_TOMBSTONE}
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2265 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2266 if public:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2267 ap_item["to"] = [NS_AP_PUBLIC]
3856 bc7f9d0a404f component AP gateway: when a repeated blog post is retracted, it converted to suitable Activity: Goffi <goffi@goffi.org> parents: 3855 diff changeset	2268 return author_actor_id, ap_item
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2269
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2270 def _message_received_trigger(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2271 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2272 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2273 message_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2274 post_treat: defer.Deferred,
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2275 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2276 """add the gateway workflow on post treatment"""
3979 996e0f84935e component AP gateway: log at debug level instead of warning when no client is set Goffi <goffi@goffi.org> parents: 3977 diff changeset	2277 if self.client is None:
996e0f84935e component AP gateway: log at debug level instead of warning when no client is set Goffi <goffi@goffi.org> parents: 3977 diff changeset	2278 log.debug(f"no client set, ignoring message: {message_elt.toXml()}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2279 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2280 post_treat.addCallback(
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2281 lambda mess_data: defer.ensureDeferred(self.on_message(client, mess_data))
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2282 )
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2283 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2284
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2285 async def on_message(self, client: SatXMPPEntity, mess_data: dict) -> dict:
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2286 """Called once message has been parsed
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2287
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2288 this method handle the conversion to AP items and posting
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2289 """
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2290 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2291 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2292 if mess_data["type"] not in ("chat", "normal"):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2293 log.warning(f"ignoring message with unexpected type: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2294 return mess_data
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2295 if not self.is_local(mess_data["from"]):
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2296 log.warning(f"ignoring non local message: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2297 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2298 if not mess_data["to"].user:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2299 log.warning(f"ignoring message addressed to gateway itself: {mess_data}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2300 return mess_data
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2301 requestor_actor_id = self.build_apurl(TYPE_ACTOR, mess_data["from"].userhost())
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2302
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2303 actor_account = self._e.unescape(mess_data["to"].user)
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2304 try:
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2305 actor_id = await self.get_ap_actor_id_from_account(actor_account)
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2306 except Exception as e:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2307 log.warning(f"Can't retrieve data on actor {actor_account}: {e}")
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2308 # TODO: send an error <message>
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2309 return mess_data
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2310 inbox = await self.get_ap_inbox_from_id(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2311 requestor_actor_id, actor_id, use_shared=False
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2312 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2313
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2314 try:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2315 language, message = next(iter(mess_data["message"].items()))
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2316 except (KeyError, StopIteration):
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2317 log.warning(f"ignoring empty message: {mess_data}")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2318 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2319
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2320 mb_data = {
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2321 "content": message,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2322 }
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2323 if language:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2324 mb_data["language"] = language
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2325 origin_id = mess_data["extra"].get("origin_id")
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2326 if origin_id:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2327 # we need to use origin ID when present to be able to retract the message
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2328 mb_data["id"] = origin_id
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2329 attachments = mess_data["extra"].get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2330 if attachments:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2331 mb_data["extra"] = {C.KEY_ATTACHMENTS: attachments}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2332
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2333 client = self.client.get_virtual_client(mess_data["from"])
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2334 ap_item = await self.mb_data_2_ap_item(client, mb_data, public=False)
4017 4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2335 ap_object = ap_item["object"]
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2336 ap_object["to"] = ap_item["to"] = [actor_id]
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2337 # we add a mention to direct message, otherwise peer is not notified in some AP
4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2338 # implementations (notably Mastodon), and the message may be missed easily.
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2339 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2340 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2341 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2342 "href": actor_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2343 "name": f"@{actor_account}",
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2344 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2345 )
4017 4a2c261646b6 component AP gateway: add mention to direct messages: Goffi <goffi@goffi.org> parents: 4014 diff changeset	2346
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2347 try:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2348 await self.ap_post(inbox, ap_item["actor"], ap_item)
4169 e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2349 except Exception as e:
e92c32014024 component AP gateway: log a warning instead of raising an exception in `onMessage`: Goffi <goffi@goffi.org> parents: 4166 diff changeset	2350 # TODO: send an error <message>
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2351 log.warning(f"Can't send message to {inbox}: {e}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2352 return mess_data
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2353
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2354 async def _on_message_retract(
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2355 self,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2356 client: SatXMPPEntity,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2357 message_elt: domish.Element,
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2358 retract_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2359 history: History,
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2360 ) -> bool:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2361 if client != self.client:
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2362 return True
d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2363 from_jid = jid.JID(message_elt["from"])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2364 if not self.is_local(from_jid):
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2365 log.debug(f"ignoring retract request from non local jid {from_jid}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2366 return False
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2367 requestor_actor_id = self.build_apurl(TYPE_ACTOR, from_jid.userhost())
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2368 to_jid = jid.JID(message_elt["to"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2369 if to_jid.host != self.client.jid.full() or not to_jid.user:
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2370 # to_jid should be a virtual JID from this gateway
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2371 raise exceptions.InternalError(f"Invalid destinee's JID: {to_jid.full()}")
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2372 ap_account = self._e.unescape(to_jid.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2373 actor_id = await self.get_ap_actor_id_from_account(ap_account)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2374 inbox = await self.get_ap_inbox_from_id(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2375 requestor_actor_id, actor_id, use_shared=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2376 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2377 url_actor, ap_item = await self.ap_delete_item(
4166 a1f7040b5a15 plugin XEP-0424: message retraction update: Goffi <goffi@goffi.org> parents: 4109 diff changeset	2378 from_jid.userhostJID(), None, retract_elt["id"], public=False
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2379 )
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2380 resp = await self.ap_post(inbox, url_actor, ap_item)
3803 d5f343939239 component AP gateway: message retractation => AP deletion Goffi <goffi@goffi.org> parents: 3793 diff changeset	2381 return False
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2382
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2383 async def _on_reference_received(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2384 self,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2385 client: SatXMPPEntity,
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2386 message_elt: domish.Element,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2387 reference_data: Dict[str, Union[str, int]],
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2388 ) -> bool:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2389 parsed_uri: dict = reference_data.get("parsed_uri")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2390 if not parsed_uri:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2391 log.warning(f"no parsed URI available in reference {reference_data}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2392 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2393
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2394 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2395 mentioned = jid.JID(parsed_uri["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2396 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2397 log.warning(f"invalid target: {reference_data['uri']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2398 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2399
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2400 if mentioned.host != self.client.jid.full() or not mentioned.user:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2401 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2402 f"ignoring mentioned user {mentioned}, it's not a JID mapping an AP "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2403 "account"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2404 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2405 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2406
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2407 ap_account = self._e.unescape(mentioned.user)
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2408 actor_id = await self.get_ap_actor_id_from_account(ap_account)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2409
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2410 parsed_anchor: dict = reference_data.get("parsed_anchor")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2411 if not parsed_anchor:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2412 log.warning(f"no XMPP anchor, ignoring reference {reference_data!r}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2413 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2414
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2415 if parsed_anchor["type"] != "pubsub":
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2416 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2417 f"ignoring reference with non pubsub anchor, this is not supported: "
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2418 "{reference_data!r}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2419 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2420 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2421
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2422 try:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2423 pubsub_service = jid.JID(parsed_anchor["path"])
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2424 except RuntimeError:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2425 log.warning(f"invalid anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2426 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2427 pubsub_node = parsed_anchor.get("node")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2428 if not pubsub_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2429 log.warning(f"missing pubsub node in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2430 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2431 pubsub_item = parsed_anchor.get("item")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2432 if not pubsub_item:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2433 log.warning(f"missing pubsub item in anchor: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2434 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2435
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2436 cached_node = await self.host.memory.storage.get_pubsub_node(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2437 client, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2438 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2439 if not cached_node:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2440 log.warning(f"Anchored node not found in cache: {reference_data['anchor']}")
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2441 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2442
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2443 cached_items, __ = await self.host.memory.storage.get_items(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2444 cached_node, item_ids=[pubsub_item]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2445 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2446 if not cached_items:
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2447 log.warning(
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2448 f"Anchored pubsub item not found in cache: {reference_data['anchor']}"
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2449 )
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2450 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2451
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2452 cached_item = cached_items[0]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2453
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2454 mb_data = await self._m.item_2_mb_data(
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2455 client, cached_item.data, pubsub_service, pubsub_node
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2456 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2457 ap_item = await self.mb_data_2_ap_item(client, mb_data)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2458 ap_object = ap_item["object"]
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2459 ap_object["to"] = [actor_id]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2460 ap_object.setdefault("tag", []).append(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2461 {
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2462 "type": TYPE_MENTION,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2463 "href": actor_id,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2464 "name": ap_account,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2465 }
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2466 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2467
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2468 requestor_actor_id = ap_item["actor"]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2469 inbox = await self.get_ap_inbox_from_id(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2470 requestor_actor_id, actor_id, use_shared=False
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2471 )
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2472
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2473 await self.ap_post(inbox, requestor_actor_id, ap_item)
3833 381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2474
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2475 return False
381340b9a9ee component AP gateway: convert XMPP mentions to AP: Goffi <goffi@goffi.org> parents: 3832 diff changeset	2476
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2477 async def new_reply_to_xmpp_item(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2478 self,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2479 client: SatXMPPEntity,
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2480 ap_item: dict,
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2481 targets: Dict[str, Set[str]],
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2482 mentions: List[Dict[str, str]],
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2483 ) -> None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2484 """We got an AP item which is a reply to an XMPP item"""
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2485 in_reply_to = ap_item["inReplyTo"]
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2486 url_type, url_args = self.parse_apurl(in_reply_to)
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2487 if url_type != "item":
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2488 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2489 "Ignoring AP item replying to an XMPP item with an unexpected URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2490 f"type({url_type!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2491 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2492 return
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2493 try:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2494 parent_item_account, parent_item_id = url_args[0], "/".join(url_args[1:])
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2495 except (IndexError, ValueError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2496 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2497 "Ignoring AP item replying to an XMPP item with invalid inReplyTo URL "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2498 f"({in_reply_to!r}):\n{pformat(ap_item)}"
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2499 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2500 return
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2501 parent_item_service, parent_item_node = await self.get_jid_and_node(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2502 parent_item_account
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2503 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2504 if parent_item_node is None:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2505 parent_item_node = self._m.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2506 items, __ = await self._p.get_items(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2507 client, parent_item_service, parent_item_node, item_ids=[parent_item_id]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2508 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2509 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2510 parent_item_elt = items[0]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2511 except IndexError:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2512 log.warning(
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2513 f"Can't find parent item at {parent_item_service} (node "
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2514 f"{parent_item_node!r})\n{pformat(ap_item)}"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2515 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2516 return
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2517 parent_item_parsed = await self._m.item_2_mb_data(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2518 client, parent_item_elt, parent_item_service, parent_item_node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2519 )
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2520 try:
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2521 comment_service = jid.JID(parent_item_parsed["comments"][0]["service"])
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2522 comment_node = parent_item_parsed["comments"][0]["node"]
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2523 except (KeyError, IndexError):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2524 # we don't have a comment node set for this item
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4051 diff changeset	2525 from libervia.backend.tools.xml_tools import pp_elt
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2526
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2527 log.info(f"{pp_elt(parent_item_elt.toXml())}")
3853 a56d5ad466b3 component AP gateway: fix wrong exception + use of mb_data["xml"] when it may be absent Goffi <goffi@goffi.org> parents: 3852 diff changeset	2528 raise NotImplementedError()
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2529 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2530 requestor_actor_id = self.build_apurl(
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2531 TYPE_ACTOR,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2532 await self.get_ap_account_from_jid_and_node(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2533 comment_service, comment_node
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2534 ),
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2535 )
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2536 __, item_elt = await self.ap_item_2_mb_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2537 requestor_actor_id, ap_item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2538 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2539 await self._p.publish(client, comment_service, comment_node, [item_elt])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2540 await self.notify_mentions(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2541 targets, mentions, comment_service, comment_node, item_elt["id"]
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2542 )
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2543
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2544 def get_ap_item_targets(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2545 self, item: Dict[str, Any]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2546 ) -> Tuple[bool, Dict[str, Set[str]], List[Dict[str, str]]]:
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2547 """Retrieve targets of an AP item, and indicate if it's a public one
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2548
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2549 @param item: AP object payload
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2550 @return: Are returned:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2551 - is_public flag, indicating if the item is world-readable
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2552 - a dict mapping target type to targets
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2553 """
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2554 targets: Dict[str, Set[str]] = {}
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2555 is_public = False
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2556 # TODO: handle "audience"
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2557 for key in ("to", "bto", "cc", "bcc"):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2558 values = item.get(key)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2559 if not values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2560 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2561 if isinstance(values, str):
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2562 values = [values]
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2563 for value in values:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2564 if value in PUBLIC_TUPLE:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2565 is_public = True
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2566 continue
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2567 if not value:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2568 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2569 if not self.is_local_url(value):
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2570 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2571 target_type = self.parse_apurl(value)[0]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2572 if target_type != TYPE_ACTOR:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2573 log.debug(f"ignoring non actor type as a target: {value}")
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2574 else:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2575 targets.setdefault(target_type, set()).add(value)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2576
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2577 mentions = []
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2578 tags = item.get("tag")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2579 if tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2580 for tag in tags:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2581 if tag.get("type") != TYPE_MENTION:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2582 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2583 href = tag.get("href")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2584 if not href:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2585 log.warning('Missing "href" field from mention object: {tag!r}')
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2586 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2587 if not self.is_local_url(href):
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2588 continue
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2589 uri_type = self.parse_apurl(href)[0]
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2590 if uri_type != TYPE_ACTOR:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2591 log.debug(f"ignoring non actor URI as a target: {href}")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2592 continue
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2593 mention = {"uri": href}
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2594 mentions.append(mention)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2595 name = tag.get("name")
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2596 if name:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2597 mention["content"] = name
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2598
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2599 return is_public, targets, mentions
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2600
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2601 async def new_ap_item(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2602 self,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2603 client: SatXMPPEntity,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2604 destinee: Optional[jid.JID],
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2605 node: str,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2606 item: dict,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2607 ) -> None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2608 """Analyse, cache and send notification for received AP item
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2609
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2610 @param destinee: jid of the destinee,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2611 @param node: XMPP pubsub node
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2612 @param item: AP object payload
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2613 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2614 is_public, targets, mentions = self.get_ap_item_targets(item)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2615 if not is_public and targets.keys() == {TYPE_ACTOR}:
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2616 # this is a direct message
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2617 await self.handle_message_ap_item(client, targets, mentions, destinee, item)
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2618 else:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2619 await self.handle_pubsub_ap_item(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2620 client, targets, mentions, destinee, node, item, is_public
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2621 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2622
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2623 def get_requestor_actor_id_from_targets(self, targets: set[str]) -> str:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2624 """Find local actor to use as requestor_actor_id from request targets.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2625
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2626 A local actor must be used to sign HTTP request, notably HTTP GET request for AP
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2627 instance checking signature, such as Mastodon when set in "secure mode".
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2628
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2629 This method check a set of targets and use the first local one.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2630
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2631 If none match, a generic local actor is used.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2632
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2633 @param targets: set of actor IDs to which the current request is sent.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2634 @return: local actor ID to use as requestor_actor_id.
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2635 """
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2636 try:
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2637 return next(t for t in targets if self.is_local_url(t))
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2638 except StopIteration:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2639 log.warning(f"Can't find local target to use as requestor ID: {targets!r}")
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2640 return self.build_apurl(TYPE_ACTOR, f"libervia@{self.public_url}")
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2641
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2642 async def handle_message_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2643 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2644 client: SatXMPPEntity,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2645 targets: dict[str, Set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2646 mentions: list[Dict[str, str]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2647 destinee: jid.JID \| None,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2648 item: dict,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2649 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2650 """Parse and deliver direct AP items translating to XMPP messages
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2651
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2652 @param targets: actors where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2653 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2654 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2655 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2656 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2657 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2658 targets_jids = {
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2659 await self.get_jid_from_id(requestor_actor_id, url) for url in targets_urls
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2660 }
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2661 if destinee is not None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2662 targets_jids.add(destinee)
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2663 mb_data = await self.ap_item_2_mb_data(requestor_actor_id, item)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2664 extra = {"origin_id": mb_data["id"]}
4023 78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2665 attachments = mb_data["extra"].get(C.KEY_ATTACHMENTS)
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2666 if attachments:
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2667 extra[C.KEY_ATTACHMENTS] = attachments
78b5f356900c component AP gateway: handle attachments Goffi <goffi@goffi.org> parents: 4017 diff changeset	2668
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2669 defer_l = []
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2670 for target_jid in targets_jids:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2671 defer_l.append(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2672 client.sendMessage(
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2673 target_jid,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2674 {"": mb_data.get("content", "")},
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2675 mb_data.get("title"),
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2676 extra=extra,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2677 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2678 )
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2679 await defer.DeferredList(defer_l)
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2680
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2681 async def notify_mentions(
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2682 self,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2683 targets: dict[str, set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2684 mentions: list[dict[str, str]],
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2685 service: jid.JID,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2686 node: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2687 item_id: str,
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2688 ) -> None:
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2689 """Send mention notifications to recipients and mentioned entities
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2690
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2691 XEP-0372 (References) is used.
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2692
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2693 Mentions are also sent to recipients as they are primary audience (see
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2694 https://www.w3.org/TR/activitystreams-vocabulary/#microsyntaxes).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2695
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2696 """
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2697 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2698 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2699 anchor = uri.build_xmpp_uri(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2700 "pubsub", path=service.full(), node=node, item=item_id
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2701 )
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2702 seen = set()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2703 # we start with explicit mentions because mentions' content will be used in the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2704 # future to fill "begin" and "end" reference attributes (we can't do it at the
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2705 # moment as there is no way to specify the XML element to use in the blog item).
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2706 for mention in mentions:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2707 mentioned_jid = await self.get_jid_from_id(requestor_actor_id, mention["uri"])
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2708 self._refs.send_reference(self.client, to_jid=mentioned_jid, anchor=anchor)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2709 seen.add(mentioned_jid)
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2710
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2711 remaining = {
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2712 await self.get_jid_from_id(requestor_actor_id, t)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2713 for t_set in targets.values()
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2714 for t in t_set
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2715 } - seen
201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2716 for target in remaining:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2717 self._refs.send_reference(self.client, to_jid=target, anchor=anchor)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2718
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2719 async def handle_pubsub_ap_item(
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2720 self,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2721 client: SatXMPPEntity,
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2722 targets: dict[str, set[str]],
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2723 mentions: list[dict[str, str]],
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2724 destinee: jid.JID \| None,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2725 node: str,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2726 item: dict,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2727 public: bool,
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2728 ) -> None:
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2729 """Analyse, cache and deliver AP items translating to Pubsub
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2730
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2731 @param targets: actors/collections where the item must be delivered
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2732 @param destinee: jid of the destinee,
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2733 @param node: XMPP pubsub node
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2734 @param item: AP object payload
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2735 @param public: True if the item is public
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2736 """
efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2737 # XXX: "public" is not used for now
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2738 targets_urls = {t for t_set in targets.values() for t in t_set}
49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2739 requestor_actor_id = self.get_requestor_actor_id_from_targets(targets_urls)
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2740 service = client.jid
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2741 in_reply_to = item.get("inReplyTo")
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2742
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2743 if in_reply_to and isinstance(in_reply_to, list):
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2744 in_reply_to = in_reply_to[0]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2745 if in_reply_to and isinstance(in_reply_to, str):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2746 if self.is_local_url(in_reply_to):
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2747 # this is a reply to an XMPP item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2748 await self.new_reply_to_xmpp_item(client, item, targets, mentions)
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2749 return
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2750
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2751 # this item is a reply to an AP item, we use or create a corresponding node
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2752 # for comments
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2753 parent_node, __ = await self.get_comments_nodes(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2754 requestor_actor_id, item["id"], in_reply_to
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2755 )
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2756 node = parent_node or node
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2757 cached_node = await self.host.memory.storage.get_pubsub_node(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2758 client,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2759 service,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2760 node,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2761 with_subscriptions=True,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2762 create=True,
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2763 create_kwargs={"subscribed": True},
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2764 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2765 else:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2766 # it is a root item (i.e. not a reply to an other item)
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2767 create = node == self._events.namespace
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2768 cached_node = await self.host.memory.storage.get_pubsub_node(
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2769 client, service, node, with_subscriptions=True, create=create
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2770 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2771 if cached_node is None:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2772 log.warning(
3764 125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2773 f"Received item in unknown node {node!r} at {service}. This may be "
125c7043b277 comp AP gateway: publish, (un)subscribe/(un)follow, public subscription/following/followers: Goffi <goffi@goffi.org> parents: 3745 diff changeset	2774 f"due to a cache purge. We synchronise the node\n{item}"
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2775 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2776 return
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2777 if item.get("type") == TYPE_EVENT:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2778 data, item_elt = await self.ap_events.ap_item_2_event_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2779 requestor_actor_id, item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2780 )
3904 0aa7023dcd08 component AP gateway: events: Goffi <goffi@goffi.org> parents: 3888 diff changeset	2781 else:
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2782 data, item_elt = await self.ap_item_2_mb_data_and_elt(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2783 requestor_actor_id, item
4259 49019947cc76 component AP Gateway: implement HTTP GET signature. Goffi <goffi@goffi.org> parents: 4183 diff changeset	2784 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2785 await self.host.memory.storage.cache_pubsub_items(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2786 client, cached_node, [item_elt], [data]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2787 )
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2788
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2789 for subscription in cached_node.subscriptions:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2790 if subscription.state != SubscriptionState.SUBSCRIBED:
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2791 continue
a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2792 self.pubsub_service.notifyPublish(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2793 service, node, [(subscription.subscriber, None, [item_elt])]
3745 a8c7e5cef0cb comp AP gateway: signature checking, caching and threads management: Goffi <goffi@goffi.org> parents: 3742 diff changeset	2794 )
3784 efc34a89e70b comp AP gateway: message conversion: Goffi <goffi@goffi.org> parents: 3764 diff changeset	2795
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2796 await self.notify_mentions(targets, mentions, service, node, item_elt["id"])
3832 201a22bfbb74 component AP gateway: convert AP mention to XEP-0372 mentions: Goffi <goffi@goffi.org> parents: 3824 diff changeset	2797
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2798 async def new_ap_delete_item(
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2799 self,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2800 client: SatXMPPEntity,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2801 destinee: Optional[jid.JID],
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2802 node: str,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2803 item: dict,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2804 ) -> None:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2805 """Analyse, cache and send notification for received AP item
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2806
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2807 @param destinee: jid of the destinee,
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2808 @param node: XMPP pubsub node
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2809 @param activity: parent AP activity
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2810 @param item: AP object payload
3844 65e5718e7710 component AP gateway: `Announce` activity implementation: Goffi <goffi@goffi.org> parents: 3843 diff changeset	2811 only the "id" field is used
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2812 """
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2813 item_id = item.get("id")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2814 if not item_id:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2815 raise exceptions.DataError('"id" attribute is missing in item')
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2816 if not item_id.startswith("http"):
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2817 raise exceptions.DataError(f"invalid id: {item_id!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2818 if self.is_local_url(item_id):
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2819 raise ValueError("Local IDs should not be used")
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2820
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2821 # we have no way to know if a deleted item is a direct one (thus a message) or one
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2822 # converted to pubsub. We check if the id is in message history to decide what to
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2823 # do.
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2824 history = await self.host.memory.storage.get(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2825 client,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2826 History,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2827 History.origin_id,
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2828 item_id,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2829 (History.messages, History.subjects),
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2830 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2831
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2832 if history is not None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2833 # it's a direct message
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2834 if history.source_jid != client.jid:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2835 log.warning(
3807 2032826cfbcf component AP gateway typing + remove unused `activity` arg from `newAPDeleteItem` Goffi <goffi@goffi.org> parents: 3804 diff changeset	2836 f"retraction received from an entity ''{client.jid}) which is "
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2837 f"not the original sender of the message ({history.source_jid}), "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2838 "hack attemps?"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2839 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2840 raise exceptions.PermissionError("forbidden")
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2841
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2842 await self._r.retract_by_history(client, history)
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2843 else:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2844 # no history in cache with this ID, it's probably a pubsub item
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2845 cached_node = await self.host.memory.storage.get_pubsub_node(
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2846 client, client.jid, node, with_subscriptions=True
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2847 )
3804 36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2848 if cached_node is None:
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2849 log.warning(
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2850 f"Received an item retract for node {node!r} at {client.jid} "
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2851 "which is not cached"
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2852 )
36b167ddbfca component AP gateway: AP delete activity => message retract: Goffi <goffi@goffi.org> parents: 3803 diff changeset	2853 raise exceptions.NotFound
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4027 diff changeset	2854 await self.host.memory.storage.delete_pubsub_items(cached_node, [item_id])
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2855 # notifyRetract is expecting domish.Element instances
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2856 item_elt = domish.Element((None, "item"))
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2857 item_elt["id"] = item_id
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2858 for subscription in cached_node.subscriptions:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2859 if subscription.state != SubscriptionState.SUBSCRIBED:
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2860 continue
b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2861 self.pubsub_service.notifyRetract(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4262 diff changeset	2862 client.jid, node, [(subscription.subscriber, None, [item_elt])]
3793 b5c9021020df component AP gateway: convert `Delete` AP activities to corresponding Pubsub `retract`: Goffi <goffi@goffi.org> parents: 3792 diff changeset	2863 )

Mercurial > libervia-backend

annotate libervia/backend/plugins/plugin_comp_ap_gateway/__init__.py @ 4306:94e0968987cd

annotate libervia/backend/plugins/plugin_comp_ap_gateway/init.py @ 4306:94e0968987cd