annotate libervia/backend/plugins/plugin_xep_0313.py @ 4306:94e0968987cd

plugin XEP-0033: code modernisation, improve delivery, data validation: - Code has been rewritten using Pydantic models and `async` coroutines for data validation and cleaner element parsing/generation. - Delivery has been completely rewritten. It now works even if server doesn't support multicast, and send to local multicast service first. Delivering to local multicast service first is due to bad support of XEP-0033 in server (notably Prosody which has an incomplete implementation), and the current impossibility to detect if a sub-domain service handles fully multicast or only for local domains. This is a workaround to have a good balance between backward compatilibity and use of bandwith, and to make it work with the incoming email gateway implementation (the gateway will only deliver to entities of its own domain). - disco feature checking now uses `async` corountines. `host` implementation still use Deferred return values for compatibility with legacy code. rel 450
author Goffi <goffi@goffi.org>
date Thu, 26 Sep 2024 16:12:01 +0200
parents 0d7bb4df2343
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
3
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT plugin for Message Archive Management (XEP-0313)
3479
be6d91572633 date update
Goffi <goffi@goffi.org>
parents: 3447
diff changeset
5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1409
diff changeset
6 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
7
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # This program is free software: you can redistribute it and/or modify
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # it under the terms of the GNU Affero General Public License as published by
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # the Free Software Foundation, either version 3 of the License, or
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
11 # (at your option) any later version.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
12
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # This program is distributed in the hope that it will be useful,
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
16 # GNU Affero General Public License for more details.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
17
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # You should have received a copy of the GNU Affero General Public License
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
19 # along with this program. If not, see <http://www.gnu.org/licenses/>.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
20
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
21 from datetime import datetime
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
22 import uuid
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
23
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
24 from dateutil import tz
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
25 from twisted.internet import defer
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
26 from twisted.words.protocols.jabber import jid
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
27 from twisted.words.protocols.jabber.error import StanzaError
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
28 from wokkel import disco
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
29 from wokkel import data_form
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
30 from wokkel import rsm
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
31 from wokkel import mam
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
32 from zope.interface import implementer
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
33
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
34 from libervia.backend.core import exceptions
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
35 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
36 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
37 from libervia.backend.core.log import getLogger
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
38 from libervia.backend.tools import xml_tools
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
39 from libervia.backend.tools.common import data_format
1285
ed2c718bfe03 tmp, plugins: fixes the imports fron sat.tmp
souliane <souliane@mailoo.org>
parents: 1284
diff changeset
40
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
41
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
42 log = getLogger(__name__)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
43
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
44 PLUGIN_INFO = {
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
45 C.PI_NAME: "Message Archive Management",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
46 C.PI_IMPORT_NAME: "XEP-0313",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
47 C.PI_TYPE: "XEP",
3447
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
48 # XEP-0431 only defines a namespace, so we register it here
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
49 C.PI_PROTOCOLS: ["XEP-0313", "XEP-0431"],
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
50 C.PI_DEPENDENCIES: ["XEP-0059", "XEP-0359"],
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
51 C.PI_MAIN: "XEP_0313",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
52 C.PI_HANDLER: "yes",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
53 C.PI_DESCRIPTION: _("""Implementation of Message Archive Management"""),
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
54 }
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
55
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
56 MAM_PREFIX = "mam_"
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
57 FILTER_PREFIX = MAM_PREFIX + "filter_"
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
58 KEY_LAST_STANZA_ID = "last_stanza_id"
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
59 MESSAGE_RESULT = "/message/result[@xmlns='{mam_ns}' and @queryid='{query_id}']"
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
60 MESSAGE_STANZA_ID = '/message/stanza-id[@xmlns="{ns_stanza_id}"]'
3447
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
61 NS_FTS = "urn:xmpp:fulltext:0"
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
62
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
63
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
64 class XEP_0313(object):
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
65 def __init__(self, host):
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
66 log.info(_("Message Archive Management plugin initialization"))
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
67 self.host = host
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
68 self.host.register_namespace("mam", mam.NS_MAM)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
69 host.register_namespace("fulltextmam", NS_FTS)
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
70 self._rsm = host.plugins["XEP-0059"]
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
71 self._sid = host.plugins["XEP-0359"]
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
72 # Deferred used to store last stanza id in order of reception
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
73 self._last_stanza_id_d = defer.Deferred()
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
74 self._last_stanza_id_d.callback(None)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
75 host.bridge.add_method(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
76 "mam_get",
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
77 ".plugin",
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
78 in_sign="sss",
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
79 out_sign="(a(sdssa{ss}a{ss}ss)ss)",
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
80 method=self._get_archives,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
81 async_=True,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
82 )
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
83
3541
888109774673 core: various changes and fixes to work with new storage and D-Bus bridge:
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
84 async def resume(self, client):
3011
93da7c6f8e0c plugin XEP-0198: retrieve missing messages + send buffered ones on hot reconnection:
Goffi <goffi@goffi.org>
parents: 2930
diff changeset
85 """Retrieve one2one messages received since the last we have in local storage"""
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
86 stanza_id_data = await self.host.memory.storage.get_privates(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
87 mam.NS_MAM, [KEY_LAST_STANZA_ID], profile=client.profile
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
88 )
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
89 stanza_id = stanza_id_data.get(KEY_LAST_STANZA_ID)
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
90 rsm_req = None
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
91 if stanza_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
92 log.info("can't retrieve last stanza ID, checking history")
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
93 last_mess = await self.host.memory.history_get(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
94 None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
95 None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
96 limit=1,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
97 filters={"not_types": C.MESS_TYPE_GROUPCHAT, "last_stanza_id": True},
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
98 profile=client.profile,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
99 )
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
100 if not last_mess:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
101 log.info(_("It seems that we have no MAM history yet"))
3357
40794e658d68 plugin XEP-0313: fix getting MAM archive when history is empty
Goffi <goffi@goffi.org>
parents: 3356
diff changeset
102 stanza_id = None
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
103 rsm_req = rsm.RSMRequest(max_=50, before="")
3357
40794e658d68 plugin XEP-0313: fix getting MAM archive when history is empty
Goffi <goffi@goffi.org>
parents: 3356
diff changeset
104 else:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
105 stanza_id = last_mess[0][-1]["stanza_id"]
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
106 if rsm_req is None:
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
107 rsm_req = rsm.RSMRequest(max_=100, after=stanza_id)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
108 mam_req = mam.MAMRequest(rsm_=rsm_req)
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
109 complete = False
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
110 count = 0
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
111 while not complete:
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
112 try:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
113 mam_data = await self.get_archives(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
114 client, mam_req, service=client.jid.userhostJID()
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
115 )
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
116 except StanzaError as e:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
117 log.warning(
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
118 f"Can't retrieve MAM archives: {e}\n"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
119 f"{xml_tools.pp_elt(mam_req.toElement())}\n"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
120 f"{xml_tools.pp_elt(e.stanza)}"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
121 )
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
122 return
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
123 except Exception as e:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
124 log.exception(f"Can't retrieve retrieve MAM archive")
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
125 return
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
126 elt_list, rsm_response, mam_response = mam_data
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
127 complete = mam_response["complete"]
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
128 # we update MAM request for next iteration
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
129 mam_req.rsm.after = rsm_response.last
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
130 # before may be set if we had no previous history
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
131 mam_req.rsm.before = None
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
132 if not elt_list:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
133 break
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
134 else:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
135 count += len(elt_list)
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
136
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
137 for idx, mess_elt in enumerate(elt_list):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
138 try:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
139 fwd_message_elt = self.get_message_from_result(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
140 client, mess_elt, mam_req
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
141 )
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
142 except exceptions.DataError:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
143 continue
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
144
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
145 try:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
146 destinee = jid.JID(fwd_message_elt["to"])
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
147 except KeyError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
148 log.warning(_('missing "to" attribute in forwarded message'))
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
149 destinee = client.jid
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
150 if destinee.userhostJID() == client.jid.userhostJID():
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
151 # message to use, we insert the forwarded message in the normal
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
152 # workflow
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
153 client.xmlstream.dispatch(fwd_message_elt)
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
154 else:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
155 # this message should be from us, we just add it to history
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
156 try:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
157 from_jid = jid.JID(fwd_message_elt["from"])
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
158 except KeyError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
159 log.warning(_('missing "from" attribute in forwarded message'))
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
160 from_jid = client.jid
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
161 if from_jid.userhostJID() != client.jid.userhostJID():
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
162 log.warning(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
163 _(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
164 "was expecting a message sent by our jid, but this one if "
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
165 "from {from_jid}, ignoring\n{xml}"
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
166 ).format(from_jid=from_jid.full(), xml=mess_elt.toXml())
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
167 )
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
168 continue
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
169 # adding message to history
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
170 mess_data = client.messageProt.parse_message(fwd_message_elt)
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
171 try:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
172 await client.messageProt.add_to_history(mess_data)
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
173 except exceptions.CancelError as e:
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
174 log.warning(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
175 "message has not been added to history: {e}".format(e=e)
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
176 )
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
177 except Exception as e:
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
178 log.error(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
179 "can't add message to history: {e}\n{xml}".format(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
180 e=e, xml=mess_elt.toXml()
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
181 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
182 )
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
183 if complete and idx == len(elt_list) - 1:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
184 # We are at the last message from archive, we store the ID to not
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
185 # ask again the same messages next time.
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
186 try:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
187 stanza_id = mess_elt.result["id"]
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
188 await self.host.memory.storage.set_private_value(
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
189 namespace=mam.NS_MAM,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
190 key=KEY_LAST_STANZA_ID,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
191 value=stanza_id,
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
192 profile=client.profile,
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
193 )
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
194 except Exception:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
195 log.exception("Can't store last stanza ID")
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
196
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
197 if not count:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
198 log.info(_("We have received no message while offline"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
199 else:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
200 log.info(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
201 _("We have received {num_mess} message(s) while offline.").format(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
202 num_mess=count
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
203 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
204 )
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
205
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
206 def profile_connected(self, client):
4002
5245b675f7ad plugin XEP-0313: don't wait for MAM to be retrieved in connection workflow:
Goffi <goffi@goffi.org>
parents: 3573
diff changeset
207 defer.ensureDeferred(self.resume(client))
3011
93da7c6f8e0c plugin XEP-0198: retrieve missing messages + send buffered ones on hot reconnection:
Goffi <goffi@goffi.org>
parents: 2930
diff changeset
208
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
209 def get_handler(self, client):
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
210 mam_client = client._mam = LiberviaMAMClient(self)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
211 return mam_client
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
212
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
213 def parse_extra(self, extra, with_rsm=True):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
214 """Parse extra dictionnary to retrieve MAM arguments
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
215
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
216 @param extra(dict): data for parse
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
217 @param with_rsm(bool): if True, RSM data will be parsed too
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
218 @return (data_form, None): request with parsed arguments
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
219 or None if no MAM arguments have been found
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
220 """
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
221 mam_args = {}
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
222 form_args = {}
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
223 for arg in ("start", "end"):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
224 try:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
225 value = extra.pop(MAM_PREFIX + arg)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
226 form_args[arg] = datetime.fromtimestamp(float(value), tz.tzutc())
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
227 except (TypeError, ValueError):
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
228 log.warning(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
229 "Bad value for {arg} filter ({value}), ignoring".format(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
230 arg=arg, value=value
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
231 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
232 )
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
233 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
234 continue
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
235
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
236 try:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
237 form_args["with_jid"] = jid.JID(extra.pop(MAM_PREFIX + "with"))
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
238 except jid.InvalidFormat:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
239 log.warning("Bad value for jid filter")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
240 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
241 pass
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
242
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
243 for name, value in extra.items():
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
244 if name.startswith(FILTER_PREFIX):
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
245 var = name[len(FILTER_PREFIX) :]
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
246 extra_fields = form_args.setdefault("extra_fields", [])
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
247 extra_fields.append(data_form.Field(var=var, value=value))
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
248
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
249 for arg in ("node", "query_id"):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
250 try:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
251 value = extra.pop(MAM_PREFIX + arg)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
252 mam_args[arg] = value
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
253 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
254 continue
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
255
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
256 if with_rsm:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
257 rsm_request = self._rsm.parse_extra(extra)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
258 if rsm_request is not None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
259 mam_args["rsm_"] = rsm_request
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
260
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
261 if form_args:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
262 mam_args["form"] = mam.buildForm(**form_args)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
263
2760
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
264 # we only set orderBy if we have other MAM args
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
265 # else we would make a MAM query while it's not expected
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
266 if "order_by" in extra and mam_args:
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
267 order_by = extra.pop("order_by")
2760
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
268 assert isinstance(order_by, list)
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
269 mam_args["orderBy"] = order_by
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
270
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
271 return mam.MAMRequest(**mam_args) if mam_args else None
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
272
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
273 def get_message_from_result(self, client, mess_elt, mam_req, service=None):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
274 """Extract usable <message/> from MAM query result
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
275
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
276 The message will be validated, and stanza-id/delay will be added if necessary.
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
277 @param mess_elt(domish.Element): result <message/> element wrapping the message
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
278 to retrieve
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
279 @param mam_req(mam.MAMRequest): request used (needed to get query_id)
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
280 @param service(jid.JID, None): MAM service where the request has been sent
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
281 None if it's user server
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
282 @return domish.Element): <message/> that can be used directly with onMessage
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
283 """
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
284 if mess_elt.name != "message":
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
285 log.warning(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
286 "unexpected stanza in archive: {xml}".format(xml=mess_elt.toXml())
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
287 )
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
288 raise exceptions.DataError("Invalid element")
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
289 service_jid = client.jid.userhostJID() if service is None else service
4222
1c30d574df2b plugin XEP-0313: fix value used in `msg_from` when `from` is not explicitely specified.
Goffi <goffi@goffi.org>
parents: 4154
diff changeset
290 mess_from = mess_elt.getAttribute("from") or client.jid.userhost()
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
291 # we check that the message has been sent by the right service
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
292 # if service is None (i.e. message expected from our own server)
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
293 # from can be server jid or user's bare jid
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
294 if mess_from != service_jid.full() and not (
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
295 service is None and mess_from == client.jid.host
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
296 ):
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
297 log.error(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
298 "Message is not from our server, something went wrong: "
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
299 "{xml}".format(xml=mess_elt.toXml())
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
300 )
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
301 raise exceptions.DataError("Invalid element")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
302 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
303 result_elt = next(mess_elt.elements(mam.NS_MAM, "result"))
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
304 forwarded_elt = next(result_elt.elements(C.NS_FORWARD, "forwarded"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
305 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
306 delay_elt = next(forwarded_elt.elements(C.NS_DELAY, "delay"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
307 except StopIteration:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
308 # delay_elt is not mandatory
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
309 delay_elt = None
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
310 fwd_message_elt = next(forwarded_elt.elements(C.NS_CLIENT, "message"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
311 except StopIteration:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
312 log.warning(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
313 "Invalid message received from MAM: {xml}".format(xml=mess_elt.toXml())
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
314 )
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
315 raise exceptions.DataError("Invalid element")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
316 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
317 if not result_elt["queryid"] == mam_req.query_id:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
318 log.error(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
319 "Unexpected query id (was expecting {query_id}): {xml}".format(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
320 query_id=mam.query_id, xml=mess_elt.toXml()
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
321 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
322 )
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
323 raise exceptions.DataError("Invalid element")
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
324 stanza_id = self._sid.get_stanza_id(fwd_message_elt, service_jid)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
325 if stanza_id is None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
326 # not stanza-id element is present, we add one so message
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
327 # will be archived with it, and we won't request several times
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
328 # the same MAM achive
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
329 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
330 stanza_id = result_elt["id"]
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
331 except AttributeError:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
332 log.warning(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
333 'Invalid MAM result: missing "id" attribute: {xml}'.format(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
334 xml=result_elt.toXml()
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
335 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
336 )
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
337 raise exceptions.DataError("Invalid element")
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
338 self._sid.add_stanza_id(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
339 client, fwd_message_elt, stanza_id, by=service_jid
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
340 )
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
341
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
342 if delay_elt is not None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
343 fwd_message_elt.addChild(delay_elt)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
344
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
345 return fwd_message_elt
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
346
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
347 def queryFields(self, client, service=None):
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
348 """Ask the server about supported fields.
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
349
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
350 @param service: entity offering the MAM service (None for user archives)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
351 @return (D(data_form.Form)): form with the implemented fields (cf XEP-0313 §4.1.5)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
352 """
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
353 return client._mam.queryFields(service)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
354
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
355 def queryArchive(self, client, mam_req, service=None):
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
356 """Query a user, MUC or pubsub archive.
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
357
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
358 @param mam_req(mam.MAMRequest): MAM query instance
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
359 @param service(jid.JID, None): entity offering the MAM service
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
360 None for user server
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
361 @return (D(domish.Element)): <IQ/> result
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
362 """
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
363 return client._mam.queryArchive(mam_req, service)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
364
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
365 def _append_message(self, elt_list, message_cb, message_elt):
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
366 if message_cb is not None:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
367 elt_list.append(message_cb(message_elt))
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
368 else:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
369 elt_list.append(message_elt)
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
370
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
371 def _query_finished(self, iq_result, client, elt_list, event):
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
372 client.xmlstream.removeObserver(event, self._append_message)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
373 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
374 fin_elt = next(iq_result.elements(mam.NS_MAM, "fin"))
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
375 except StopIteration:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
376 raise exceptions.DataError("Invalid MAM result")
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
377
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
378 mam_response = {
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
379 "complete": C.bool(fin_elt.getAttribute("complete", C.BOOL_FALSE)),
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
380 "stable": C.bool(fin_elt.getAttribute("stable", C.BOOL_TRUE)),
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
381 }
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
382
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
383 try:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
384 rsm_response = rsm.RSMResponse.fromElement(fin_elt)
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
385 except rsm.RSMNotFoundError:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
386 rsm_response = None
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
387
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
388 return (elt_list, rsm_response, mam_response)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
389
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
390 def serialize_archive_result(self, data, client, mam_req, service):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
391 elt_list, rsm_response, mam_response = data
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
392 mess_list = []
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
393 for elt in elt_list:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
394 fwd_message_elt = self.get_message_from_result(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
395 client, elt, mam_req, service=service
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
396 )
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
397 mess_data = client.messageProt.parse_message(fwd_message_elt)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
398 mess_list.append(client.message_get_bridge_args(mess_data))
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
399 metadata = {"rsm": self._rsm.response2dict(rsm_response), "mam": mam_response}
3356
569f4cf7183b plugin XEP-0313: fixed `MAMGet` signature
Goffi <goffi@goffi.org>
parents: 3308
diff changeset
400 return mess_list, data_format.serialise(metadata), client.profile
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
401
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
402 def _get_archives(self, service, extra_ser, profile_key):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
403 """
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
404 @return: tuple with:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
405 - list of message with same data as in bridge.message_new
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
406 - response metadata with:
3308
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
407 - rsm data (first, last, count, index)
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
408 - mam data (complete, stable)
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
409 - profile
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
410 """
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
411 client = self.host.get_client(profile_key)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
412 service = jid.JID(service) if service else None
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
413 extra = data_format.deserialise(extra_ser, {})
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
414 mam_req = self.parse_extra(extra)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
415
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
416 d = self.get_archives(client, mam_req, service=service)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
417 d.addCallback(self.serialize_archive_result, client, mam_req, service)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
418 return d
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
419
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
420 def get_archives(self, client, query, service=None, message_cb=None):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
421 """Query archive and gather page result
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
422
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
423 @param query(mam.MAMRequest): MAM request
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
424 @param service(jid.JID, None): MAM service to use
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
425 None to use our own server
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
426 @param message_cb(callable, None): callback to use on each message
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
427 this method can be used to unwrap messages
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
428 @return (tuple[list[domish.Element], rsm.RSMResponse, dict): result data with:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
429 - list of found elements
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
430 - RSM response
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
431 - MAM response, which is a dict with following value:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
432 - complete: a boolean which is True if all items have been received
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
433 - stable: a boolean which is False if items order may be changed
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
434 """
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
435 if query.query_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
436 query.query_id = str(uuid.uuid4())
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
437 elt_list = []
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
438 event = MESSAGE_RESULT.format(mam_ns=mam.NS_MAM, query_id=query.query_id)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
439 client.xmlstream.addObserver(event, self._append_message, 0, elt_list, message_cb)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
440 d = self.queryArchive(client, query, service)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
441 d.addCallback(self._query_finished, client, elt_list, event)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
442 return d
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
443
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
444 def get_prefs(self, client, service=None):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
445 """Retrieve the current user preferences.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
446
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
447 @param service: entity offering the MAM service (None for user archives)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
448 @return: the server response as a Deferred domish.Element
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
449 """
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
450 # http://xmpp.org/extensions/xep-0313.html#prefs
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
451 return client._mam.queryPrefs(service)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
452
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
453 def _set_prefs(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
454 self,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
455 service_s=None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
456 default="roster",
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
457 always=None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
458 never=None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
459 profile_key=C.PROF_KEY_NONE,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
460 ):
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
461 service = jid.JID(service_s) if service_s else None
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
462 always_jid = [jid.JID(entity) for entity in always]
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
463 never_jid = [jid.JID(entity) for entity in never]
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
464 # TODO: why not build here a MAMPrefs object instead of passing the args separately?
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
465 return self.setPrefs(service, default, always_jid, never_jid, profile_key)
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
466
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
467 def setPrefs(self, client, service=None, default="roster", always=None, never=None):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
468 """Set news user preferences.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
469
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
470 @param service: entity offering the MAM service (None for user archives)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
471 @param default (unicode): a value in ('always', 'never', 'roster')
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
472 @param always (list): a list of JID instances
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
473 @param never (list): a list of JID instances
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
474 @param profile_key (unicode): %(doc_profile_key)s
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
475 @return: the server response as a Deferred domish.Element
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
476 """
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
477 # http://xmpp.org/extensions/xep-0313.html#prefs
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
478 return client._mam.setPrefs(service, default, always, never)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
479
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
480 def on_message_stanza_id(self, message_elt, client):
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
481 """Called when a message with a stanza-id is received
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
482
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
483 the messages' stanza ids are stored when received, so the last one can be used
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
484 to retrieve missing history on next connection
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
485 @param message_elt(domish.Element): <message> with a stanza-id
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
486 """
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
487 if message_elt.getAttribute("type") == C.MESS_TYPE_GROUPCHAT:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
488 # groupchat message MAM is handled by XEP-0045
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
489 return
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
490 service_jid = client.jid.userhostJID()
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
491 stanza_id = self._sid.get_stanza_id(message_elt, service_jid)
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
492 if stanza_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
493 log.debug("Ignoring <message>, stanza id is not from our server")
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
494 else:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
495 # we use self._last_stanza_id_d do be sure that last_stanza_id is stored in
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
496 # the order of reception
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
497 self._last_stanza_id_d.addCallback(
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
498 lambda __: self.host.memory.storage.set_private_value(
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
499 namespace=mam.NS_MAM,
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
500 key=KEY_LAST_STANZA_ID,
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
501 value=stanza_id,
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
502 profile=client.profile,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
503 )
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
504 )
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
505
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
506
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
507 @implementer(disco.IDisco)
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
508 class LiberviaMAMClient(mam.MAMClient):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
509
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
510 def __init__(self, plugin_parent):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
511 self.plugin_parent = plugin_parent
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
512
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
513 @property
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
514 def host(self):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
515 return self.parent.host_app
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
516
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
517 def connectionInitialized(self):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
518 observer_xpath = MESSAGE_STANZA_ID.format(
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
519 ns_stanza_id=self.host.ns_map["stanza_id"]
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4222
diff changeset
520 )
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
521 self.xmlstream.addObserver(
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
522 observer_xpath, self.plugin_parent.on_message_stanza_id, client=self.parent
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
523 )
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
524
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
525 def getDiscoInfo(self, requestor, target, nodeIdentifier=""):
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
526 return [disco.DiscoFeature(mam.NS_MAM)]
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
527
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
528 def getDiscoItems(self, requestor, target, nodeIdentifier=""):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
529 return []