annotate libervia/backend/plugins/plugin_xep_0313.py @ 4201:6a8cc6d668a4

docker (demo): create file-sharing and demo account in init script
author Goffi <goffi@goffi.org>
date Thu, 14 Dec 2023 09:19:32 +0100
parents 85f5e6225aa1
children 1c30d574df2b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
3
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT plugin for Message Archive Management (XEP-0313)
3479
be6d91572633 date update
Goffi <goffi@goffi.org>
parents: 3447
diff changeset
5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1409
diff changeset
6 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
7
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # This program is free software: you can redistribute it and/or modify
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # it under the terms of the GNU Affero General Public License as published by
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # the Free Software Foundation, either version 3 of the License, or
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
11 # (at your option) any later version.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
12
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # This program is distributed in the hope that it will be useful,
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
16 # GNU Affero General Public License for more details.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
17
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # You should have received a copy of the GNU Affero General Public License
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
19 # along with this program. If not, see <http://www.gnu.org/licenses/>.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
20
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
21 from datetime import datetime
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
22 import uuid
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
23
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
24 from dateutil import tz
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
25 from twisted.internet import defer
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
26 from twisted.words.protocols.jabber import jid
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
27 from twisted.words.protocols.jabber.error import StanzaError
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
28 from wokkel import disco
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
29 from wokkel import data_form
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
30 from wokkel import rsm
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
31 from wokkel import mam
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
32 from zope.interface import implementer
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
33
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
34 from libervia.backend.core import exceptions
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
35 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
36 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
37 from libervia.backend.core.log import getLogger
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
38 from libervia.backend.tools import xml_tools
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
39 from libervia.backend.tools.common import data_format
1285
ed2c718bfe03 tmp, plugins: fixes the imports fron sat.tmp
souliane <souliane@mailoo.org>
parents: 1284
diff changeset
40
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
41
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
42 log = getLogger(__name__)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
43
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
44 PLUGIN_INFO = {
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
45 C.PI_NAME: "Message Archive Management",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
46 C.PI_IMPORT_NAME: "XEP-0313",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
47 C.PI_TYPE: "XEP",
3447
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
48 # XEP-0431 only defines a namespace, so we register it here
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
49 C.PI_PROTOCOLS: ["XEP-0313", "XEP-0431"],
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
50 C.PI_DEPENDENCIES: ["XEP-0059", "XEP-0359"],
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
51 C.PI_MAIN: "XEP_0313",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
52 C.PI_HANDLER: "yes",
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
53 C.PI_DESCRIPTION: _("""Implementation of Message Archive Management"""),
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
54 }
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
55
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
56 MAM_PREFIX = "mam_"
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
57 FILTER_PREFIX = MAM_PREFIX + "filter_"
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
58 KEY_LAST_STANZA_ID = "last_stanza_id"
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
59 MESSAGE_RESULT = "/message/result[@xmlns='{mam_ns}' and @queryid='{query_id}']"
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
60 MESSAGE_STANZA_ID = '/message/stanza-id[@xmlns="{ns_stanza_id}"]'
3447
c3f07c0cb2de plugin XEP-0431: Fulltext Search for MAM implementation
Goffi <goffi@goffi.org>
parents: 3359
diff changeset
61 NS_FTS = "urn:xmpp:fulltext:0"
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
62
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
63
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
64 class XEP_0313(object):
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
65 def __init__(self, host):
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
66 log.info(_("Message Archive Management plugin initialization"))
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
67 self.host = host
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
68 self.host.register_namespace("mam", mam.NS_MAM)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
69 host.register_namespace("fulltextmam", NS_FTS)
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
70 self._rsm = host.plugins["XEP-0059"]
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
71 self._sid = host.plugins["XEP-0359"]
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
72 # Deferred used to store last stanza id in order of reception
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
73 self._last_stanza_id_d = defer.Deferred()
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
74 self._last_stanza_id_d.callback(None)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
75 host.bridge.add_method(
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
76 "mam_get", ".plugin", in_sign='sss',
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
77 out_sign='(a(sdssa{ss}a{ss}ss)ss)', method=self._get_archives,
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
78 async_=True)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
79
3541
888109774673 core: various changes and fixes to work with new storage and D-Bus bridge:
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
80 async def resume(self, client):
3011
93da7c6f8e0c plugin XEP-0198: retrieve missing messages + send buffered ones on hot reconnection:
Goffi <goffi@goffi.org>
parents: 2930
diff changeset
81 """Retrieve one2one messages received since the last we have in local storage"""
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
82 stanza_id_data = await self.host.memory.storage.get_privates(
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
83 mam.NS_MAM, [KEY_LAST_STANZA_ID], profile=client.profile)
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
84 stanza_id = stanza_id_data.get(KEY_LAST_STANZA_ID)
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
85 rsm_req = None
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
86 if stanza_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
87 log.info("can't retrieve last stanza ID, checking history")
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
88 last_mess = await self.host.memory.history_get(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
89 None, None, limit=1, filters={'not_types': C.MESS_TYPE_GROUPCHAT,
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
90 'last_stanza_id': True},
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
91 profile=client.profile)
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
92 if not last_mess:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
93 log.info(_("It seems that we have no MAM history yet"))
3357
40794e658d68 plugin XEP-0313: fix getting MAM archive when history is empty
Goffi <goffi@goffi.org>
parents: 3356
diff changeset
94 stanza_id = None
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
95 rsm_req = rsm.RSMRequest(max_=50, before="")
3357
40794e658d68 plugin XEP-0313: fix getting MAM archive when history is empty
Goffi <goffi@goffi.org>
parents: 3356
diff changeset
96 else:
40794e658d68 plugin XEP-0313: fix getting MAM archive when history is empty
Goffi <goffi@goffi.org>
parents: 3356
diff changeset
97 stanza_id = last_mess[0][-1]['stanza_id']
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
98 if rsm_req is None:
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
99 rsm_req = rsm.RSMRequest(max_=100, after=stanza_id)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
100 mam_req = mam.MAMRequest(rsm_=rsm_req)
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
101 complete = False
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
102 count = 0
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
103 while not complete:
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
104 try:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
105 mam_data = await self.get_archives(client, mam_req,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
106 service=client.jid.userhostJID())
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
107 except StanzaError as e:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
108 log.warning(
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
109 f"Can't retrieve MAM archives: {e}\n"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
110 f"{xml_tools.pp_elt(mam_req.toElement())}\n"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
111 f"{xml_tools.pp_elt(e.stanza)}"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
112 )
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
113 return
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
114 except Exception as e:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
115 log.exception(
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
116 f"Can't retrieve retrieve MAM archive"
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
117 )
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
118 return
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
119 elt_list, rsm_response, mam_response = mam_data
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
120 complete = mam_response["complete"]
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
121 # we update MAM request for next iteration
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
122 mam_req.rsm.after = rsm_response.last
3561
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
123 # before may be set if we had no previous history
4dad134a82c6 plugin XEP-0313: on cold start (without known archive) we only request 50 last messages
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
124 mam_req.rsm.before = None
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
125 if not elt_list:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
126 break
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
127 else:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
128 count += len(elt_list)
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
129
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
130 for idx, mess_elt in enumerate(elt_list):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
131 try:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
132 fwd_message_elt = self.get_message_from_result(
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
133 client, mess_elt, mam_req)
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
134 except exceptions.DataError:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
135 continue
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
136
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
137 try:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
138 destinee = jid.JID(fwd_message_elt['to'])
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
139 except KeyError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
140 log.warning(_('missing "to" attribute in forwarded message'))
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
141 destinee = client.jid
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
142 if destinee.userhostJID() == client.jid.userhostJID():
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
143 # message to use, we insert the forwarded message in the normal
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
144 # workflow
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
145 client.xmlstream.dispatch(fwd_message_elt)
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
146 else:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
147 # this message should be from us, we just add it to history
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
148 try:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
149 from_jid = jid.JID(fwd_message_elt['from'])
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
150 except KeyError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
151 log.warning(_('missing "from" attribute in forwarded message'))
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
152 from_jid = client.jid
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
153 if from_jid.userhostJID() != client.jid.userhostJID():
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
154 log.warning(_(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
155 'was expecting a message sent by our jid, but this one if '
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
156 'from {from_jid}, ignoring\n{xml}').format(
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
157 from_jid=from_jid.full(), xml=mess_elt.toXml()))
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
158 continue
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
159 # adding message to history
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
160 mess_data = client.messageProt.parse_message(fwd_message_elt)
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
161 try:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
162 await client.messageProt.add_to_history(mess_data)
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
163 except exceptions.CancelError as e:
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
164 log.warning(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
165 "message has not been added to history: {e}".format(e=e))
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
166 except Exception as e:
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
167 log.error(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
168 "can't add message to history: {e}\n{xml}"
3023
1f74cd0f22c3 plugin XEP-0313: log issues in addToHistory instead of raising error
Goffi <goffi@goffi.org>
parents: 3011
diff changeset
169 .format(e=e, xml=mess_elt.toXml()))
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
170 if complete and idx == len(elt_list) - 1:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
171 # We are at the last message from archive, we store the ID to not
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
172 # ask again the same messages next time.
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
173 try:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
174 stanza_id = mess_elt.result["id"]
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
175 await self.host.memory.storage.set_private_value(
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
176 namespace=mam.NS_MAM,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
177 key=KEY_LAST_STANZA_ID,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
178 value=stanza_id,
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
179 profile=client.profile
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
180 )
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
181 except Exception:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
182 log.exception("Can't store last stanza ID")
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
183
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
184 if not count:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
185 log.info(_("We have received no message while offline"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
186 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
187 log.info(_("We have received {num_mess} message(s) while offline.")
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
188 .format(num_mess=count))
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
189
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
190 def profile_connected(self, client):
4002
5245b675f7ad plugin XEP-0313: don't wait for MAM to be retrieved in connection workflow:
Goffi <goffi@goffi.org>
parents: 3573
diff changeset
191 defer.ensureDeferred(self.resume(client))
3011
93da7c6f8e0c plugin XEP-0198: retrieve missing messages + send buffered ones on hot reconnection:
Goffi <goffi@goffi.org>
parents: 2930
diff changeset
192
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
193 def get_handler(self, client):
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
194 mam_client = client._mam = LiberviaMAMClient(self)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
195 return mam_client
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
196
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
197 def parse_extra(self, extra, with_rsm=True):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
198 """Parse extra dictionnary to retrieve MAM arguments
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
199
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
200 @param extra(dict): data for parse
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
201 @param with_rsm(bool): if True, RSM data will be parsed too
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
202 @return (data_form, None): request with parsed arguments
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
203 or None if no MAM arguments have been found
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
204 """
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
205 mam_args = {}
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
206 form_args = {}
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
207 for arg in ("start", "end"):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
208 try:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
209 value = extra.pop(MAM_PREFIX + arg)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
210 form_args[arg] = datetime.fromtimestamp(float(value), tz.tzutc())
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
211 except (TypeError, ValueError):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
212 log.warning("Bad value for {arg} filter ({value}), ignoring".format(
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
213 arg=arg, value=value))
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
214 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
215 continue
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
216
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
217 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
218 form_args["with_jid"] = jid.JID(extra.pop(
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
219 MAM_PREFIX + "with"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
220 except (jid.InvalidFormat):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
221 log.warning("Bad value for jid filter")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
222 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
223 pass
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
224
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
225 for name, value in extra.items():
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
226 if name.startswith(FILTER_PREFIX):
2763
c4190d5340ab XEP-0059: max value check:
Goffi <goffi@goffi.org>
parents: 2760
diff changeset
227 var = name[len(FILTER_PREFIX):]
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
228 extra_fields = form_args.setdefault("extra_fields", [])
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
229 extra_fields.append(data_form.Field(var=var, value=value))
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
230
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
231 for arg in ("node", "query_id"):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
232 try:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
233 value = extra.pop(MAM_PREFIX + arg)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
234 mam_args[arg] = value
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
235 except KeyError:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
236 continue
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
237
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
238 if with_rsm:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
239 rsm_request = self._rsm.parse_extra(extra)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
240 if rsm_request is not None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
241 mam_args["rsm_"] = rsm_request
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
242
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
243 if form_args:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
244 mam_args["form"] = mam.buildForm(**form_args)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
245
2760
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
246 # we only set orderBy if we have other MAM args
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
247 # else we would make a MAM query while it's not expected
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
248 if "order_by" in extra and mam_args:
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
249 order_by = extra.pop("order_by")
2760
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
250 assert isinstance(order_by, list)
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
251 mam_args["orderBy"] = order_by
3480d4fdf83a plugins XEP-0060, XEP-0313: implemented Order-By protoXEP:
Goffi <goffi@goffi.org>
parents: 2718
diff changeset
252
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
253 return mam.MAMRequest(**mam_args) if mam_args else None
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
254
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
255 def get_message_from_result(self, client, mess_elt, mam_req, service=None):
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
256 """Extract usable <message/> from MAM query result
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
257
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
258 The message will be validated, and stanza-id/delay will be added if necessary.
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
259 @param mess_elt(domish.Element): result <message/> element wrapping the message
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
260 to retrieve
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
261 @param mam_req(mam.MAMRequest): request used (needed to get query_id)
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
262 @param service(jid.JID, None): MAM service where the request has been sent
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
263 None if it's user server
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
264 @return domish.Element): <message/> that can be used directly with onMessage
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
265 """
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
266 if mess_elt.name != "message":
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
267 log.warning("unexpected stanza in archive: {xml}".format(
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
268 xml=mess_elt.toXml()))
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
269 raise exceptions.DataError("Invalid element")
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
270 service_jid = client.jid.userhostJID() if service is None else service
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
271 mess_from = mess_elt.getAttribute("from") or client.jid.userhostJID()
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
272 # we check that the message has been sent by the right service
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
273 # if service is None (i.e. message expected from our own server)
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
274 # from can be server jid or user's bare jid
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
275 if (mess_from != service_jid.full()
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
276 and not (service is None and mess_from == client.jid.host)):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
277 log.error("Message is not from our server, something went wrong: "
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
278 "{xml}".format(xml=mess_elt.toXml()))
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
279 raise exceptions.DataError("Invalid element")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
280 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
281 result_elt = next(mess_elt.elements(mam.NS_MAM, "result"))
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
282 forwarded_elt = next(result_elt.elements(C.NS_FORWARD, "forwarded"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
283 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
284 delay_elt = next(forwarded_elt.elements(C.NS_DELAY, "delay"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
285 except StopIteration:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
286 # delay_elt is not mandatory
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
287 delay_elt = None
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
288 fwd_message_elt = next(forwarded_elt.elements(C.NS_CLIENT, "message"))
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
289 except StopIteration:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
290 log.warning("Invalid message received from MAM: {xml}".format(
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
291 xml=mess_elt.toXml()))
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
292 raise exceptions.DataError("Invalid element")
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
293 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
294 if not result_elt["queryid"] == mam_req.query_id:
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
295 log.error("Unexpected query id (was expecting {query_id}): {xml}"
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
296 .format(query_id=mam.query_id, xml=mess_elt.toXml()))
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
297 raise exceptions.DataError("Invalid element")
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
298 stanza_id = self._sid.get_stanza_id(fwd_message_elt,
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
299 service_jid)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
300 if stanza_id is None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
301 # not stanza-id element is present, we add one so message
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
302 # will be archived with it, and we won't request several times
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
303 # the same MAM achive
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
304 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
305 stanza_id = result_elt["id"]
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
306 except AttributeError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
307 log.warning('Invalid MAM result: missing "id" attribute: {xml}'
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
308 .format(xml=result_elt.toXml()))
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
309 raise exceptions.DataError("Invalid element")
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
310 self._sid.add_stanza_id(client, fwd_message_elt, stanza_id, by=service_jid)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
311
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
312 if delay_elt is not None:
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
313 fwd_message_elt.addChild(delay_elt)
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
314
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
315 return fwd_message_elt
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
316
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
317 def queryFields(self, client, service=None):
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
318 """Ask the server about supported fields.
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
319
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
320 @param service: entity offering the MAM service (None for user archives)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
321 @return (D(data_form.Form)): form with the implemented fields (cf XEP-0313 §4.1.5)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
322 """
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
323 return client._mam.queryFields(service)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
324
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
325 def queryArchive(self, client, mam_req, service=None):
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
326 """Query a user, MUC or pubsub archive.
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
327
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
328 @param mam_req(mam.MAMRequest): MAM query instance
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
329 @param service(jid.JID, None): entity offering the MAM service
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
330 None for user server
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
331 @return (D(domish.Element)): <IQ/> result
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
332 """
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
333 return client._mam.queryArchive(mam_req, service)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
334
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
335 def _append_message(self, elt_list, message_cb, message_elt):
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
336 if message_cb is not None:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
337 elt_list.append(message_cb(message_elt))
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
338 else:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
339 elt_list.append(message_elt)
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
340
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
341 def _query_finished(self, iq_result, client, elt_list, event):
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
342 client.xmlstream.removeObserver(event, self._append_message)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
343 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
344 fin_elt = next(iq_result.elements(mam.NS_MAM, "fin"))
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
345 except StopIteration:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
346 raise exceptions.DataError("Invalid MAM result")
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
347
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
348 mam_response = {"complete": C.bool(fin_elt.getAttribute("complete", C.BOOL_FALSE)),
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
349 "stable": C.bool(fin_elt.getAttribute("stable", C.BOOL_TRUE))}
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
350
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
351 try:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
352 rsm_response = rsm.RSMResponse.fromElement(fin_elt)
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
353 except rsm.RSMNotFoundError:
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
354 rsm_response = None
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
355
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
356 return (elt_list, rsm_response, mam_response)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
357
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
358 def serialize_archive_result(self, data, client, mam_req, service):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
359 elt_list, rsm_response, mam_response = data
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
360 mess_list = []
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
361 for elt in elt_list:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
362 fwd_message_elt = self.get_message_from_result(client, elt, mam_req,
2713
19000c506d0c plugin XEP-0313: improvments to prepare MUC MAM:
Goffi <goffi@goffi.org>
parents: 2701
diff changeset
363 service=service)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
364 mess_data = client.messageProt.parse_message(fwd_message_elt)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
365 mess_list.append(client.message_get_bridge_args(mess_data))
3308
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
366 metadata = {
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
367 'rsm': self._rsm.response2dict(rsm_response),
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
368 'mam': mam_response
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
369 }
3356
569f4cf7183b plugin XEP-0313: fixed `MAMGet` signature
Goffi <goffi@goffi.org>
parents: 3308
diff changeset
370 return mess_list, data_format.serialise(metadata), client.profile
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
371
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
372 def _get_archives(self, service, extra_ser, profile_key):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
373 """
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
374 @return: tuple with:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
375 - list of message with same data as in bridge.message_new
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
376 - response metadata with:
3308
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
377 - rsm data (first, last, count, index)
384283adcce1 plugins XEP-0059, XEP-0060, XEP-0277, XEP-0313: better serialisation:
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
378 - mam data (complete, stable)
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
379 - profile
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
380 """
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
381 client = self.host.get_client(profile_key)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
382 service = jid.JID(service) if service else None
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
383 extra = data_format.deserialise(extra_ser, {})
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
384 mam_req = self.parse_extra(extra)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
385
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
386 d = self.get_archives(client, mam_req, service=service)
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
387 d.addCallback(self.serialize_archive_result, client, mam_req, service)
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
388 return d
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
389
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
390 def get_archives(self, client, query, service=None, message_cb=None):
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
391 """Query archive and gather page result
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
392
2718
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
393 @param query(mam.MAMRequest): MAM request
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
394 @param service(jid.JID, None): MAM service to use
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
395 None to use our own server
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
396 @param message_cb(callable, None): callback to use on each message
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
397 this method can be used to unwrap messages
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
398 @return (tuple[list[domish.Element], rsm.RSMResponse, dict): result data with:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
399 - list of found elements
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
400 - RSM response
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
401 - MAM response, which is a dict with following value:
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
402 - complete: a boolean which is True if all items have been received
bb6adaa580ee plugin XEP-0313, XEP-0045: loop MAM requests until whole archive is retrieved:
Goffi <goffi@goffi.org>
parents: 2713
diff changeset
403 - stable: a boolean which is False if items order may be changed
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
404 """
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
405 if query.query_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
406 query.query_id = str(uuid.uuid4())
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
407 elt_list = []
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
408 event = MESSAGE_RESULT.format(mam_ns=mam.NS_MAM, query_id=query.query_id)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
409 client.xmlstream.addObserver(event, self._append_message, 0, elt_list, message_cb)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
410 d = self.queryArchive(client, query, service)
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
411 d.addCallback(self._query_finished, client, elt_list, event)
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
412 return d
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
413
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
414 def get_prefs(self, client, service=None):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
415 """Retrieve the current user preferences.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
416
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
417 @param service: entity offering the MAM service (None for user archives)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
418 @return: the server response as a Deferred domish.Element
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
419 """
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
420 # http://xmpp.org/extensions/xep-0313.html#prefs
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
421 return client._mam.queryPrefs(service)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
422
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
423 def _set_prefs(self, service_s=None, default="roster", always=None, never=None,
2701
2ea2369ae7de plugin XEP-0313: implementation of MAM for messages:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
424 profile_key=C.PROF_KEY_NONE):
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
425 service = jid.JID(service_s) if service_s else None
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
426 always_jid = [jid.JID(entity) for entity in always]
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
427 never_jid = [jid.JID(entity) for entity in never]
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
428 # TODO: why not build here a MAMPrefs object instead of passing the args separately?
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
429 return self.setPrefs(service, default, always_jid, never_jid, profile_key)
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
430
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
431 def setPrefs(self, client, service=None, default="roster", always=None, never=None):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
432 """Set news user preferences.
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
433
1284
41ffe2c2dddc plugin XEP-0313: update (still draft)
souliane <souliane@mailoo.org>
parents: 1277
diff changeset
434 @param service: entity offering the MAM service (None for user archives)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
435 @param default (unicode): a value in ('always', 'never', 'roster')
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
436 @param always (list): a list of JID instances
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
437 @param never (list): a list of JID instances
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
438 @param profile_key (unicode): %(doc_profile_key)s
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
439 @return: the server response as a Deferred domish.Element
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
440 """
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
441 # http://xmpp.org/extensions/xep-0313.html#prefs
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
442 return client._mam.setPrefs(service, default, always, never)
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
443
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
444 def on_message_stanza_id(self, message_elt, client):
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
445 """Called when a message with a stanza-id is received
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
446
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
447 the messages' stanza ids are stored when received, so the last one can be used
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
448 to retrieve missing history on next connection
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
449 @param message_elt(domish.Element): <message> with a stanza-id
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
450 """
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
451 if message_elt.getAttribute("type") == C.MESS_TYPE_GROUPCHAT:
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
452 # groupchat message MAM is handled by XEP-0045
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
453 return
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
454 service_jid = client.jid.userhostJID()
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
455 stanza_id = self._sid.get_stanza_id(message_elt, service_jid)
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
456 if stanza_id is None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
457 log.debug("Ignoring <message>, stanza id is not from our server")
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
458 else:
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
459 # we use self._last_stanza_id_d do be sure that last_stanza_id is stored in
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
460 # the order of reception
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
461 self._last_stanza_id_d.addCallback(
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
462 lambda __: self.host.memory.storage.set_private_value(
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
463 namespace=mam.NS_MAM,
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
464 key=KEY_LAST_STANZA_ID,
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
465 value=stanza_id,
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
466 profile=client.profile))
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
467
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
468
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
469 @implementer(disco.IDisco)
4154
85f5e6225aa1 plugin XEP-0313: better error logging + store last stanza ID when retrieving archives + small improvments
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
470 class LiberviaMAMClient(mam.MAMClient):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
471
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
472 def __init__(self, plugin_parent):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
473 self.plugin_parent = plugin_parent
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
474
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
475 @property
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
476 def host(self):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
477 return self.parent.host_app
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
478
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
479 def connectionInitialized(self):
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
480 observer_xpath = MESSAGE_STANZA_ID.format(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 3023
diff changeset
481 ns_stanza_id=self.host.ns_map['stanza_id'])
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
482 self.xmlstream.addObserver(
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 4002
diff changeset
483 observer_xpath, self.plugin_parent.on_message_stanza_id, client=self.parent
2930
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
484 )
32b6893240e0 plugin XEP-0313: fixed archive retrieval on connection:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
485
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
486 def getDiscoInfo(self, requestor, target, nodeIdentifier=""):
1776
4fc1bf1af48f plugin XEP-0313: cleaning and improvments:
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
487 return [disco.DiscoFeature(mam.NS_MAM)]
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
488
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
489 def getDiscoItems(self, requestor, target, nodeIdentifier=""):
1277
3a3e3014f9f8 plugin XEP-0313: first draft:
souliane <souliane@mailoo.org>
parents:
diff changeset
490 return []