annotate sat/plugins/plugin_misc_extra_pep.py @ 2716:06160b529da6

core (memory/sqlite): changed history constraint /!\ Database schema change /!\ History was using a unique constraint on `profile_id, timestamp, source, dest, source_res, dest_res`, which can cause trouble because several messages send quickly by the same person can have a common timestamp (specially with delayed messages where precision is second), resulting in message loss. The new constraint use `profile_id, stanza_id, source, dest` where `stanza_id` is XEP-0359 stanza_id, so it's unique by definition, and no message should be lost anymore. Because sqlite doesn't support altering table with a constraint change, we have to create new tables and copy old data to new one, which can be pretty long. Sqlite update mechanism with "specifics" has been fixed when several updates are applied (e.g. moving from v5 to v7) and a specific is in the workflow.
author Goffi <goffi@goffi.org>
date Sun, 09 Dec 2018 14:07:26 +0100
parents 56f94936df1e
children ab2696e34d29
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1746
diff changeset
1 #!/usr/bin/env python2
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
3
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT plugin for displaying messages from extra PEP services
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
5 # Copyright (C) 2015 Adrien Cossa (souliane@mailoo.org)
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
6
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # (at your option) any later version.
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
11
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
16
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
19
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
20 from sat.core.i18n import _, D_
2145
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
21 from sat.core.constants import Const as C
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
22 from sat.core.log import getLogger
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
23
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
24 log = getLogger(__name__)
1744
9a48e09044eb plugin extra_pep, params: fixed insecure building of jids_list
Goffi <goffi@goffi.org>
parents: 1741
diff changeset
25 from sat.memory import params
9a48e09044eb plugin extra_pep, params: fixed insecure building of jids_list
Goffi <goffi@goffi.org>
parents: 1741
diff changeset
26 from twisted.words.protocols.jabber import jid
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
27
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
28
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
29 PLUGIN_INFO = {
2145
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
30 C.PI_NAME: "Extra PEP",
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
31 C.PI_IMPORT_NAME: "EXTRA-PEP",
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
32 C.PI_TYPE: "MISC",
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
33 C.PI_PROTOCOLS: [],
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
34 C.PI_DEPENDENCIES: [],
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
35 C.PI_RECOMMENDATIONS: [],
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
36 C.PI_MAIN: "ExtraPEP",
33c8c4973743 core (plugins): added missing contants + use of new constants in PLUGIN_INFO
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
37 C.PI_HANDLER: "no",
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
38 C.PI_DESCRIPTION: _(u"""Display messages from extra PEP services"""),
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
39 }
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
40
1744
9a48e09044eb plugin extra_pep, params: fixed insecure building of jids_list
Goffi <goffi@goffi.org>
parents: 1741
diff changeset
41
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
42 PARAM_KEY = u"Misc"
1746
0dfabe746ec4 plugin extra_pep: replaced name by a short one
Goffi <goffi@goffi.org>
parents: 1744
diff changeset
43 PARAM_NAME = u"blogs"
0dfabe746ec4 plugin extra_pep: replaced name by a short one
Goffi <goffi@goffi.org>
parents: 1744
diff changeset
44 PARAM_LABEL = u"Blog authors following list"
1744
9a48e09044eb plugin extra_pep, params: fixed insecure building of jids_list
Goffi <goffi@goffi.org>
parents: 1741
diff changeset
45 PARAM_DEFAULT = (jid.JID("salut-a-toi@libervia.org"),)
9a48e09044eb plugin extra_pep, params: fixed insecure building of jids_list
Goffi <goffi@goffi.org>
parents: 1741
diff changeset
46
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
47
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
48 class ExtraPEP(object):
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
49
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
50 params = """
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
51 <params>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
52 <individual>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
53 <category name="%(category_name)s" label="%(category_label)s">
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
54 <param name="%(param_name)s" label="%(param_label)s" type="jids_list" security="0">
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
55 %(jids)s
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
56 </param>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
57 </category>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
58 </individual>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
59 </params>
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
60 """ % {
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
61 "category_name": PARAM_KEY,
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
62 "category_label": D_(PARAM_KEY),
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
63 "param_name": PARAM_NAME,
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
64 "param_label": D_(PARAM_LABEL),
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
65 "jids": u"\n".join({elt.toXml() for elt in params.createJidElts(PARAM_DEFAULT)}),
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
66 }
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
67
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
68 def __init__(self, host):
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
69 log.info(_(u"Plugin Extra PEP initialization"))
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
70 self.host = host
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
71 host.memory.updateParams(self.params)
1728
0eaa1a409dfb plugin XEP-0277, EXTRA-PEP: get massive blogs from all the roster also retrieve items from a special JIDs list defined in EXTRA-PEP
souliane <souliane@mailoo.org>
parents:
diff changeset
72
1741
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
73 def getFollowedEntities(self, profile_key):
cc31dd72526d plugin misc_extra_pep: use a user parameter (jids_list) instead of a general configuration value
souliane <souliane@mailoo.org>
parents: 1728
diff changeset
74 return self.host.memory.getParamA(PARAM_NAME, PARAM_KEY, profile_key=profile_key)