annotate sat_pubsub/pgsql_storage.py @ 430:5a0ada3b61ca

Full-Text Search implementation: /!\ pgsql schema needs to be updated /!\ /!\ Minimal PostgreSQL required version is now 12 /!\ A new options is available to specify main language of a node. By default a `generic` language is used (which uses the `simple` configuration in PostgreSQL). When a node owner changes the language, the index is rebuilt accordingly. It is possible to have item specific language for multilingual nodes (but for the moment the search is done with node language, so the results won't be good). If an item language is explicitely set in `item_languages`, the FTS configuration won't be affected by node FTS language setting. Search is parsed with `websearch_to_tsquery` for now, but this parser doesn't handle prefix matching, so it may be replaced in the future. SetConfiguration now only updates the modified values, this avoid triggering the FTS re-indexing on each config change. `_checkNodeExists` is not called anymore as we can check if a row has been modified to see if the node exists, this avoid a useless query. Item storing has been slighly improved with a useless SELECT and condition removed. To avoid 2 schema updates in a row, the `sat_pubsub_update_5_6.sql` file also prepares the implementation of XEP-0346 by updating nodes with a schema and creating the suitable template nodes.
author Goffi <goffi@goffi.org>
date Fri, 11 Dec 2020 17:18:52 +0100
parents 0526073ff2ab
children 920440200570
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
1 #!/usr/bin/env python3
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
2 #-*- coding: utf-8 -*-
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
3
384
aa3a464df605 dates update
Goffi <goffi@goffi.org>
parents: 378
diff changeset
4 # Copyright (c) 2012-2019 Jérôme Poisson
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
5 # Copyright (c) 2013-2016 Adrien Cossa
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
6 # Copyright (c) 2003-2011 Ralph Meijer
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
7
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
8
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
9 # This program is free software: you can redistribute it and/or modify
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
10 # it under the terms of the GNU Affero General Public License as published by
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
11 # the Free Software Foundation, either version 3 of the License, or
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
12 # (at your option) any later version.
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
13
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
14 # This program is distributed in the hope that it will be useful,
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
15 # but WITHOUT ANY WARRANTY; without even the implied warranty of
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
16 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
17 # GNU Affero General Public License for more details.
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
18
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
19 # You should have received a copy of the GNU Affero General Public License
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
20 # along with this program. If not, see <http://www.gnu.org/licenses/>.
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
21 # --
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
22
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
23 # This program is based on Idavoll (http://idavoll.ik.nu/),
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
24 # originaly written by Ralph Meijer (http://ralphm.net/blog/)
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
25 # It is sublicensed under AGPL v3 (or any later version) as allowed by the original
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
26 # license.
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
27
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
28 # --
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
29
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
30 # Here is a copy of the original license:
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
31
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
32 # Copyright (c) 2003-2011 Ralph Meijer
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
33
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
34 # Permission is hereby granted, free of charge, to any person obtaining
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
35 # a copy of this software and associated documentation files (the
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
36 # "Software"), to deal in the Software without restriction, including
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
37 # without limitation the rights to use, copy, modify, merge, publish,
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
38 # distribute, sublicense, and/or sell copies of the Software, and to
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
39 # permit persons to whom the Software is furnished to do so, subject to
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
40 # the following conditions:
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
41
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
42 # The above copyright notice and this permission notice shall be
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
43 # included in all copies or substantial portions of the Software.
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
44
312
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
45 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
46 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
47 # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
48 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
49 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
50 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
5d7c3787672e fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents: 311
diff changeset
51 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
233
564ae55219e1 sublicensed under AGPL V3
Goffi <goffi@goffi.org>
parents: 232
diff changeset
52
155
5191ba7c4df8 Work towards first release 0.5.0.
Ralph Meijer <ralphm@ik.nu>
parents: 148
diff changeset
53
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
54 import copy, logging
202
77c61e2b8c75 Use `domish.Element`s to represent items, instead of serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents: 198
diff changeset
55
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
56 from zope.interface import implementer
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
57
344
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
58 from twisted.internet import reactor
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
59 from twisted.internet import defer
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
60 from twisted.words.protocols.jabber import jid
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
61 from twisted.python import log
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
62
271
232002e132db pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents: 270
diff changeset
63 from wokkel import generic
429
0526073ff2ab pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents: 424
diff changeset
64 from wokkel.pubsub import Subscription, BadRequest
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
65
317
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
66 from sat_pubsub import error
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
67 from sat_pubsub import iidavoll
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
68 from sat_pubsub import const
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
69 from sat_pubsub import container
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
70 from sat_pubsub import exceptions
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
71 import uuid
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
72 import psycopg2
270
f17034e4cf4a fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents: 265
diff changeset
73 import psycopg2.extensions
271
232002e132db pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents: 270
diff changeset
74 # we wants psycopg2 to return us unicode, not str
270
f17034e4cf4a fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents: 265
diff changeset
75 psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
f17034e4cf4a fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents: 265
diff changeset
76 psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY)
244
3ecc94407e36 item access_model (not finished)
Goffi <goffi@goffi.org>
parents: 243
diff changeset
77
271
232002e132db pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents: 270
diff changeset
78 # parseXml manage str, but we get unicode
232002e132db pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents: 270
diff changeset
79 parseXml = lambda unicode_data: generic.parseXml(unicode_data.encode('utf-8'))
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
80 ITEMS_SEQ_NAME = 'node_{node_id}_seq'
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
81 PEP_COL_NAME = 'pep'
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
82 CURRENT_VERSION = '6'
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
83 # retrieve the maximum integer item id + 1
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
84 NEXT_ITEM_ID_QUERY = r"SELECT COALESCE(max(item::integer)+1,1) as val from items where node_id={node_id} and item ~ E'^\\d+$'"
271
232002e132db pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents: 270
diff changeset
85
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
86
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
87 def withPEP(query, values, pep, recipient):
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
88 """Helper method to facilitate PEP management
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
89
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
90 @param query: SQL query basis
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
91 @param values: current values to replace in query
364
8bd8be6815ab completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents: 357
diff changeset
92 @param pep(bool): True if we are in PEP mode
8bd8be6815ab completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents: 357
diff changeset
93 @param recipient(jid.JID): jid of the recipient
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
94 @return: query + PEP AND check,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
95 recipient's bare jid is added to value if needed
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
96 """
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
97 if pep:
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
98 pep_check="AND {}=%s".format(PEP_COL_NAME)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
99 values=list(values) + [recipient.userhost()]
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
100 else:
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
101 pep_check="AND {} IS NULL".format(PEP_COL_NAME)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
102 return "{} {}".format(query, pep_check), values
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
103
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
104
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
105 @implementer(iidavoll.IStorage)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
106 class Storage:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
107
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
108 fts_languages = ['generic']
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
109 defaultConfig = {
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
110 'leaf': {
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
111 const.OPT_PERSIST_ITEMS: True,
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
112 const.OPT_MAX_ITEMS: 'max',
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
113 const.OPT_DELIVER_PAYLOADS: True,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
114 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub',
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
115 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT,
260
f0cd02c032b3 publish model management
Goffi <goffi@goffi.org>
parents: 259
diff changeset
116 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT,
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
117 const.OPT_SERIAL_IDS: False,
403
1dc606612405 implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents: 402
diff changeset
118 const.OPT_CONSISTENT_PUBLISHER: False,
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
119 const.OPT_FTS_LANGUAGE: const.VAL_FTS_GENERIC,
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
120 },
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
121 'collection': {
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
122 const.OPT_DELIVER_PAYLOADS: True,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
123 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub',
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
124 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT,
260
f0cd02c032b3 publish model management
Goffi <goffi@goffi.org>
parents: 259
diff changeset
125 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT,
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
126 }
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
127 }
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
128
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
129 def __init__(self, dbpool):
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
130 self.dbpool = dbpool
344
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
131 d = self.dbpool.runQuery("SELECT value FROM metadata WHERE key='version'")
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
132 d.addCallbacks(self._checkVersion, self._versionEb)
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
133
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
134 def _checkVersion(self, row):
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
135 version = row[0].value
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
136 if version != CURRENT_VERSION:
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
137 logging.error("Bad database schema version ({current}), please upgrade to {needed}".format(
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
138 current=version, needed=CURRENT_VERSION))
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
139 reactor.stop()
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
140
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
141 def _versionEb(self, failure):
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
142 logging.error("Can't check schema version: {reason}".format(reason=failure))
8cf1be9572f8 pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents: 341
diff changeset
143 reactor.stop()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
144
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
145 def _buildNode(self, row):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
146 """Build a note class from database result row"""
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
147 if not row:
167
ef22e4150caa Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents: 163
diff changeset
148 raise error.NodeNotFound()
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
149
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
150 if row[2] == 'leaf':
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
151 configuration = {
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
152 const.OPT_PERSIST_ITEMS: row[3],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
153 const.OPT_MAX_ITEMS: 'max' if row[4] == 0 else str(row[4]),
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
154 const.OPT_DELIVER_PAYLOADS: row[5],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
155 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
156 const.OPT_ACCESS_MODEL:row[7],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
157 const.OPT_PUBLISH_MODEL:row[8],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
158 const.OPT_SERIAL_IDS:row[9],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
159 const.OPT_CONSISTENT_PUBLISHER:row[10],
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
160 const.OPT_FTS_LANGUAGE: row[11],
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
161 }
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
162 schema = row[12]
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
163 if schema is not None:
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
164 schema = parseXml(schema)
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
165 node = LeafNode(row[0], row[1], configuration, schema)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
166 node.dbpool = self.dbpool
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
167 return node
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
168 elif row[2] == 'collection':
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
169 configuration = {
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
170 const.OPT_DELIVER_PAYLOADS: row[5],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
171 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
172 const.OPT_ACCESS_MODEL: row[7],
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
173 const.OPT_PUBLISH_MODEL:row[8],
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
174 }
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
175 node = CollectionNode(row[0], row[1], configuration, None)
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
176 node.dbpool = self.dbpool
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
177 return node
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
178 else:
429
0526073ff2ab pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents: 424
diff changeset
179 raise BadRequest(text="Unknown node type !")
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
180
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
181 def getFTSLanguages(self):
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
182 """Get list of available languages for full text search"""
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
183 return self.dbpool.runInteraction(self._getFTSLanguages)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
184
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
185 def _getFTSLanguages(self, cursor):
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
186 cursor.execute("SELECT cfgname FROM pg_ts_config")
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
187 result = [r.cfgname for r in cursor.fetchall()]
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
188 result.remove("simple")
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
189 result.insert(0, "generic")
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
190 Node.fts_languages = self.fts_languages = result
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
191 return result
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
192
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
193 def getNodeById(self, nodeDbId):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
194 """Get node using database ID insted of pubsub identifier
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
195
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
196 @param nodeDbId(unicode): database ID
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
197 """
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
198 return self.dbpool.runInteraction(self._getNodeById, nodeDbId)
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
199
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
200 def _getNodeById(self, cursor, nodeDbId):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
201 cursor.execute("""SELECT node_id,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
202 node,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
203 node_type,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
204 persist_items,
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
205 max_items,
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
206 deliver_payloads,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
207 send_last_published_item,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
208 access_model,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
209 publish_model,
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
210 serial_ids,
403
1dc606612405 implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents: 402
diff changeset
211 consistent_publisher,
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
212 fts_language,
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
213 schema::text,
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
214 pep
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
215 FROM nodes
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
216 WHERE node_id=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
217 (nodeDbId,))
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
218 row = cursor.fetchone()
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
219 return self._buildNode(row)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
220
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
221 def getNode(self, nodeIdentifier, pep, recipient=None):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
222 return self.dbpool.runInteraction(self._getNode, nodeIdentifier, pep, recipient)
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
223
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
224 def _getNode(self, cursor, nodeIdentifier, pep, recipient):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
225 cursor.execute(*withPEP("""SELECT node_id,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
226 node,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
227 node_type,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
228 persist_items,
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
229 max_items,
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
230 deliver_payloads,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
231 send_last_published_item,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
232 access_model,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
233 publish_model,
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
234 serial_ids,
403
1dc606612405 implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents: 402
diff changeset
235 consistent_publisher,
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
236 fts_language,
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
237 schema::text,
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
238 pep
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
239 FROM nodes
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
240 WHERE node=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
241 (nodeIdentifier,), pep, recipient))
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
242 row = cursor.fetchone()
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
243 return self._buildNode(row)
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
244
349
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
245 def getNodeIds(self, pep, recipient, allowed_accesses=None):
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
246 """retrieve ids of existing nodes
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
247
364
8bd8be6815ab completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents: 357
diff changeset
248 @param pep(bool): True if it's a PEP request
8bd8be6815ab completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents: 357
diff changeset
249 @param recipient(jid.JID, None): recipient of the PEP request
349
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
250 @param allowed_accesses(None, set): only nodes with access
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
251 in this set will be returned
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
252 None to return all nodes
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
253 @return (list[unicode]): ids of nodes
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
254 """
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
255 if not pep:
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
256 query = "SELECT node from nodes WHERE pep is NULL"
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
257 values = []
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
258 else:
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
259 query = "SELECT node from nodes WHERE pep=%s"
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
260 values = [recipient.userhost()]
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
261
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
262 if allowed_accesses is not None:
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
263 query += "AND access_model IN %s"
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
264 values.append(tuple(allowed_accesses))
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
265
20b82fb8de02 backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents: 347
diff changeset
266 d = self.dbpool.runQuery(query, values)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
267 d.addCallback(lambda results: [r[0] for r in results])
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
268 return d
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
269
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
270 def createNode(self, nodeIdentifier, owner, config, schema, pep, recipient=None):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
271 return self.dbpool.runInteraction(self._createNode, nodeIdentifier,
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
272 owner, config, schema, pep, recipient)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
273
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
274 def _createNode(self, cursor, nodeIdentifier, owner, config, schema, pep, recipient):
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
275 if config['pubsub#node_type'] != 'leaf':
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
276 raise error.NoCollections()
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
277
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
278 owner = owner.userhost()
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
279
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
280 try:
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
281 cursor.execute("""INSERT INTO nodes
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
282 (node,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
283 node_type,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
284 persist_items,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
285 deliver_payloads,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
286 send_last_published_item,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
287 access_model,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
288 publish_model,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
289 serial_ids,
403
1dc606612405 implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents: 402
diff changeset
290 consistent_publisher,
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
291 fts_language,
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
292 schema,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
293 pep)
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
294 VALUES
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
295 (%s, 'leaf', %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)""",
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
296 (nodeIdentifier,
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
297 config['pubsub#persist_items'],
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
298 config['pubsub#deliver_payloads'],
240
70c8bb90d75f added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents: 235
diff changeset
299 config['pubsub#send_last_published_item'],
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
300 config[const.OPT_ACCESS_MODEL],
260
f0cd02c032b3 publish model management
Goffi <goffi@goffi.org>
parents: 259
diff changeset
301 config[const.OPT_PUBLISH_MODEL],
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
302 config[const.OPT_SERIAL_IDS],
403
1dc606612405 implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents: 402
diff changeset
303 config[const.OPT_CONSISTENT_PUBLISHER],
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
304 config[const.OPT_FTS_LANGUAGE],
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
305 schema,
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
306 recipient.userhost() if pep else None
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
307 )
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
308 )
332
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
309 except cursor._pool.dbapi.IntegrityError as e:
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
310 if e.pgcode == "23505":
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
311 # unique_violation
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
312 raise error.NodeExists()
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
313 else:
31cbd8b9fa7f pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents: 331
diff changeset
314 raise error.InvalidConfigurationOption()
167
ef22e4150caa Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents: 163
diff changeset
315
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
316 cursor.execute(*withPEP("""SELECT node_id FROM nodes WHERE node=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
317 (nodeIdentifier,), pep, recipient));
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
318 node_id = cursor.fetchone()[0]
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
319
276
b757c29b20d7 import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents: 271
diff changeset
320 cursor.execute("""SELECT 1 as bool from entities where jid=%s""",
227
8540825f85e0 Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents: 206
diff changeset
321 (owner,))
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
322
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
323 if not cursor.fetchone():
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
324 # XXX: we can NOT rely on the previous query! Commit is needed now because
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
325 # if the entry exists the next query will leave the database in a corrupted
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
326 # state: the solution is to rollback. I tried with other methods like
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
327 # "WHERE NOT EXISTS" but none of them worked, so the following solution
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
328 # looks like the sole - unless you have auto-commit on. More info
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
329 # about this issue: http://cssmay.com/question/tag/tag-psycopg2
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
330 cursor.connection.commit()
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
331 try:
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
332 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""",
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
333 (owner,))
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
334 except psycopg2.IntegrityError as e:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
335 cursor.connection.rollback()
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
336 logging.warning("during node creation: %s" % e.message)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
337
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
338 cursor.execute("""INSERT INTO affiliations
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
339 (node_id, entity_id, affiliation)
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
340 SELECT %s, entity_id, 'owner' FROM
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
341 (SELECT entity_id FROM entities
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
342 WHERE jid=%s) as e""",
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
343 (node_id, owner))
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
344
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
345 if config[const.OPT_ACCESS_MODEL] == const.VAL_AMODEL_PUBLISHER_ROSTER:
259
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
346 if const.OPT_ROSTER_GROUPS_ALLOWED in config:
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
347 allowed_groups = config[const.OPT_ROSTER_GROUPS_ALLOWED]
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
348 else:
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
349 allowed_groups = []
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
350 for group in allowed_groups:
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
351 #TODO: check that group are actually in roster
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
352 cursor.execute("""INSERT INTO node_groups_authorized (node_id, groupname)
6fe7da6b4b32 node "roster" access model management
Goffi <goffi@goffi.org>
parents: 255
diff changeset
353 VALUES (%s,%s)""" , (node_id, group))
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
354 # XXX: affiliations can't be set on during node creation (at least not with XEP-0060 alone)
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
355 # so whitelist affiliations need to be done afterward
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
356
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
357 # no we may have to do extra things according to config options
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
358 default_conf = self.defaultConfig['leaf']
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
359 # XXX: trigger works on node creation because OPT_SERIAL_IDS is False in defaultConfig
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
360 # if this value is changed, the _configurationTriggers method should be adapted.
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
361 Node._configurationTriggers(cursor, node_id, default_conf, config)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
362
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
363 def deleteNodeByDbId(self, db_id):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
364 """Delete a node using directly its database id"""
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
365 return self.dbpool.runInteraction(self._deleteNodeByDbId, db_id)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
366
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
367 def _deleteNodeByDbId(self, cursor, db_id):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
368 cursor.execute("""DELETE FROM nodes WHERE node_id=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
369 (db_id,))
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
370
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
371 if cursor.rowcount != 1:
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
372 raise error.NodeNotFound()
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
373
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
374 def deleteNode(self, nodeIdentifier, pep, recipient=None):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
375 return self.dbpool.runInteraction(self._deleteNode, nodeIdentifier, pep, recipient)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
376
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
377 def _deleteNode(self, cursor, nodeIdentifier, pep, recipient):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
378 cursor.execute(*withPEP("""DELETE FROM nodes WHERE node=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
379 (nodeIdentifier,), pep, recipient))
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
380
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
381 if cursor.rowcount != 1:
167
ef22e4150caa Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents: 163
diff changeset
382 raise error.NodeNotFound()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
383
331
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
384 def getAffiliations(self, entity, nodeIdentifier, pep, recipient=None):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
385 return self.dbpool.runInteraction(self._getAffiliations, entity, nodeIdentifier, pep, recipient)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
386
331
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
387 def _getAffiliations(self, cursor, entity, nodeIdentifier, pep, recipient=None):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
388 query = ["""SELECT node, affiliation FROM entities
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
389 NATURAL JOIN affiliations
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
390 NATURAL JOIN nodes
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
391 WHERE jid=%s"""]
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
392 args = [entity.userhost()]
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
393
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
394 if nodeIdentifier is not None:
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
395 query.append("AND node=%s")
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
396 args.append(nodeIdentifier)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
397
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
398 cursor.execute(*withPEP(' '.join(query), args, pep, recipient))
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
399 rows = cursor.fetchall()
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
400 return [tuple(r) for r in rows]
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
401
357
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
402 def getSubscriptions(self, entity, nodeIdentifier=None, pep=False, recipient=None):
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
403 """retrieve subscriptions of an entity
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
404
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
405 @param entity(jid.JID): entity to check
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
406 @param nodeIdentifier(unicode, None): node identifier
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
407 None to retrieve all subscriptions
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
408 @param pep: True if we are in PEP mode
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
409 @param recipient: jid of the recipient
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
410 """
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
411
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
412 def toSubscriptions(rows):
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
413 subscriptions = []
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
414 for row in rows:
357
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
415 subscriber = jid.internJID('%s/%s' % (row.jid,
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
416 row.resource))
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
417 subscription = Subscription(row.node, subscriber, row.state)
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
418 subscriptions.append(subscription)
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
419 return subscriptions
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
420
357
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
421 query = ["""SELECT node,
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
422 jid,
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
423 resource,
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
424 state
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
425 FROM entities
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
426 NATURAL JOIN subscriptions
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
427 NATURAL JOIN nodes
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
428 WHERE jid=%s"""]
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
429
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
430 args = [entity.userhost()]
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
431
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
432 if nodeIdentifier is not None:
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
433 query.append("AND node=%s")
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
434 args.append(nodeIdentifier)
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
435
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
436 d = self.dbpool.runQuery(*withPEP(' '.join(query), args, pep, recipient))
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
437 d.addCallback(toSubscriptions)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
438 return d
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
439
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
440 def getDefaultConfiguration(self, nodeType):
366
81e6d4a516c3 storage (pgsql): getDefaultConfiguration now returns a shallow copy of default conf dict, to avoid modification.
Goffi <goffi@goffi.org>
parents: 365
diff changeset
441 return self.defaultConfig[nodeType].copy()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
442
337
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
443 def formatLastItems(self, result):
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
444 last_items = []
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
445 for pep_jid_s, node, data, item_access_model in result:
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
446 pep_jid = jid.JID(pep_jid_s)
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
447 item = generic.stripNamespace(parseXml(data))
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
448 last_items.append((pep_jid, node, item, item_access_model))
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
449 return last_items
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
450
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
451 def getLastItems(self, entities, nodes, node_accesses, item_accesses, pep):
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
452 """get last item for several nodes and entities in a single request"""
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
453 if not entities or not nodes or not node_accesses or not item_accesses:
429
0526073ff2ab pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents: 424
diff changeset
454 raise BadRequest(text="entities, nodes and accesses must not be empty")
337
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
455 if node_accesses != ('open',) or item_accesses != ('open',):
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
456 raise NotImplementedError('only "open" access model is handled for now')
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
457 if not pep:
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
458 raise NotImplementedError("getLastItems is only implemented for PEP at the moment")
351
2098295747fd pgsql: cast items.data to text when needed + db schema version bump.
Goffi <goffi@goffi.org>
parents: 349
diff changeset
459 d = self.dbpool.runQuery("""SELECT DISTINCT ON (node_id) pep, node, data::text, items.access_model
337
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
460 FROM items
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
461 NATURAL JOIN nodes
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
462 WHERE nodes.pep IN %s
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
463 AND node IN %s
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
464 AND nodes.access_model in %s
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
465 AND items.access_model in %s
374
40e5edd7ea11 storage: changed items order:
Goffi <goffi@goffi.org>
parents: 369
diff changeset
466 ORDER BY node_id DESC, items.updated DESC""",
337
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
467 (tuple([e.userhost() for e in entities]),
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
468 nodes,
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
469 node_accesses,
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
470 item_accesses))
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
471 d.addCallback(self.formatLastItems)
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
472 return d
57a3051ee435 storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents: 332
diff changeset
473
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
474
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
475 @implementer(iidavoll.INode)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
476 class Node:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
477
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
478
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
479 def __init__(self, nodeDbId, nodeIdentifier, config, schema):
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
480 self.nodeDbId = nodeDbId
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
481 self.nodeIdentifier = nodeIdentifier
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
482 self._config = config
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
483 self._schema = schema
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
484
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
485 def _checkNodeExists(self, cursor):
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
486 cursor.execute("""SELECT 1 as exist FROM nodes WHERE node_id=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
487 (self.nodeDbId,))
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
488 if not cursor.fetchone():
167
ef22e4150caa Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents: 163
diff changeset
489 raise error.NodeNotFound()
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
490
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
491 def getType(self):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
492 return self.nodeType
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
493
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
494 def getOwners(self):
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
495 d = self.dbpool.runQuery("""SELECT jid FROM nodes NATURAL JOIN affiliations NATURAL JOIN entities WHERE node_id=%s and affiliation='owner'""", (self.nodeDbId,))
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
496 d.addCallback(lambda rows: [jid.JID(r[0]) for r in rows])
243
42048e37699e added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents: 240
diff changeset
497 return d
42048e37699e added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents: 240
diff changeset
498
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
499 def getConfiguration(self):
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
500 return self._config
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
501
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
502 def getNextId(self):
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
503 """return XMPP item id usable for next item to publish
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
504
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
505 the return value will be next int if serila_ids is set,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
506 else an UUID will be returned
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
507 """
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
508 if self._config[const.OPT_SERIAL_IDS]:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
509 d = self.dbpool.runQuery("SELECT nextval('{seq_name}')".format(
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
510 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId)))
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
511 d.addCallback(lambda rows: str(rows[0][0]))
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
512 return d
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
513 else:
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
514 return defer.succeed(str(uuid.uuid4()))
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
515
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
516 @staticmethod
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
517 def _configurationTriggers(cursor, node_id, old_config, new_config):
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
518 """trigger database relative actions needed when a config is changed
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
519
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
520 @param cursor(): current db cursor
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
521 @param node_id(unicode): database ID of the node
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
522 @param old_config(dict): config of the node before the change
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
523 @param new_config(dict): new options that will be changed
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
524 """
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
525 if const.OPT_SERIAL_IDS not in new_config:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
526 return
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
527 serial_ids = new_config[const.OPT_SERIAL_IDS]
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
528 if serial_ids != old_config[const.OPT_SERIAL_IDS]:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
529 # serial_ids option has been modified,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
530 # we need to handle corresponding sequence
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
531
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
532 # XXX: we use .format in following queries because values
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
533 # are generated by ourself
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
534 seq_name = ITEMS_SEQ_NAME.format(node_id=node_id)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
535 if serial_ids:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
536 # the next query get the max value +1 of all XMPP items ids
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
537 # which are integers, and default to 1
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
538 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=node_id))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
539 next_val = cursor.fetchone()[0]
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
540 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
541 cursor.execute("CREATE SEQUENCE {seq_name} START {next_val} OWNED BY nodes.node_id".format(
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
542 seq_name = seq_name,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
543 next_val = next_val))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
544 else:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
545 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
546
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
547 def setConfiguration(self, options):
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
548 to_delete = []
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
549 for option, value in options.items():
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
550 try:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
551 if self._config[option] == value:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
552 to_delete.append(option)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
553 except KeyError:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
554 raise BadRequest(text=f"Invalid option: {option!r}")
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
555
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
556 for option in to_delete:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
557 options.remove(option)
180
fc687620599b Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents: 173
diff changeset
558
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
559 if not options:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
560 return
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
561
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
562 if options.get(const.OPT_MAX_ITEMS) == "max":
424
3fce48c0a44d psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents: 422
diff changeset
563 # XXX: "max" is default value for config we must convert
3fce48c0a44d psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents: 422
diff changeset
564 # it to an interger. See backend's _doSetNodeConfiguration comment
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
565 options[const.OPT_MAX_ITEMS] = "0"
424
3fce48c0a44d psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents: 422
diff changeset
566
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
567 if ((const.OPT_FTS_LANGUAGE in options
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
568 and options[const.OPT_FTS_LANGUAGE] not in self.fts_languages)):
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
569 raise BadRequest(text=
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
570 f"invalid {const.OPT_FTS_LANGUAGE} value: "
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
571 f"{options[const.OPT_FTS_LANGUAGE]!r}"
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
572 )
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
573
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
574
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
575 d = self.dbpool.runInteraction(self._setConfiguration, options)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
576 d.addCallback(self._updateCachedConfiguration, options)
124
c4ee16bc48e5 Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents: 121
diff changeset
577 return d
c4ee16bc48e5 Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents: 121
diff changeset
578
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
579 def _setConfiguration(self, cursor, options):
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
580 self._configurationTriggers(cursor, self.nodeDbId, self._config, options)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
581 # options names all follow the scheme "pubsub#{col_name}"
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
582 col_names = (o[7:] for o in options)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
583 values = ','.join(f"{name}=%s" for name in col_names)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
584 cursor.execute(f"UPDATE nodes SET {values} WHERE node_id=%s",
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
585 (*options.values(), self.nodeDbId))
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
586 if cursor.rowcount == 0:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
587 raise error.NodeNotFound()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
588
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
589 def _updateCachedConfiguration(self, __, options):
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
590 self._config.update(options)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
591
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
592 def getSchema(self):
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
593 return self._schema
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
594
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
595 def setSchema(self, schema):
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
596 d = self.dbpool.runInteraction(self._setSchema, schema)
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
597 d.addCallback(self._setCachedSchema, schema)
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
598 return d
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
599
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
600 def _setSchema(self, cursor, schema):
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
601 self._checkNodeExists(cursor)
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
602 cursor.execute("""UPDATE nodes SET schema=%s
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
603 WHERE node_id=%s""",
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
604 (schema.toXml() if schema else None,
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
605 self.nodeDbId))
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
606
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
607 def _setCachedSchema(self, __, schema):
352
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
608 self._schema = schema
efbdca10f0fb schema: node schema implementation
Goffi <goffi@goffi.org>
parents: 351
diff changeset
609
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
610 def getMetaData(self):
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
611 config = copy.copy(self._config)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
612 config["pubsub#node_type"] = self.nodeType
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
613 return config
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
614
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
615 def getAffiliation(self, entity):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
616 return self.dbpool.runInteraction(self._getAffiliation, entity)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
617
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
618 def _getAffiliation(self, cursor, entity):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
619 self._checkNodeExists(cursor)
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
620 cursor.execute("""SELECT affiliation FROM affiliations
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
621 NATURAL JOIN nodes
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
622 NATURAL JOIN entities
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
623 WHERE node_id=%s AND jid=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
624 (self.nodeDbId,
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
625 entity.userhost()))
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
626
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
627 try:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
628 return cursor.fetchone()[0]
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
629 except TypeError:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
630 return None
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
631
240
70c8bb90d75f added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents: 235
diff changeset
632 def getAccessModel(self):
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
633 return self._config[const.OPT_ACCESS_MODEL]
240
70c8bb90d75f added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents: 235
diff changeset
634
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
635 def getSubscription(self, subscriber):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
636 return self.dbpool.runInteraction(self._getSubscription, subscriber)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
637
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
638 def _getSubscription(self, cursor, subscriber):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
639 self._checkNodeExists(cursor)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
640
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
641 userhost = subscriber.userhost()
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
642 resource = subscriber.resource or ''
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
643
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
644 cursor.execute("""SELECT state FROM subscriptions
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
645 NATURAL JOIN nodes
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
646 NATURAL JOIN entities
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
647 WHERE node_id=%s AND jid=%s AND resource=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
648 (self.nodeDbId,
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
649 userhost,
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
650 resource))
276
b757c29b20d7 import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents: 271
diff changeset
651
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
652 row = cursor.fetchone()
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
653 if not row:
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
654 return None
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
655 else:
227
8540825f85e0 Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents: 206
diff changeset
656 return Subscription(self.nodeIdentifier, subscriber, row[0])
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
657
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
658 def getSubscriptions(self, state=None):
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
659 return self.dbpool.runInteraction(self._getSubscriptions, state)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
660
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
661 def _getSubscriptions(self, cursor, state):
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
662 self._checkNodeExists(cursor)
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
663
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
664 query = """SELECT node, jid, resource, state,
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
665 subscription_type, subscription_depth
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
666 FROM subscriptions
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
667 NATURAL JOIN nodes
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
668 NATURAL JOIN entities
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
669 WHERE node_id=%s"""
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
670 values = [self.nodeDbId]
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
671
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
672 if state:
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
673 query += " AND state=%s"
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
674 values.append(state)
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
675
276
b757c29b20d7 import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents: 271
diff changeset
676 cursor.execute(query, values)
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
677 rows = cursor.fetchall()
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
678
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
679 subscriptions = []
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
680 for row in rows:
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
681 subscriber = jid.JID('%s/%s' % (row.jid, row.resource))
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
682
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
683 options = {}
357
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
684 if row.subscription_type:
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
685 options['pubsub#subscription_type'] = row.subscription_type;
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
686 if row.subscription_depth:
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
687 options['pubsub#subscription_depth'] = row.subscription_depth;
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
688
357
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
689 subscriptions.append(Subscription(row.node, subscriber,
1167e48e5f52 handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents: 356
diff changeset
690 row.state, options))
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
691
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
692 return subscriptions
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
693
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
694 def addSubscription(self, subscriber, state, config):
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
695 return self.dbpool.runInteraction(self._addSubscription, subscriber,
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
696 state, config)
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
697
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
698 def _addSubscription(self, cursor, subscriber, state, config):
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
699 self._checkNodeExists(cursor)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
700
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
701 userhost = subscriber.userhost()
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
702 resource = subscriber.resource or ''
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
703
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
704 subscription_type = config.get('pubsub#subscription_type')
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
705 subscription_depth = config.get('pubsub#subscription_depth')
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
706
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
707 try:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
708 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""",
227
8540825f85e0 Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents: 206
diff changeset
709 (userhost,))
8540825f85e0 Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents: 206
diff changeset
710 except cursor._pool.dbapi.IntegrityError:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
711 cursor.connection.rollback()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
712
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
713 try:
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
714 cursor.execute("""INSERT INTO subscriptions
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
715 (node_id, entity_id, resource, state,
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
716 subscription_type, subscription_depth)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
717 SELECT %s, entity_id, %s, %s, %s, %s FROM
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
718 (SELECT entity_id FROM entities
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
719 WHERE jid=%s) AS ent_id""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
720 (self.nodeDbId,
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
721 resource,
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
722 state,
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
723 subscription_type,
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
724 subscription_depth,
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
725 userhost))
227
8540825f85e0 Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents: 206
diff changeset
726 except cursor._pool.dbapi.IntegrityError:
167
ef22e4150caa Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents: 163
diff changeset
727 raise error.SubscriptionExists()
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
728
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
729 def removeSubscription(self, subscriber):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
730 return self.dbpool.runInteraction(self._removeSubscription,
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
731 subscriber)
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
732
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
733 def _removeSubscription(self, cursor, subscriber):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
734 self._checkNodeExists(cursor)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
735
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
736 userhost = subscriber.userhost()
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
737 resource = subscriber.resource or ''
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
738
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
739 cursor.execute("""DELETE FROM subscriptions WHERE
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
740 node_id=%s AND
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
741 entity_id=(SELECT entity_id FROM entities
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
742 WHERE jid=%s) AND
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
743 resource=%s""",
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
744 (self.nodeDbId,
136
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
745 userhost,
327de183f48d Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents: 127
diff changeset
746 resource))
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
747 if cursor.rowcount != 1:
171
bc269696ef42 Reply with the correct error condition on subscription when not subscribed.
Ralph Meijer <ralphm@ik.nu>
parents: 170
diff changeset
748 raise error.NotSubscribed()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
749
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
750 return None
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
751
341
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
752 def setSubscriptions(self, subscriptions):
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
753 return self.dbpool.runInteraction(self._setSubscriptions, subscriptions)
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
754
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
755 def _setSubscriptions(self, cursor, subscriptions):
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
756 self._checkNodeExists(cursor)
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
757
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
758 entities = self.getOrCreateEntities(cursor, [s.subscriber for s in subscriptions])
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
759 entities_map = {jid.JID(e.jid): e for e in entities}
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
760
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
761 # then we construct values for subscriptions update according to entity_id we just got
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
762 placeholders = ','.join(len(subscriptions) * ["%s"])
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
763 values = []
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
764 for subscription in subscriptions:
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
765 entity_id = entities_map[subscription.subscriber].entity_id
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
766 resource = subscription.subscriber.resource or ''
341
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
767 values.append((self.nodeDbId, entity_id, resource, subscription.state, None, None))
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
768 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
769 cursor.execute("INSERT INTO subscriptions(node_id, entity_id, resource, state, subscription_type, subscription_depth) VALUES " + placeholders + " ON CONFLICT (entity_id, resource, node_id) DO UPDATE SET state=EXCLUDED.state", [v for v in values])
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
770
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
771 def isSubscribed(self, entity):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
772 return self.dbpool.runInteraction(self._isSubscribed, entity)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
773
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
774 def _isSubscribed(self, cursor, entity):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
775 self._checkNodeExists(cursor)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
776
276
b757c29b20d7 import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents: 271
diff changeset
777 cursor.execute("""SELECT 1 as bool FROM entities
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
778 NATURAL JOIN subscriptions
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
779 NATURAL JOIN nodes
146
b4490bdc77e5 Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents: 145
diff changeset
780 WHERE entities.jid=%s
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
781 AND node_id=%s AND state='subscribed'""",
146
b4490bdc77e5 Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents: 145
diff changeset
782 (entity.userhost(),
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
783 self.nodeDbId))
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
784
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
785 return cursor.fetchone() is not None
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
786
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
787 def getAffiliations(self):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
788 return self.dbpool.runInteraction(self._getAffiliations)
145
f393bccec4bc Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents: 142
diff changeset
789
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
790 def _getAffiliations(self, cursor):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
791 self._checkNodeExists(cursor)
145
f393bccec4bc Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents: 142
diff changeset
792
f393bccec4bc Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents: 142
diff changeset
793 cursor.execute("""SELECT jid, affiliation FROM nodes
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
794 NATURAL JOIN affiliations
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
795 NATURAL JOIN entities
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
796 WHERE node_id=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
797 (self.nodeDbId,))
145
f393bccec4bc Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents: 142
diff changeset
798 result = cursor.fetchall()
180
fc687620599b Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents: 173
diff changeset
799
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
800 return {jid.internJID(r[0]): r[1] for r in result}
145
f393bccec4bc Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents: 142
diff changeset
801
331
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
802 def getOrCreateEntities(self, cursor, entities_jids):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
803 """Get entity_id from entities in entities table
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
804
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
805 Entities will be inserted it they don't exist
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
806 @param entities_jid(list[jid.JID]): entities to get or create
341
b49f75a26156 backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents: 337
diff changeset
807 @return list[record(entity_id,jid)]]: list of entity_id and jid (as plain string)
331
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
808 both existing and inserted entities are returned
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
809 """
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
810 # cf. http://stackoverflow.com/a/35265559
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
811 placeholders = ','.join(len(entities_jids) * ["(%s)"])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
812 query = (
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
813 """
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
814 WITH
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
815 jid_values (jid) AS (
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
816 VALUES {placeholders}
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
817 ),
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
818 inserted (entity_id, jid) AS (
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
819 INSERT INTO entities (jid)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
820 SELECT jid
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
821 FROM jid_values
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
822 ON CONFLICT DO NOTHING
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
823 RETURNING entity_id, jid
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
824 )
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
825 SELECT e.entity_id, e.jid
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
826 FROM entities e JOIN jid_values jv ON jv.jid = e.jid
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
827 UNION ALL
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
828 SELECT entity_id, jid
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
829 FROM inserted""".format(placeholders=placeholders))
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
830 cursor.execute(query, [j.userhost() for j in entities_jids])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
831 return cursor.fetchall()
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
832
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
833 def setAffiliations(self, affiliations):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
834 return self.dbpool.runInteraction(self._setAffiliations, affiliations)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
835
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
836 def _setAffiliations(self, cursor, affiliations):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
837 self._checkNodeExists(cursor)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
838
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
839 entities = self.getOrCreateEntities(cursor, affiliations)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
840
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
841 # then we construct values for affiliations update according to entity_id we just got
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
842 placeholders = ','.join(len(affiliations) * ["(%s,%s,%s)"])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
843 values = []
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
844 for e in entities:
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
845 values.extend((e.entity_id, affiliations[jid.JID(e.jid)], self.nodeDbId))
331
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
846
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
847 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
848 cursor.execute("INSERT INTO affiliations(entity_id,affiliation,node_id) VALUES " + placeholders + " ON CONFLICT (entity_id,node_id) DO UPDATE SET affiliation=EXCLUDED.affiliation", values)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
849
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
850 def deleteAffiliations(self, entities):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
851 return self.dbpool.runInteraction(self._deleteAffiliations, entities)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
852
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
853 def _deleteAffiliations(self, cursor, entities):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
854 """delete affiliations and subscriptions for this entity"""
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
855 self._checkNodeExists(cursor)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
856 placeholders = ','.join(len(entities) * ["%s"])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
857 cursor.execute("DELETE FROM affiliations WHERE node_id=%s AND entity_id in (SELECT entity_id FROM entities WHERE jid IN (" + placeholders + ")) RETURNING entity_id", [self.nodeDbId] + [e.userhost() for e in entities])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
858
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
859 rows = cursor.fetchall()
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
860 placeholders = ','.join(len(rows) * ["%s"])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
861 cursor.execute("DELETE FROM subscriptions WHERE node_id=%s AND entity_id in (" + placeholders + ")", [self.nodeDbId] + [r[0] for r in rows])
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
862
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
863 def getAuthorizedGroups(self):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
864 return self.dbpool.runInteraction(self._getNodeGroups)
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
865
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
866 def _getAuthorizedGroups(self, cursor):
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
867 cursor.execute("SELECT groupname FROM node_groups_authorized NATURAL JOIN nodes WHERE node=%s",
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
868 (self.nodeDbId,))
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
869 rows = cursor.fetchall()
e93a9fd329d9 affiliations XMPP handling:
Goffi <goffi@goffi.org>
parents: 330
diff changeset
870 return [row[0] for row in rows]
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
871
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
872
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
873 @implementer(iidavoll.ILeafNode)
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
874 class LeafNode(Node):
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
875
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
876
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
877 nodeType = 'leaf'
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
878
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
879 def getOrderBy(self, ext_data, direction='DESC'):
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
880 """Return ORDER BY clause corresponding to Order By key in ext_data
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
881
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
882 @param ext_data (dict): extra data as used in getItems
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
883 @param direction (unicode): ORDER BY direction (ASC or DESC)
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
884 @return (unicode): ORDER BY clause to use
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
885 """
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
886 keys = ext_data.get('order_by')
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
887 if not keys:
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
888 return 'ORDER BY updated ' + direction
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
889 cols_statmnt = []
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
890 for key in keys:
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
891 if key == 'creation':
378
22832c1d2827 storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents: 375
diff changeset
892 column = 'item_id' # could work with items.created too
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
893 elif key == 'modification':
378
22832c1d2827 storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents: 375
diff changeset
894 column = 'updated'
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
895 else:
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
896 log.msg("WARNING: Unknown order by key: {key}".format(key=key))
378
22832c1d2827 storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents: 375
diff changeset
897 column = 'updated'
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
898 cols_statmnt.append(column + ' ' + direction)
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
899
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
900 return "ORDER BY " + ",".join([col for col in cols_statmnt])
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
901
393
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
902 @defer.inlineCallbacks
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
903 def storeItems(self, items_data, publisher):
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
904 # XXX: runInteraction doesn't seem to work when there are several "insert"
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
905 # or "update".
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
906 # Before the unpacking was done in _storeItems, but this was causing trouble
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
907 # in case of multiple items_data. So this has now be moved here.
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
908 # FIXME: investigate the issue with runInteraction
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
909 for item_data in items_data:
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
910 yield self.dbpool.runInteraction(self._storeItems, item_data, publisher)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
911
393
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
912 def _storeItems(self, cursor, item_data, publisher):
250
eb14b8d30cba fine tuning per-item permissions
Goffi <goffi@goffi.org>
parents: 249
diff changeset
913 self._checkNodeExists(cursor)
393
728b08c0d000 storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents: 390
diff changeset
914 self._storeItem(cursor, item_data, publisher)
244
3ecc94407e36 item access_model (not finished)
Goffi <goffi@goffi.org>
parents: 243
diff changeset
915
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
916 def _storeItem(self, cursor, item_data, publisher):
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
917 # first try to insert the item
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
918 # - if it fails (conflict), and the item is new and we have serial_ids options,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
919 # current id will be recomputed using next item id query (note that is not perfect, as
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
920 # table is not locked and this can fail if two items are added at the same time
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
921 # but this can only happen with serial_ids and if future ids have been set by a client,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
922 # this case should be rare enough to consider this situation acceptable)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
923 # - if item insertion fail and the item is not new, we do an update
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
924 # - in other cases, exception is raised
301
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
925 item, access_model, item_config = item_data.item, item_data.access_model, item_data.config
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
926 data = item.toXml()
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
927 data_fts_cfg = self._config[const.OPT_FTS_LANGUAGE]
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
928 if data_fts_cfg == const.VAL_FTS_GENERIC:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
929 data_fts_cfg = "simple"
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
930
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
931 insert_query = (
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
932 "INSERT INTO items(node_id, item, publisher, data, access_model, "
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
933 "data_fts_cfg) VALUES (%s, %s, %s, %s, %s, %s) RETURNING item_id"
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
934 )
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
935 insert_data = [self.nodeDbId,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
936 item["id"],
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
937 publisher.full(),
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
938 data,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
939 access_model,
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
940 data_fts_cfg,
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
941 ]
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
942
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
943 try:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
944 cursor.execute(insert_query, insert_data)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
945 except cursor._pool.dbapi.IntegrityError as e:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
946 if e.pgcode != "23505":
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
947 # we only handle unique_violation, every other exception must be raised
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
948 raise e
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
949 cursor.connection.rollback()
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
950 # the item already exist
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
951 if item_data.new:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
952 # the item is new
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
953 if self._config[const.OPT_SERIAL_IDS]:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
954 # this can happen with serial_ids, if a item has been stored
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
955 # with a future id (generated by XMPP client)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
956 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=self.nodeDbId))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
957 next_id = cursor.fetchone()[0]
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
958 # we update the sequence, so we can skip conflicting ids
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
959 cursor.execute("SELECT setval('{seq_name}', %s)".format(
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
960 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId)), [next_id])
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
961 # and now we can retry the query with the new id
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
962 item['id'] = insert_data[1] = str(next_id)
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
963 # item saved in DB must also be updated with the new id
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
964 insert_data[3] = item.toXml()
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
965 cursor.execute(insert_query, insert_data)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
966 else:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
967 # but if we have not serial_ids, we have a real problem
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
968 raise e
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
969 else:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
970 # this is an update
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
971 cursor.execute("""UPDATE items SET updated=now(), publisher=%s, data=%s
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
972 WHERE node_id=%s AND items.item=%s
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
973 RETURNING item_id""",
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
974 (publisher.full(),
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
975 data,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
976 self.nodeDbId,
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
977 item["id"]))
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
978 if cursor.rowcount != 1:
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
979 raise exceptions.InternalError("item has not been updated correctly")
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
980 item_id = cursor.fetchone()[0];
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
981 self._storeCategories(cursor, item_id, item_data.categories, update=True)
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
982 return
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
983
301
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
984 item_id = cursor.fetchone()[0];
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
985 self._storeCategories(cursor, item_id, item_data.categories)
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
986
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
987 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER:
250
eb14b8d30cba fine tuning per-item permissions
Goffi <goffi@goffi.org>
parents: 249
diff changeset
988 if const.OPT_ROSTER_GROUPS_ALLOWED in item_config:
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
989 # XXX: needed to force list if there is only one value
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
990 item_config.fields[const.OPT_ROSTER_GROUPS_ALLOWED].fieldType='list-multi'
250
eb14b8d30cba fine tuning per-item permissions
Goffi <goffi@goffi.org>
parents: 249
diff changeset
991 allowed_groups = item_config[const.OPT_ROSTER_GROUPS_ALLOWED]
245
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
992 else:
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
993 allowed_groups = []
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
994 for group in allowed_groups:
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
995 #TODO: check that group are actually in roster
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
996 cursor.execute("""INSERT INTO item_groups_authorized (item_id, groupname)
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
997 VALUES (%s,%s)""" , (item_id, group))
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
998 # TODO: whitelist access model
245
e11e99246be5 allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents: 244
diff changeset
999
422
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1000 max_items = self._config.get(const.OPT_MAX_ITEMS, 'max')
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1001
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1002 if max_items != 'max':
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1003 try:
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1004 max_items = int(self._config[const.OPT_MAX_ITEMS])
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1005 except ValueError:
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1006 log.err(f"Invalid max_items value: {max_items!r}")
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1007 else:
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1008 if max_items > 0:
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1009 # we delete all items above the requested max
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1010 cursor.execute(
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1011 "DELETE FROM items WHERE node_id=%s and item_id in (SELECT "
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1012 "item_id FROM items WHERE node_id=%s ORDER BY items.updated "
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1013 "DESC, items.item_id DESC OFFSET %s)" ,
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1014 (self.nodeDbId, self.nodeDbId, max_items)
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1015 )
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1016
c21f31355ab9 configuration: "max_items" option:
Goffi <goffi@goffi.org>
parents: 414
diff changeset
1017
301
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1018 def _storeCategories(self, cursor, item_id, categories, update=False):
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1019 # TODO: handle canonical form
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1020 if update:
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1021 cursor.execute("""DELETE FROM item_categories
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1022 WHERE item_id=%s""", (item_id,))
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1023
402
724e39d596a9 psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents: 393
diff changeset
1024 # we use a set to avoid duplicates
724e39d596a9 psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents: 393
diff changeset
1025 for category in set(categories):
301
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1026 cursor.execute("""INSERT INTO item_categories (item_id, category)
05c875a13a62 categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents: 297
diff changeset
1027 VALUES (%s, %s)""", (item_id, category))
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
1028
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1029 def removeItems(self, itemIdentifiers):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
1030 return self.dbpool.runInteraction(self._removeItems, itemIdentifiers)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1031
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1032 def _removeItems(self, cursor, itemIdentifiers):
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1033 self._checkNodeExists(cursor)
180
fc687620599b Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents: 173
diff changeset
1034
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1035 deleted = []
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1036
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1037 for itemIdentifier in itemIdentifiers:
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1038 cursor.execute("""DELETE FROM items WHERE
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1039 node_id=%s AND
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1040 item=%s""",
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1041 (self.nodeDbId,
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1042 itemIdentifier))
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1043
142
812300cdbc22 Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents: 136
diff changeset
1044 if cursor.rowcount:
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1045 deleted.append(itemIdentifier)
142
812300cdbc22 Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents: 136
diff changeset
1046
812300cdbc22 Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents: 136
diff changeset
1047 return deleted
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
1048
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1049 def getItems(self, authorized_groups, unrestricted, maxItems=None, ext_data=None):
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1050 """ Get all authorised items
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1051
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1052 @param authorized_groups: we want to get items that these groups can access
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1053 @param unrestricted: if true, don't check permissions (i.e.: get all items)
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1054 @param maxItems: nb of items we want to get
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1055 @param ext_data: options for extra features like RSM and MAM
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1056
317
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1057 @return: list of container.ItemData
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1058 if unrestricted is False, access_model and config will be None
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1059 """
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1060 if ext_data is None:
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1061 ext_data = {}
347
f33406fcab5c pgsql: fixed _getItems call
Goffi <goffi@goffi.org>
parents: 346
diff changeset
1062 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=False)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1063
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1064 def getItemsIds(self, authorized_groups, unrestricted, maxItems=None, ext_data=None):
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1065 """ Get all authorised items ids
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1066
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1067 @param authorized_groups: we want to get items that these groups can access
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1068 @param unrestricted: if true, don't check permissions (i.e.: get all items)
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1069 @param maxItems: nb of items we want to get
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1070 @param ext_data: options for extra features like RSM and MAM
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1071
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1072 @return list(unicode): list of ids
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1073 """
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1074 if ext_data is None:
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1075 ext_data = {}
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1076 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=True)
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1077
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1078 def _appendSourcesAndFilters(self, query, args, authorized_groups, unrestricted, ext_data):
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1079 """append sources and filters to sql query requesting items and return ORDER BY
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1080
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1081 arguments query, args, authorized_groups, unrestricted and ext_data are the same as for
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1082 _getItems
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1083 """
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1084 # SOURCES
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1085 query.append("FROM nodes INNER JOIN items USING (node_id)")
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1086
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1087 if unrestricted:
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1088 query_filters = ["WHERE node_id=%s"]
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1089 args.append(self.nodeDbId)
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1090 else:
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1091 query.append("LEFT JOIN item_groups_authorized USING (item_id)")
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1092 args.append(self.nodeDbId)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1093 if authorized_groups:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1094 get_groups = " or (items.access_model='roster' and groupname in %s)"
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1095 args.append(authorized_groups)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1096 else:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1097 get_groups = ""
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1098
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1099 query_filters = ["WHERE node_id=%s AND (items.access_model='open'" + get_groups + ")"]
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1100
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1101 # FILTERS
282
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1102 if 'filters' in ext_data: # MAM filters
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1103 for filter_ in ext_data['filters']:
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1104 if filter_.var == 'start':
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1105 query_filters.append("AND created>=%s")
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1106 args.append(filter_.value)
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1107 elif filter_.var == 'end':
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1108 query_filters.append("AND created<=%s")
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1109 args.append(filter_.value)
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1110 elif filter_.var == 'with':
282
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1111 jid_s = filter_.value
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1112 if '/' in jid_s:
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1113 query_filters.append("AND publisher=%s")
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1114 args.append(filter_.value)
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1115 else:
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1116 query_filters.append("AND publisher LIKE %s")
414
ccb2a22ea0fc Python 3 port:
Goffi <goffi@goffi.org>
parents: 406
diff changeset
1117 args.append("{}%".format(filter_.value))
430
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1118 elif filter_.var == const.MAM_FILTER_FTS:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1119 fts_cfg = self._config[const.OPT_FTS_LANGUAGE]
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1120 if fts_cfg == const.VAL_FTS_GENERIC:
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1121 fts_cfg = "simple"
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1122 query_filters.append(
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1123 "AND data_fts @@ websearch_to_tsquery(%s, %s)"
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1124 )
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1125 args.append(fts_cfg)
5a0ada3b61ca Full-Text Search implementation:
Goffi <goffi@goffi.org>
parents: 429
diff changeset
1126 args.append(filter_.value)
323
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1127 elif filter_.var == const.MAM_FILTER_CATEGORY:
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1128 query.append("LEFT JOIN item_categories USING (item_id)")
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1129 query_filters.append("AND category=%s")
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1130 args.append(filter_.value)
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1131 else:
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1132 log.msg("WARNING: unknown filter: {}".format(filter_.encode('utf-8')))
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1133
8496af26be45 mam: added filtering by category
Goffi <goffi@goffi.org>
parents: 322
diff changeset
1134 query.extend(query_filters)
282
7d54ff2eeaf2 actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents: 278
diff changeset
1135
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
1136 return self.getOrderBy(ext_data)
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1137
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1138 def _getItems(self, cursor, authorized_groups, unrestricted, maxItems, ext_data, ids_only):
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1139 self._checkNodeExists(cursor)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1140
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1141 if maxItems == 0:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1142 return []
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1143
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1144 args = []
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1145
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1146 # SELECT
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1147 if ids_only:
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1148 query = ["SELECT item"]
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1149 else:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1150 query = ["SELECT data::text,items.access_model,item_id,created,updated"]
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1151
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1152 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data)
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1153
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1154 if 'rsm' in ext_data:
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1155 rsm = ext_data['rsm']
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1156 maxItems = rsm.max
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1157 if rsm.index is not None:
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1158 # We need to know the item_id of corresponding to the index (offset) of the current query
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1159 # so we execute the query to look for the item_id
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1160 tmp_query = query[:]
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1161 tmp_args = args[:]
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1162 tmp_query[0] = "SELECT item_id"
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1163 tmp_query.append("{} LIMIT 1 OFFSET %s".format(query_order))
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1164 tmp_args.append(rsm.index)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1165 cursor.execute(' '.join(query), args)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1166 # FIXME: bad index is not managed yet
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1167 item_id = cursor.fetchall()[0][0]
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1168
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1169 # now that we have the id, we can use it
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1170 query.append("AND item_id<=%s")
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1171 args.append(item_id)
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1172 elif rsm.before is not None:
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1173 if rsm.before != '':
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1174 query.append("AND item_id>(SELECT item_id FROM items WHERE item=%s LIMIT 1)")
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1175 args.append(rsm.before)
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1176 if maxItems is not None:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1177 # if we have maxItems (i.e. a limit), we need to reverse order
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1178 # in a first query to get the right items
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1179 query.insert(0,"SELECT * from (")
375
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
1180 query.append(self.getOrderBy(ext_data, direction='ASC'))
9a787881b824 implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents: 374
diff changeset
1181 query.append("LIMIT %s) as x")
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1182 args.append(maxItems)
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1183 elif rsm.after:
319
a51947371625 pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents: 318
diff changeset
1184 query.append("AND item_id<(SELECT item_id FROM items WHERE item=%s LIMIT 1)")
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1185 args.append(rsm.after)
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1186
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1187 query.append(query_order)
278
8a71486c3e95 implements RSM (XEP-0059)
souliane <souliane@mailoo.org>
parents: 276
diff changeset
1188
297
4115999d85e9 fixes behavior when maxItems is 0
souliane <souliane@mailoo.org>
parents: 294
diff changeset
1189 if maxItems is not None:
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1190 query.append("LIMIT %s")
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1191 args.append(maxItems)
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
1192
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1193 cursor.execute(' '.join(query), args)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1194
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1195 result = cursor.fetchall()
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1196 if unrestricted and not ids_only:
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1197 # with unrestricted query, we need to fill the access_list for a roster access items
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1198 ret = []
356
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1199 for item_data in result:
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1200 item = generic.stripNamespace(parseXml(item_data.data))
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1201 access_model = item_data.access_model
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1202 item_id = item_data.item_id
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1203 created = item_data.created
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1204 updated = item_data.updated
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1205 access_list = {}
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1206 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER:
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1207 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,))
356
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1208 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r.groupname for r in cursor.fetchall()]
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1209
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1210 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated))
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1211 # TODO: whitelist item access model
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1212 return ret
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1213
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1214 if ids_only:
356
95c83899b5e9 pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents: 352
diff changeset
1215 return [r.item for r in result]
346
3bbab2173ebc implemented disco items:
Goffi <goffi@goffi.org>
parents: 344
diff changeset
1216 else:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1217 items_data = [container.ItemData(generic.stripNamespace(parseXml(r.data)), r.access_model, created=r.created, updated=r.updated) for r in result]
317
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1218 return items_data
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
1219
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1220 def getItemsById(self, authorized_groups, unrestricted, itemIdentifiers):
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1221 """Get items which are in the given list
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1222
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1223 @param authorized_groups: we want to get items that these groups can access
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1224 @param unrestricted: if true, don't check permissions
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1225 @param itemIdentifiers: list of ids of the items we want to get
317
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1226 @return: list of container.ItemData
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1227 ItemData.config will contains access_list (managed as a dictionnary with same key as for item_config)
34adc4a8aa64 new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents: 312
diff changeset
1228 if unrestricted is False, access_model and config will be None
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1229 """
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1230 return self.dbpool.runInteraction(self._getItemsById, authorized_groups, unrestricted, itemIdentifiers)
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1231
248
50f6ee966da8 item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents: 247
diff changeset
1232 def _getItemsById(self, cursor, authorized_groups, unrestricted, itemIdentifiers):
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1233 self._checkNodeExists(cursor)
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1234 ret = []
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1235 if unrestricted: #we get everything without checking permissions
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1236 for itemIdentifier in itemIdentifiers:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1237 cursor.execute("""SELECT data::text,items.access_model,item_id,created,updated FROM nodes
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1238 INNER JOIN items USING (node_id)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1239 WHERE node_id=%s AND item=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1240 (self.nodeDbId,
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1241 itemIdentifier))
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1242 result = cursor.fetchone()
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1243 if not result:
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1244 raise error.ItemNotFound()
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
1245
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1246 item = generic.stripNamespace(parseXml(result[0]))
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1247 access_model = result[1]
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1248 item_id = result[2]
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1249 created= result[3]
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1250 updated= result[4]
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1251 access_list = {}
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1252 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER:
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1253 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,))
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1254 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r[0] for r in cursor.fetchall()]
330
82d1259b3e36 backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents: 324
diff changeset
1255 #TODO: WHITELIST access_model
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1256
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1257 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated))
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1258 else: #we check permission before returning items
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1259 for itemIdentifier in itemIdentifiers:
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1260 args = [self.nodeDbId, itemIdentifier]
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1261 if authorized_groups:
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1262 args.append(authorized_groups)
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1263 cursor.execute("""SELECT data::text, created, updated FROM nodes
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1264 INNER JOIN items USING (node_id)
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1265 LEFT JOIN item_groups_authorized USING (item_id)
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1266 WHERE node_id=%s AND item=%s AND
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
1267 (items.access_model='open' """ +
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1268 ("or (items.access_model='roster' and groupname in %s)" if authorized_groups else '') + ")",
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1269 args)
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1270
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1271 result = cursor.fetchone()
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1272 if result:
367
a772f7dac930 backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents: 366
diff changeset
1273 ret.append(container.ItemData(generic.stripNamespace(parseXml(result[0])), created=result[1], updated=result[2]))
262
7b821432d012 fixed node auto-create feature
souliane <souliane@mailoo.org>
parents: 260
diff changeset
1274
252
25a1dc7181cc full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents: 250
diff changeset
1275 return ret
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1276
318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1277 def getItemsCount(self, authorized_groups, unrestricted, ext_data=None):
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1278 """Count expected number of items in a getItems query
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1279
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1280 @param authorized_groups: we want to get items that these groups can access
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1281 @param unrestricted: if true, don't check permissions (i.e.: get all items)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1282 @param ext_data: options for extra features like RSM and MAM
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1283 """
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1284 if ext_data is None:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1285 ext_data = {}
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1286 return self.dbpool.runInteraction(self._getItemsCount, authorized_groups, unrestricted, ext_data)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1287
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1288 def _getItemsCount(self, cursor, authorized_groups, unrestricted, ext_data):
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1289 self._checkNodeExists(cursor)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1290 args = []
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1291
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1292 # SELECT
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1293 query = ["SELECT count(1)"]
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1294
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1295 self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1296
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1297 cursor.execute(' '.join(query), args)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1298 return cursor.fetchall()[0][0]
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1299
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1300 def getItemsIndex(self, item_id, authorized_groups, unrestricted, ext_data=None):
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1301 """Get expected index of first item in the window of a getItems query
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1302
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1303 @param item_id: id of the item
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1304 @param authorized_groups: we want to get items that these groups can access
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1305 @param unrestricted: if true, don't check permissions (i.e.: get all items)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1306 @param ext_data: options for extra features like RSM and MAM
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1307 """
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1308 if ext_data is None:
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1309 ext_data = {}
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1310 return self.dbpool.runInteraction(self._getItemsIndex, item_id, authorized_groups, unrestricted, ext_data)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1311
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1312 def _getItemsIndex(self, cursor, item_id, authorized_groups, unrestricted, ext_data):
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1313 self._checkNodeExists(cursor)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1314 args = []
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1315
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1316 # SELECT
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1317 query = []
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1318
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1319 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1320
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1321 query_select = "SELECT row_number from (SELECT row_number() OVER ({}), item".format(query_order)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1322 query.insert(0, query_select)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1323 query.append(") as x WHERE item=%s")
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1324 args.append(item_id)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1325
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1326 cursor.execute(' '.join(query), args)
d13526c0eb32 RSM improvments/refactoring:
Goffi <goffi@goffi.org>
parents: 317
diff changeset
1327 # XXX: row_number start at 1, but we want that index start at 0
324
e73e42b4f6ff pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents: 323
diff changeset
1328 try:
e73e42b4f6ff pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents: 323
diff changeset
1329 return cursor.fetchall()[0][0] - 1
e73e42b4f6ff pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents: 323
diff changeset
1330 except IndexError:
e73e42b4f6ff pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents: 323
diff changeset
1331 raise error.NodeNotFound()
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1332
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1333 def getItemsPublishers(self, itemIdentifiers):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1334 """Get the publishers for all given identifiers
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1335
308
087b705493a6 fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents: 301
diff changeset
1336 @return (dict[unicode, jid.JID]): map of itemIdentifiers to publisher
087b705493a6 fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents: 301
diff changeset
1337 if item is not found, key is skipped in resulting dict
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1338 """
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1339 return self.dbpool.runInteraction(self._getItemsPublishers, itemIdentifiers)
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1340
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1341 def _getItemsPublishers(self, cursor, itemIdentifiers):
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1342 self._checkNodeExists(cursor)
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1343 ret = {}
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1344 for itemIdentifier in itemIdentifiers:
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1345 cursor.execute("""SELECT publisher FROM items
390
fba96e95e329 storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents: 384
diff changeset
1346 WHERE node_id=%s AND item=%s""",
fba96e95e329 storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents: 384
diff changeset
1347 (self.nodeDbId, itemIdentifier,))
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1348 result = cursor.fetchone()
308
087b705493a6 fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents: 301
diff changeset
1349 if result:
087b705493a6 fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents: 301
diff changeset
1350 ret[itemIdentifier] = jid.JID(result[0])
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1351 return ret
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1352
107
d252d793f0ed Initial revision.
Ralph Meijer <ralphm@ik.nu>
parents:
diff changeset
1353 def purge(self):
204
b4bf0a5ce50d Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents: 202
diff changeset
1354 return self.dbpool.runInteraction(self._purge)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1355
127
d3689da18ed2 Don't use encode('utf-8') on serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents: 124
diff changeset
1356 def _purge(self, cursor):
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1357 self._checkNodeExists(cursor)
121
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1358
4f0113adb7ed Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents: 113
diff changeset
1359 cursor.execute("""DELETE FROM items WHERE
294
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1360 node_id=%s""",
df1edebb0466 PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents: 283
diff changeset
1361 (self.nodeDbId,))
198
e404775b12df Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents: 181
diff changeset
1362
263
9dfd3890e646 added the constant FLAG_RETRACT_ALLOW_PUBLISHER to allow a publisher to retract an item he has published in a node of "open" publish model.
souliane <souliane@mailoo.org>
parents: 262
diff changeset
1363
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
1364 class CollectionNode(Node):
146
b4490bdc77e5 Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents: 145
diff changeset
1365
206
274a45d2a5ab Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents: 204
diff changeset
1366 nodeType = 'collection'