Mercurial > libervia-pubsub
annotate sat_pubsub/pgsql_storage.py @ 430:5a0ada3b61ca
Full-Text Search implementation:
/!\ pgsql schema needs to be updated /!\
/!\ Minimal PostgreSQL required version is now 12 /!\
A new options is available to specify main language of a node. By default a `generic`
language is used (which uses the `simple` configuration in PostgreSQL). When a node owner
changes the language, the index is rebuilt accordingly. It is possible to have item
specific language for multilingual nodes (but for the moment the search is done with node
language, so the results won't be good). If an item language is explicitely set in
`item_languages`, the FTS configuration won't be affected by node FTS language setting.
Search is parsed with `websearch_to_tsquery` for now, but this parser doesn't handle
prefix matching, so it may be replaced in the future.
SetConfiguration now only updates the modified values, this avoid triggering the FTS
re-indexing on each config change. `_checkNodeExists` is not called anymore as we can
check if a row has been modified to see if the node exists, this avoid a useless query.
Item storing has been slighly improved with a useless SELECT and condition removed.
To avoid 2 schema updates in a row, the `sat_pubsub_update_5_6.sql` file also prepares the
implementation of XEP-0346 by updating nodes with a schema and creating the suitable
template nodes.
author | Goffi <goffi@goffi.org> |
---|---|
date | Fri, 11 Dec 2020 17:18:52 +0100 |
parents | 0526073ff2ab |
children | 920440200570 |
rev | line source |
---|---|
414 | 1 #!/usr/bin/env python3 |
233 | 2 #-*- coding: utf-8 -*- |
3 | |
384 | 4 # Copyright (c) 2012-2019 Jérôme Poisson |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
5 # Copyright (c) 2013-2016 Adrien Cossa |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
6 # Copyright (c) 2003-2011 Ralph Meijer |
233 | 7 |
8 | |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
9 # This program is free software: you can redistribute it and/or modify |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
10 # it under the terms of the GNU Affero General Public License as published by |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
11 # the Free Software Foundation, either version 3 of the License, or |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
12 # (at your option) any later version. |
233 | 13 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
14 # This program is distributed in the hope that it will be useful, |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
15 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
16 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
17 # GNU Affero General Public License for more details. |
233 | 18 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
19 # You should have received a copy of the GNU Affero General Public License |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
20 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
21 # -- |
233 | 22 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
23 # This program is based on Idavoll (http://idavoll.ik.nu/), |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
24 # originaly written by Ralph Meijer (http://ralphm.net/blog/) |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
25 # It is sublicensed under AGPL v3 (or any later version) as allowed by the original |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
26 # license. |
233 | 27 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
28 # -- |
233 | 29 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
30 # Here is a copy of the original license: |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
31 |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
32 # Copyright (c) 2003-2011 Ralph Meijer |
233 | 33 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
34 # Permission is hereby granted, free of charge, to any person obtaining |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
35 # a copy of this software and associated documentation files (the |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
36 # "Software"), to deal in the Software without restriction, including |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
37 # without limitation the rights to use, copy, modify, merge, publish, |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
38 # distribute, sublicense, and/or sell copies of the Software, and to |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
39 # permit persons to whom the Software is furnished to do so, subject to |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
40 # the following conditions: |
233 | 41 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
42 # The above copyright notice and this permission notice shall be |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
43 # included in all copies or substantial portions of the Software. |
233 | 44 |
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
45 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
46 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
47 # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
48 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
49 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
50 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION |
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
51 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
233 | 52 |
155
5191ba7c4df8
Work towards first release 0.5.0.
Ralph Meijer <ralphm@ik.nu>
parents:
148
diff
changeset
|
53 |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
54 import copy, logging |
202
77c61e2b8c75
Use `domish.Element`s to represent items, instead of serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents:
198
diff
changeset
|
55 |
414 | 56 from zope.interface import implementer |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
57 |
344
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
58 from twisted.internet import reactor |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
59 from twisted.internet import defer |
107 | 60 from twisted.words.protocols.jabber import jid |
323 | 61 from twisted.python import log |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
62 |
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
63 from wokkel import generic |
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
64 from wokkel.pubsub import Subscription, BadRequest |
107 | 65 |
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
66 from sat_pubsub import error |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
67 from sat_pubsub import iidavoll |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
68 from sat_pubsub import const |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
69 from sat_pubsub import container |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
70 from sat_pubsub import exceptions |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
71 import uuid |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
72 import psycopg2 |
270
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
73 import psycopg2.extensions |
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
74 # we wants psycopg2 to return us unicode, not str |
270
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
75 psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) |
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
76 psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY) |
244 | 77 |
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
78 # parseXml manage str, but we get unicode |
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
79 parseXml = lambda unicode_data: generic.parseXml(unicode_data.encode('utf-8')) |
414 | 80 ITEMS_SEQ_NAME = 'node_{node_id}_seq' |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
81 PEP_COL_NAME = 'pep' |
430 | 82 CURRENT_VERSION = '6' |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
83 # retrieve the maximum integer item id + 1 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
84 NEXT_ITEM_ID_QUERY = r"SELECT COALESCE(max(item::integer)+1,1) as val from items where node_id={node_id} and item ~ E'^\\d+$'" |
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
85 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
86 |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
87 def withPEP(query, values, pep, recipient): |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
88 """Helper method to facilitate PEP management |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
89 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
90 @param query: SQL query basis |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
91 @param values: current values to replace in query |
364
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
92 @param pep(bool): True if we are in PEP mode |
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
93 @param recipient(jid.JID): jid of the recipient |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
94 @return: query + PEP AND check, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
95 recipient's bare jid is added to value if needed |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
96 """ |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
97 if pep: |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
98 pep_check="AND {}=%s".format(PEP_COL_NAME) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
99 values=list(values) + [recipient.userhost()] |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
100 else: |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
101 pep_check="AND {} IS NULL".format(PEP_COL_NAME) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
102 return "{} {}".format(query, pep_check), values |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
103 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
104 |
414 | 105 @implementer(iidavoll.IStorage) |
107 | 106 class Storage: |
107 | |
430 | 108 fts_languages = ['generic'] |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
109 defaultConfig = { |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
110 'leaf': { |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
111 const.OPT_PERSIST_ITEMS: True, |
422 | 112 const.OPT_MAX_ITEMS: 'max', |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
113 const.OPT_DELIVER_PAYLOADS: True, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
114 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub', |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
115 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT, |
260 | 116 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT, |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
117 const.OPT_SERIAL_IDS: False, |
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
118 const.OPT_CONSISTENT_PUBLISHER: False, |
430 | 119 const.OPT_FTS_LANGUAGE: const.VAL_FTS_GENERIC, |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
120 }, |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
121 'collection': { |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
122 const.OPT_DELIVER_PAYLOADS: True, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
123 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub', |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
124 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT, |
260 | 125 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT, |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
126 } |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
127 } |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
128 |
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
129 def __init__(self, dbpool): |
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
130 self.dbpool = dbpool |
344
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
131 d = self.dbpool.runQuery("SELECT value FROM metadata WHERE key='version'") |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
132 d.addCallbacks(self._checkVersion, self._versionEb) |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
133 |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
134 def _checkVersion(self, row): |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
135 version = row[0].value |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
136 if version != CURRENT_VERSION: |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
137 logging.error("Bad database schema version ({current}), please upgrade to {needed}".format( |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
138 current=version, needed=CURRENT_VERSION)) |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
139 reactor.stop() |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
140 |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
141 def _versionEb(self, failure): |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
142 logging.error("Can't check schema version: {reason}".format(reason=failure)) |
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
143 reactor.stop() |
107 | 144 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
145 def _buildNode(self, row): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
146 """Build a note class from database result row""" |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
147 if not row: |
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
148 raise error.NodeNotFound() |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
149 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
150 if row[2] == 'leaf': |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
151 configuration = { |
422 | 152 const.OPT_PERSIST_ITEMS: row[3], |
153 const.OPT_MAX_ITEMS: 'max' if row[4] == 0 else str(row[4]), | |
154 const.OPT_DELIVER_PAYLOADS: row[5], | |
155 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6], | |
156 const.OPT_ACCESS_MODEL:row[7], | |
157 const.OPT_PUBLISH_MODEL:row[8], | |
158 const.OPT_SERIAL_IDS:row[9], | |
159 const.OPT_CONSISTENT_PUBLISHER:row[10], | |
430 | 160 const.OPT_FTS_LANGUAGE: row[11], |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
161 } |
430 | 162 schema = row[12] |
352 | 163 if schema is not None: |
164 schema = parseXml(schema) | |
165 node = LeafNode(row[0], row[1], configuration, schema) | |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
166 node.dbpool = self.dbpool |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
167 return node |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
168 elif row[2] == 'collection': |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
169 configuration = { |
422 | 170 const.OPT_DELIVER_PAYLOADS: row[5], |
171 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6], | |
172 const.OPT_ACCESS_MODEL: row[7], | |
173 const.OPT_PUBLISH_MODEL:row[8], | |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
174 } |
352 | 175 node = CollectionNode(row[0], row[1], configuration, None) |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
176 node.dbpool = self.dbpool |
107 | 177 return node |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
178 else: |
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
179 raise BadRequest(text="Unknown node type !") |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
180 |
430 | 181 def getFTSLanguages(self): |
182 """Get list of available languages for full text search""" | |
183 return self.dbpool.runInteraction(self._getFTSLanguages) | |
184 | |
185 def _getFTSLanguages(self, cursor): | |
186 cursor.execute("SELECT cfgname FROM pg_ts_config") | |
187 result = [r.cfgname for r in cursor.fetchall()] | |
188 result.remove("simple") | |
189 result.insert(0, "generic") | |
190 Node.fts_languages = self.fts_languages = result | |
191 return result | |
192 | |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
193 def getNodeById(self, nodeDbId): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
194 """Get node using database ID insted of pubsub identifier |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
195 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
196 @param nodeDbId(unicode): database ID |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
197 """ |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
198 return self.dbpool.runInteraction(self._getNodeById, nodeDbId) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
199 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
200 def _getNodeById(self, cursor, nodeDbId): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
201 cursor.execute("""SELECT node_id, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
202 node, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
203 node_type, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
204 persist_items, |
422 | 205 max_items, |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
206 deliver_payloads, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
207 send_last_published_item, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
208 access_model, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
209 publish_model, |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
210 serial_ids, |
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
211 consistent_publisher, |
430 | 212 fts_language, |
352 | 213 schema::text, |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
214 pep |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
215 FROM nodes |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
216 WHERE node_id=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
217 (nodeDbId,)) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
218 row = cursor.fetchone() |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
219 return self._buildNode(row) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
220 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
221 def getNode(self, nodeIdentifier, pep, recipient=None): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
222 return self.dbpool.runInteraction(self._getNode, nodeIdentifier, pep, recipient) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
223 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
224 def _getNode(self, cursor, nodeIdentifier, pep, recipient): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
225 cursor.execute(*withPEP("""SELECT node_id, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
226 node, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
227 node_type, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
228 persist_items, |
422 | 229 max_items, |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
230 deliver_payloads, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
231 send_last_published_item, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
232 access_model, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
233 publish_model, |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
234 serial_ids, |
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
235 consistent_publisher, |
430 | 236 fts_language, |
352 | 237 schema::text, |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
238 pep |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
239 FROM nodes |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
240 WHERE node=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
241 (nodeIdentifier,), pep, recipient)) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
242 row = cursor.fetchone() |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
243 return self._buildNode(row) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
244 |
349
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
245 def getNodeIds(self, pep, recipient, allowed_accesses=None): |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
246 """retrieve ids of existing nodes |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
247 |
364
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
248 @param pep(bool): True if it's a PEP request |
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
249 @param recipient(jid.JID, None): recipient of the PEP request |
349
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
250 @param allowed_accesses(None, set): only nodes with access |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
251 in this set will be returned |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
252 None to return all nodes |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
253 @return (list[unicode]): ids of nodes |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
254 """ |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
255 if not pep: |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
256 query = "SELECT node from nodes WHERE pep is NULL" |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
257 values = [] |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
258 else: |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
259 query = "SELECT node from nodes WHERE pep=%s" |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
260 values = [recipient.userhost()] |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
261 |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
262 if allowed_accesses is not None: |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
263 query += "AND access_model IN %s" |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
264 values.append(tuple(allowed_accesses)) |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
265 |
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
266 d = self.dbpool.runQuery(query, values) |
107 | 267 d.addCallback(lambda results: [r[0] for r in results]) |
268 return d | |
269 | |
352 | 270 def createNode(self, nodeIdentifier, owner, config, schema, pep, recipient=None): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
271 return self.dbpool.runInteraction(self._createNode, nodeIdentifier, |
352 | 272 owner, config, schema, pep, recipient) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
273 |
352 | 274 def _createNode(self, cursor, nodeIdentifier, owner, config, schema, pep, recipient): |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
275 if config['pubsub#node_type'] != 'leaf': |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
276 raise error.NoCollections() |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
277 |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
278 owner = owner.userhost() |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
279 |
107 | 280 try: |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
281 cursor.execute("""INSERT INTO nodes |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
282 (node, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
283 node_type, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
284 persist_items, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
285 deliver_payloads, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
286 send_last_published_item, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
287 access_model, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
288 publish_model, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
289 serial_ids, |
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
290 consistent_publisher, |
430 | 291 fts_language, |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
292 schema, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
293 pep) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
294 VALUES |
430 | 295 (%s, 'leaf', %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)""", |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
296 (nodeIdentifier, |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
297 config['pubsub#persist_items'], |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
298 config['pubsub#deliver_payloads'], |
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
299 config['pubsub#send_last_published_item'], |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
300 config[const.OPT_ACCESS_MODEL], |
260 | 301 config[const.OPT_PUBLISH_MODEL], |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
302 config[const.OPT_SERIAL_IDS], |
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
303 config[const.OPT_CONSISTENT_PUBLISHER], |
430 | 304 config[const.OPT_FTS_LANGUAGE], |
352 | 305 schema, |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
306 recipient.userhost() if pep else None |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
307 ) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
308 ) |
332
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
309 except cursor._pool.dbapi.IntegrityError as e: |
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
310 if e.pgcode == "23505": |
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
311 # unique_violation |
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
312 raise error.NodeExists() |
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
313 else: |
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
314 raise error.InvalidConfigurationOption() |
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
315 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
316 cursor.execute(*withPEP("""SELECT node_id FROM nodes WHERE node=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
317 (nodeIdentifier,), pep, recipient)); |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
318 node_id = cursor.fetchone()[0] |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
319 |
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
320 cursor.execute("""SELECT 1 as bool from entities where jid=%s""", |
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
321 (owner,)) |
107 | 322 |
323 if not cursor.fetchone(): | |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
324 # XXX: we can NOT rely on the previous query! Commit is needed now because |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
325 # if the entry exists the next query will leave the database in a corrupted |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
326 # state: the solution is to rollback. I tried with other methods like |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
327 # "WHERE NOT EXISTS" but none of them worked, so the following solution |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
328 # looks like the sole - unless you have auto-commit on. More info |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
329 # about this issue: http://cssmay.com/question/tag/tag-psycopg2 |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
330 cursor.connection.commit() |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
331 try: |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
332 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""", |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
333 (owner,)) |
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
334 except psycopg2.IntegrityError as e: |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
335 cursor.connection.rollback() |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
336 logging.warning("during node creation: %s" % e.message) |
107 | 337 |
338 cursor.execute("""INSERT INTO affiliations | |
339 (node_id, entity_id, affiliation) | |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
340 SELECT %s, entity_id, 'owner' FROM |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
341 (SELECT entity_id FROM entities |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
342 WHERE jid=%s) as e""", |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
343 (node_id, owner)) |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
344 |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
345 if config[const.OPT_ACCESS_MODEL] == const.VAL_AMODEL_PUBLISHER_ROSTER: |
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
346 if const.OPT_ROSTER_GROUPS_ALLOWED in config: |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
347 allowed_groups = config[const.OPT_ROSTER_GROUPS_ALLOWED] |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
348 else: |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
349 allowed_groups = [] |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
350 for group in allowed_groups: |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
351 #TODO: check that group are actually in roster |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
352 cursor.execute("""INSERT INTO node_groups_authorized (node_id, groupname) |
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
353 VALUES (%s,%s)""" , (node_id, group)) |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
354 # XXX: affiliations can't be set on during node creation (at least not with XEP-0060 alone) |
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
355 # so whitelist affiliations need to be done afterward |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
356 |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
357 # no we may have to do extra things according to config options |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
358 default_conf = self.defaultConfig['leaf'] |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
359 # XXX: trigger works on node creation because OPT_SERIAL_IDS is False in defaultConfig |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
360 # if this value is changed, the _configurationTriggers method should be adapted. |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
361 Node._configurationTriggers(cursor, node_id, default_conf, config) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
362 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
363 def deleteNodeByDbId(self, db_id): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
364 """Delete a node using directly its database id""" |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
365 return self.dbpool.runInteraction(self._deleteNodeByDbId, db_id) |
107 | 366 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
367 def _deleteNodeByDbId(self, cursor, db_id): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
368 cursor.execute("""DELETE FROM nodes WHERE node_id=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
369 (db_id,)) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
370 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
371 if cursor.rowcount != 1: |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
372 raise error.NodeNotFound() |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
373 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
374 def deleteNode(self, nodeIdentifier, pep, recipient=None): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
375 return self.dbpool.runInteraction(self._deleteNode, nodeIdentifier, pep, recipient) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
376 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
377 def _deleteNode(self, cursor, nodeIdentifier, pep, recipient): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
378 cursor.execute(*withPEP("""DELETE FROM nodes WHERE node=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
379 (nodeIdentifier,), pep, recipient)) |
107 | 380 |
381 if cursor.rowcount != 1: | |
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
382 raise error.NodeNotFound() |
107 | 383 |
331 | 384 def getAffiliations(self, entity, nodeIdentifier, pep, recipient=None): |
385 return self.dbpool.runInteraction(self._getAffiliations, entity, nodeIdentifier, pep, recipient) | |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
386 |
331 | 387 def _getAffiliations(self, cursor, entity, nodeIdentifier, pep, recipient=None): |
388 query = ["""SELECT node, affiliation FROM entities | |
389 NATURAL JOIN affiliations | |
390 NATURAL JOIN nodes | |
391 WHERE jid=%s"""] | |
392 args = [entity.userhost()] | |
393 | |
394 if nodeIdentifier is not None: | |
395 query.append("AND node=%s") | |
396 args.append(nodeIdentifier) | |
397 | |
398 cursor.execute(*withPEP(' '.join(query), args, pep, recipient)) | |
399 rows = cursor.fetchall() | |
400 return [tuple(r) for r in rows] | |
107 | 401 |
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
402 def getSubscriptions(self, entity, nodeIdentifier=None, pep=False, recipient=None): |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
403 """retrieve subscriptions of an entity |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
404 |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
405 @param entity(jid.JID): entity to check |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
406 @param nodeIdentifier(unicode, None): node identifier |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
407 None to retrieve all subscriptions |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
408 @param pep: True if we are in PEP mode |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
409 @param recipient: jid of the recipient |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
410 """ |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
411 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
412 def toSubscriptions(rows): |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
413 subscriptions = [] |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
414 for row in rows: |
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
415 subscriber = jid.internJID('%s/%s' % (row.jid, |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
416 row.resource)) |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
417 subscription = Subscription(row.node, subscriber, row.state) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
418 subscriptions.append(subscription) |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
419 return subscriptions |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
420 |
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
421 query = ["""SELECT node, |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
422 jid, |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
423 resource, |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
424 state |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
425 FROM entities |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
426 NATURAL JOIN subscriptions |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
427 NATURAL JOIN nodes |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
428 WHERE jid=%s"""] |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
429 |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
430 args = [entity.userhost()] |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
431 |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
432 if nodeIdentifier is not None: |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
433 query.append("AND node=%s") |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
434 args.append(nodeIdentifier) |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
435 |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
436 d = self.dbpool.runQuery(*withPEP(' '.join(query), args, pep, recipient)) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
437 d.addCallback(toSubscriptions) |
107 | 438 return d |
439 | |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
440 def getDefaultConfiguration(self, nodeType): |
366
81e6d4a516c3
storage (pgsql): getDefaultConfiguration now returns a shallow copy of default conf dict, to avoid modification.
Goffi <goffi@goffi.org>
parents:
365
diff
changeset
|
441 return self.defaultConfig[nodeType].copy() |
107 | 442 |
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
443 def formatLastItems(self, result): |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
444 last_items = [] |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
445 for pep_jid_s, node, data, item_access_model in result: |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
446 pep_jid = jid.JID(pep_jid_s) |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
447 item = generic.stripNamespace(parseXml(data)) |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
448 last_items.append((pep_jid, node, item, item_access_model)) |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
449 return last_items |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
450 |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
451 def getLastItems(self, entities, nodes, node_accesses, item_accesses, pep): |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
452 """get last item for several nodes and entities in a single request""" |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
453 if not entities or not nodes or not node_accesses or not item_accesses: |
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
454 raise BadRequest(text="entities, nodes and accesses must not be empty") |
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
455 if node_accesses != ('open',) or item_accesses != ('open',): |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
456 raise NotImplementedError('only "open" access model is handled for now') |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
457 if not pep: |
414 | 458 raise NotImplementedError("getLastItems is only implemented for PEP at the moment") |
351
2098295747fd
pgsql: cast items.data to text when needed + db schema version bump.
Goffi <goffi@goffi.org>
parents:
349
diff
changeset
|
459 d = self.dbpool.runQuery("""SELECT DISTINCT ON (node_id) pep, node, data::text, items.access_model |
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
460 FROM items |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
461 NATURAL JOIN nodes |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
462 WHERE nodes.pep IN %s |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
463 AND node IN %s |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
464 AND nodes.access_model in %s |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
465 AND items.access_model in %s |
374 | 466 ORDER BY node_id DESC, items.updated DESC""", |
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
467 (tuple([e.userhost() for e in entities]), |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
468 nodes, |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
469 node_accesses, |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
470 item_accesses)) |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
471 d.addCallback(self.formatLastItems) |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
472 return d |
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
473 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
474 |
414 | 475 @implementer(iidavoll.INode) |
107 | 476 class Node: |
477 | |
478 | |
352 | 479 def __init__(self, nodeDbId, nodeIdentifier, config, schema): |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
480 self.nodeDbId = nodeDbId |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
481 self.nodeIdentifier = nodeIdentifier |
107 | 482 self._config = config |
352 | 483 self._schema = schema |
107 | 484 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
485 def _checkNodeExists(self, cursor): |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
486 cursor.execute("""SELECT 1 as exist FROM nodes WHERE node_id=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
487 (self.nodeDbId,)) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
488 if not cursor.fetchone(): |
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
489 raise error.NodeNotFound() |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
490 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
491 def getType(self): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
492 return self.nodeType |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
493 |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
494 def getOwners(self): |
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
495 d = self.dbpool.runQuery("""SELECT jid FROM nodes NATURAL JOIN affiliations NATURAL JOIN entities WHERE node_id=%s and affiliation='owner'""", (self.nodeDbId,)) |
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
496 d.addCallback(lambda rows: [jid.JID(r[0]) for r in rows]) |
243
42048e37699e
added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents:
240
diff
changeset
|
497 return d |
42048e37699e
added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents:
240
diff
changeset
|
498 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
499 def getConfiguration(self): |
107 | 500 return self._config |
501 | |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
502 def getNextId(self): |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
503 """return XMPP item id usable for next item to publish |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
504 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
505 the return value will be next int if serila_ids is set, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
506 else an UUID will be returned |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
507 """ |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
508 if self._config[const.OPT_SERIAL_IDS]: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
509 d = self.dbpool.runQuery("SELECT nextval('{seq_name}')".format( |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
510 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId))) |
414 | 511 d.addCallback(lambda rows: str(rows[0][0])) |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
512 return d |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
513 else: |
414 | 514 return defer.succeed(str(uuid.uuid4())) |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
515 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
516 @staticmethod |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
517 def _configurationTriggers(cursor, node_id, old_config, new_config): |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
518 """trigger database relative actions needed when a config is changed |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
519 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
520 @param cursor(): current db cursor |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
521 @param node_id(unicode): database ID of the node |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
522 @param old_config(dict): config of the node before the change |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
523 @param new_config(dict): new options that will be changed |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
524 """ |
430 | 525 if const.OPT_SERIAL_IDS not in new_config: |
526 return | |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
527 serial_ids = new_config[const.OPT_SERIAL_IDS] |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
528 if serial_ids != old_config[const.OPT_SERIAL_IDS]: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
529 # serial_ids option has been modified, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
530 # we need to handle corresponding sequence |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
531 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
532 # XXX: we use .format in following queries because values |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
533 # are generated by ourself |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
534 seq_name = ITEMS_SEQ_NAME.format(node_id=node_id) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
535 if serial_ids: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
536 # the next query get the max value +1 of all XMPP items ids |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
537 # which are integers, and default to 1 |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
538 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=node_id)) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
539 next_val = cursor.fetchone()[0] |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
540 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name)) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
541 cursor.execute("CREATE SEQUENCE {seq_name} START {next_val} OWNED BY nodes.node_id".format( |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
542 seq_name = seq_name, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
543 next_val = next_val)) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
544 else: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
545 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name)) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
546 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
547 def setConfiguration(self, options): |
430 | 548 to_delete = [] |
549 for option, value in options.items(): | |
550 try: | |
551 if self._config[option] == value: | |
552 to_delete.append(option) | |
553 except KeyError: | |
554 raise BadRequest(text=f"Invalid option: {option!r}") | |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
555 |
430 | 556 for option in to_delete: |
557 options.remove(option) | |
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
558 |
430 | 559 if not options: |
560 return | |
561 | |
562 if options.get(const.OPT_MAX_ITEMS) == "max": | |
424
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
563 # XXX: "max" is default value for config we must convert |
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
564 # it to an interger. See backend's _doSetNodeConfiguration comment |
430 | 565 options[const.OPT_MAX_ITEMS] = "0" |
424
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
566 |
430 | 567 if ((const.OPT_FTS_LANGUAGE in options |
568 and options[const.OPT_FTS_LANGUAGE] not in self.fts_languages)): | |
569 raise BadRequest(text= | |
570 f"invalid {const.OPT_FTS_LANGUAGE} value: " | |
571 f"{options[const.OPT_FTS_LANGUAGE]!r}" | |
572 ) | |
573 | |
574 | |
575 d = self.dbpool.runInteraction(self._setConfiguration, options) | |
576 d.addCallback(self._updateCachedConfiguration, options) | |
124
c4ee16bc48e5
Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents:
121
diff
changeset
|
577 return d |
c4ee16bc48e5
Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents:
121
diff
changeset
|
578 |
430 | 579 def _setConfiguration(self, cursor, options): |
580 self._configurationTriggers(cursor, self.nodeDbId, self._config, options) | |
581 # options names all follow the scheme "pubsub#{col_name}" | |
582 col_names = (o[7:] for o in options) | |
583 values = ','.join(f"{name}=%s" for name in col_names) | |
584 cursor.execute(f"UPDATE nodes SET {values} WHERE node_id=%s", | |
585 (*options.values(), self.nodeDbId)) | |
586 if cursor.rowcount == 0: | |
587 raise error.NodeNotFound() | |
107 | 588 |
430 | 589 def _updateCachedConfiguration(self, __, options): |
590 self._config.update(options) | |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
591 |
352 | 592 def getSchema(self): |
593 return self._schema | |
594 | |
595 def setSchema(self, schema): | |
596 d = self.dbpool.runInteraction(self._setSchema, schema) | |
597 d.addCallback(self._setCachedSchema, schema) | |
598 return d | |
599 | |
600 def _setSchema(self, cursor, schema): | |
601 self._checkNodeExists(cursor) | |
602 cursor.execute("""UPDATE nodes SET schema=%s | |
603 WHERE node_id=%s""", | |
604 (schema.toXml() if schema else None, | |
605 self.nodeDbId)) | |
606 | |
430 | 607 def _setCachedSchema(self, __, schema): |
352 | 608 self._schema = schema |
609 | |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
610 def getMetaData(self): |
107 | 611 config = copy.copy(self._config) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
612 config["pubsub#node_type"] = self.nodeType |
107 | 613 return config |
614 | |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
615 def getAffiliation(self, entity): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
616 return self.dbpool.runInteraction(self._getAffiliation, entity) |
107 | 617 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
618 def _getAffiliation(self, cursor, entity): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
619 self._checkNodeExists(cursor) |
107 | 620 cursor.execute("""SELECT affiliation FROM affiliations |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
621 NATURAL JOIN nodes |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
622 NATURAL JOIN entities |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
623 WHERE node_id=%s AND jid=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
624 (self.nodeDbId, |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
625 entity.userhost())) |
107 | 626 |
627 try: | |
628 return cursor.fetchone()[0] | |
629 except TypeError: | |
630 return None | |
631 | |
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
632 def getAccessModel(self): |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
633 return self._config[const.OPT_ACCESS_MODEL] |
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
634 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
635 def getSubscription(self, subscriber): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
636 return self.dbpool.runInteraction(self._getSubscription, subscriber) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
637 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
638 def _getSubscription(self, cursor, subscriber): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
639 self._checkNodeExists(cursor) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
640 |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
641 userhost = subscriber.userhost() |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
642 resource = subscriber.resource or '' |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
643 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
644 cursor.execute("""SELECT state FROM subscriptions |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
645 NATURAL JOIN nodes |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
646 NATURAL JOIN entities |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
647 WHERE node_id=%s AND jid=%s AND resource=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
648 (self.nodeDbId, |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
649 userhost, |
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
650 resource)) |
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
651 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
652 row = cursor.fetchone() |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
653 if not row: |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
654 return None |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
655 else: |
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
656 return Subscription(self.nodeIdentifier, subscriber, row[0]) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
657 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
658 def getSubscriptions(self, state=None): |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
659 return self.dbpool.runInteraction(self._getSubscriptions, state) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
660 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
661 def _getSubscriptions(self, cursor, state): |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
662 self._checkNodeExists(cursor) |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
663 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
664 query = """SELECT node, jid, resource, state, |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
665 subscription_type, subscription_depth |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
666 FROM subscriptions |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
667 NATURAL JOIN nodes |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
668 NATURAL JOIN entities |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
669 WHERE node_id=%s""" |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
670 values = [self.nodeDbId] |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
671 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
672 if state: |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
673 query += " AND state=%s" |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
674 values.append(state) |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
675 |
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
676 cursor.execute(query, values) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
677 rows = cursor.fetchall() |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
678 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
679 subscriptions = [] |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
680 for row in rows: |
414 | 681 subscriber = jid.JID('%s/%s' % (row.jid, row.resource)) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
682 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
683 options = {} |
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
684 if row.subscription_type: |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
685 options['pubsub#subscription_type'] = row.subscription_type; |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
686 if row.subscription_depth: |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
687 options['pubsub#subscription_depth'] = row.subscription_depth; |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
688 |
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
689 subscriptions.append(Subscription(row.node, subscriber, |
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
690 row.state, options)) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
691 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
692 return subscriptions |
107 | 693 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
694 def addSubscription(self, subscriber, state, config): |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
695 return self.dbpool.runInteraction(self._addSubscription, subscriber, |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
696 state, config) |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
697 |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
698 def _addSubscription(self, cursor, subscriber, state, config): |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
699 self._checkNodeExists(cursor) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
700 |
107 | 701 userhost = subscriber.userhost() |
702 resource = subscriber.resource or '' | |
703 | |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
704 subscription_type = config.get('pubsub#subscription_type') |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
705 subscription_depth = config.get('pubsub#subscription_depth') |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
706 |
107 | 707 try: |
708 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""", | |
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
709 (userhost,)) |
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
710 except cursor._pool.dbapi.IntegrityError: |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
711 cursor.connection.rollback() |
107 | 712 |
713 try: | |
714 cursor.execute("""INSERT INTO subscriptions | |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
715 (node_id, entity_id, resource, state, |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
716 subscription_type, subscription_depth) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
717 SELECT %s, entity_id, %s, %s, %s, %s FROM |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
718 (SELECT entity_id FROM entities |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
719 WHERE jid=%s) AS ent_id""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
720 (self.nodeDbId, |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
721 resource, |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
722 state, |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
723 subscription_type, |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
724 subscription_depth, |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
725 userhost)) |
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
726 except cursor._pool.dbapi.IntegrityError: |
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
727 raise error.SubscriptionExists() |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
728 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
729 def removeSubscription(self, subscriber): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
730 return self.dbpool.runInteraction(self._removeSubscription, |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
731 subscriber) |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
732 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
733 def _removeSubscription(self, cursor, subscriber): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
734 self._checkNodeExists(cursor) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
735 |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
736 userhost = subscriber.userhost() |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
737 resource = subscriber.resource or '' |
107 | 738 |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
739 cursor.execute("""DELETE FROM subscriptions WHERE |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
740 node_id=%s AND |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
741 entity_id=(SELECT entity_id FROM entities |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
742 WHERE jid=%s) AND |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
743 resource=%s""", |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
744 (self.nodeDbId, |
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
745 userhost, |
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
746 resource)) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
747 if cursor.rowcount != 1: |
171
bc269696ef42
Reply with the correct error condition on subscription when not subscribed.
Ralph Meijer <ralphm@ik.nu>
parents:
170
diff
changeset
|
748 raise error.NotSubscribed() |
107 | 749 |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
750 return None |
107 | 751 |
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
752 def setSubscriptions(self, subscriptions): |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
753 return self.dbpool.runInteraction(self._setSubscriptions, subscriptions) |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
754 |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
755 def _setSubscriptions(self, cursor, subscriptions): |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
756 self._checkNodeExists(cursor) |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
757 |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
758 entities = self.getOrCreateEntities(cursor, [s.subscriber for s in subscriptions]) |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
759 entities_map = {jid.JID(e.jid): e for e in entities} |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
760 |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
761 # then we construct values for subscriptions update according to entity_id we just got |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
762 placeholders = ','.join(len(subscriptions) * ["%s"]) |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
763 values = [] |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
764 for subscription in subscriptions: |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
765 entity_id = entities_map[subscription.subscriber].entity_id |
414 | 766 resource = subscription.subscriber.resource or '' |
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
767 values.append((self.nodeDbId, entity_id, resource, subscription.state, None, None)) |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
768 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5 |
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
769 cursor.execute("INSERT INTO subscriptions(node_id, entity_id, resource, state, subscription_type, subscription_depth) VALUES " + placeholders + " ON CONFLICT (entity_id, resource, node_id) DO UPDATE SET state=EXCLUDED.state", [v for v in values]) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
770 |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
771 def isSubscribed(self, entity): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
772 return self.dbpool.runInteraction(self._isSubscribed, entity) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
773 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
774 def _isSubscribed(self, cursor, entity): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
775 self._checkNodeExists(cursor) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
776 |
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
777 cursor.execute("""SELECT 1 as bool FROM entities |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
778 NATURAL JOIN subscriptions |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
779 NATURAL JOIN nodes |
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
780 WHERE entities.jid=%s |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
781 AND node_id=%s AND state='subscribed'""", |
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
782 (entity.userhost(), |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
783 self.nodeDbId)) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
784 |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
785 return cursor.fetchone() is not None |
107 | 786 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
787 def getAffiliations(self): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
788 return self.dbpool.runInteraction(self._getAffiliations) |
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
789 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
790 def _getAffiliations(self, cursor): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
791 self._checkNodeExists(cursor) |
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
792 |
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
793 cursor.execute("""SELECT jid, affiliation FROM nodes |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
794 NATURAL JOIN affiliations |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
795 NATURAL JOIN entities |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
796 WHERE node_id=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
797 (self.nodeDbId,)) |
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
798 result = cursor.fetchall() |
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
799 |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
800 return {jid.internJID(r[0]): r[1] for r in result} |
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
801 |
331 | 802 def getOrCreateEntities(self, cursor, entities_jids): |
803 """Get entity_id from entities in entities table | |
804 | |
805 Entities will be inserted it they don't exist | |
806 @param entities_jid(list[jid.JID]): entities to get or create | |
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
807 @return list[record(entity_id,jid)]]: list of entity_id and jid (as plain string) |
331 | 808 both existing and inserted entities are returned |
809 """ | |
810 # cf. http://stackoverflow.com/a/35265559 | |
811 placeholders = ','.join(len(entities_jids) * ["(%s)"]) | |
812 query = ( | |
813 """ | |
814 WITH | |
815 jid_values (jid) AS ( | |
816 VALUES {placeholders} | |
817 ), | |
818 inserted (entity_id, jid) AS ( | |
819 INSERT INTO entities (jid) | |
820 SELECT jid | |
821 FROM jid_values | |
822 ON CONFLICT DO NOTHING | |
823 RETURNING entity_id, jid | |
824 ) | |
825 SELECT e.entity_id, e.jid | |
826 FROM entities e JOIN jid_values jv ON jv.jid = e.jid | |
827 UNION ALL | |
828 SELECT entity_id, jid | |
829 FROM inserted""".format(placeholders=placeholders)) | |
830 cursor.execute(query, [j.userhost() for j in entities_jids]) | |
831 return cursor.fetchall() | |
832 | |
833 def setAffiliations(self, affiliations): | |
834 return self.dbpool.runInteraction(self._setAffiliations, affiliations) | |
835 | |
836 def _setAffiliations(self, cursor, affiliations): | |
837 self._checkNodeExists(cursor) | |
838 | |
839 entities = self.getOrCreateEntities(cursor, affiliations) | |
840 | |
841 # then we construct values for affiliations update according to entity_id we just got | |
842 placeholders = ','.join(len(affiliations) * ["(%s,%s,%s)"]) | |
843 values = [] | |
414 | 844 for e in entities: |
845 values.extend((e.entity_id, affiliations[jid.JID(e.jid)], self.nodeDbId)) | |
331 | 846 |
847 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5 | |
848 cursor.execute("INSERT INTO affiliations(entity_id,affiliation,node_id) VALUES " + placeholders + " ON CONFLICT (entity_id,node_id) DO UPDATE SET affiliation=EXCLUDED.affiliation", values) | |
849 | |
850 def deleteAffiliations(self, entities): | |
851 return self.dbpool.runInteraction(self._deleteAffiliations, entities) | |
852 | |
853 def _deleteAffiliations(self, cursor, entities): | |
854 """delete affiliations and subscriptions for this entity""" | |
855 self._checkNodeExists(cursor) | |
856 placeholders = ','.join(len(entities) * ["%s"]) | |
857 cursor.execute("DELETE FROM affiliations WHERE node_id=%s AND entity_id in (SELECT entity_id FROM entities WHERE jid IN (" + placeholders + ")) RETURNING entity_id", [self.nodeDbId] + [e.userhost() for e in entities]) | |
858 | |
859 rows = cursor.fetchall() | |
860 placeholders = ','.join(len(rows) * ["%s"]) | |
861 cursor.execute("DELETE FROM subscriptions WHERE node_id=%s AND entity_id in (" + placeholders + ")", [self.nodeDbId] + [r[0] for r in rows]) | |
862 | |
863 def getAuthorizedGroups(self): | |
864 return self.dbpool.runInteraction(self._getNodeGroups) | |
865 | |
866 def _getAuthorizedGroups(self, cursor): | |
867 cursor.execute("SELECT groupname FROM node_groups_authorized NATURAL JOIN nodes WHERE node=%s", | |
868 (self.nodeDbId,)) | |
869 rows = cursor.fetchall() | |
870 return [row[0] for row in rows] | |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
871 |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
872 |
414 | 873 @implementer(iidavoll.ILeafNode) |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
874 class LeafNode(Node): |
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
875 |
107 | 876 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
877 nodeType = 'leaf' |
107 | 878 |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
879 def getOrderBy(self, ext_data, direction='DESC'): |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
880 """Return ORDER BY clause corresponding to Order By key in ext_data |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
881 |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
882 @param ext_data (dict): extra data as used in getItems |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
883 @param direction (unicode): ORDER BY direction (ASC or DESC) |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
884 @return (unicode): ORDER BY clause to use |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
885 """ |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
886 keys = ext_data.get('order_by') |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
887 if not keys: |
414 | 888 return 'ORDER BY updated ' + direction |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
889 cols_statmnt = [] |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
890 for key in keys: |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
891 if key == 'creation': |
378
22832c1d2827
storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents:
375
diff
changeset
|
892 column = 'item_id' # could work with items.created too |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
893 elif key == 'modification': |
378
22832c1d2827
storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents:
375
diff
changeset
|
894 column = 'updated' |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
895 else: |
414 | 896 log.msg("WARNING: Unknown order by key: {key}".format(key=key)) |
378
22832c1d2827
storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents:
375
diff
changeset
|
897 column = 'updated' |
414 | 898 cols_statmnt.append(column + ' ' + direction) |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
899 |
414 | 900 return "ORDER BY " + ",".join([col for col in cols_statmnt]) |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
901 |
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
902 @defer.inlineCallbacks |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
903 def storeItems(self, items_data, publisher): |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
904 # XXX: runInteraction doesn't seem to work when there are several "insert" |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
905 # or "update". |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
906 # Before the unpacking was done in _storeItems, but this was causing trouble |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
907 # in case of multiple items_data. So this has now be moved here. |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
908 # FIXME: investigate the issue with runInteraction |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
909 for item_data in items_data: |
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
910 yield self.dbpool.runInteraction(self._storeItems, item_data, publisher) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
911 |
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
912 def _storeItems(self, cursor, item_data, publisher): |
250 | 913 self._checkNodeExists(cursor) |
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
914 self._storeItem(cursor, item_data, publisher) |
244 | 915 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
916 def _storeItem(self, cursor, item_data, publisher): |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
917 # first try to insert the item |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
918 # - if it fails (conflict), and the item is new and we have serial_ids options, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
919 # current id will be recomputed using next item id query (note that is not perfect, as |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
920 # table is not locked and this can fail if two items are added at the same time |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
921 # but this can only happen with serial_ids and if future ids have been set by a client, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
922 # this case should be rare enough to consider this situation acceptable) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
923 # - if item insertion fail and the item is not new, we do an update |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
924 # - in other cases, exception is raised |
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
925 item, access_model, item_config = item_data.item, item_data.access_model, item_data.config |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
926 data = item.toXml() |
430 | 927 data_fts_cfg = self._config[const.OPT_FTS_LANGUAGE] |
928 if data_fts_cfg == const.VAL_FTS_GENERIC: | |
929 data_fts_cfg = "simple" | |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
930 |
430 | 931 insert_query = ( |
932 "INSERT INTO items(node_id, item, publisher, data, access_model, " | |
933 "data_fts_cfg) VALUES (%s, %s, %s, %s, %s, %s) RETURNING item_id" | |
934 ) | |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
935 insert_data = [self.nodeDbId, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
936 item["id"], |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
937 publisher.full(), |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
938 data, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
939 access_model, |
430 | 940 data_fts_cfg, |
941 ] | |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
942 |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
943 try: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
944 cursor.execute(insert_query, insert_data) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
945 except cursor._pool.dbapi.IntegrityError as e: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
946 if e.pgcode != "23505": |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
947 # we only handle unique_violation, every other exception must be raised |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
948 raise e |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
949 cursor.connection.rollback() |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
950 # the item already exist |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
951 if item_data.new: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
952 # the item is new |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
953 if self._config[const.OPT_SERIAL_IDS]: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
954 # this can happen with serial_ids, if a item has been stored |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
955 # with a future id (generated by XMPP client) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
956 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=self.nodeDbId)) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
957 next_id = cursor.fetchone()[0] |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
958 # we update the sequence, so we can skip conflicting ids |
414 | 959 cursor.execute("SELECT setval('{seq_name}', %s)".format( |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
960 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId)), [next_id]) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
961 # and now we can retry the query with the new id |
414 | 962 item['id'] = insert_data[1] = str(next_id) |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
963 # item saved in DB must also be updated with the new id |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
964 insert_data[3] = item.toXml() |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
965 cursor.execute(insert_query, insert_data) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
966 else: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
967 # but if we have not serial_ids, we have a real problem |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
968 raise e |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
969 else: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
970 # this is an update |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
971 cursor.execute("""UPDATE items SET updated=now(), publisher=%s, data=%s |
430 | 972 WHERE node_id=%s AND items.item=%s |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
973 RETURNING item_id""", |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
974 (publisher.full(), |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
975 data, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
976 self.nodeDbId, |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
977 item["id"])) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
978 if cursor.rowcount != 1: |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
979 raise exceptions.InternalError("item has not been updated correctly") |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
980 item_id = cursor.fetchone()[0]; |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
981 self._storeCategories(cursor, item_id, item_data.categories, update=True) |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
982 return |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
983 |
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
984 item_id = cursor.fetchone()[0]; |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
985 self._storeCategories(cursor, item_id, item_data.categories) |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
986 |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
987 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
250 | 988 if const.OPT_ROSTER_GROUPS_ALLOWED in item_config: |
422 | 989 # XXX: needed to force list if there is only one value |
990 item_config.fields[const.OPT_ROSTER_GROUPS_ALLOWED].fieldType='list-multi' | |
250 | 991 allowed_groups = item_config[const.OPT_ROSTER_GROUPS_ALLOWED] |
245
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
992 else: |
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
993 allowed_groups = [] |
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
994 for group in allowed_groups: |
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
995 #TODO: check that group are actually in roster |
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
996 cursor.execute("""INSERT INTO item_groups_authorized (item_id, groupname) |
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
997 VALUES (%s,%s)""" , (item_id, group)) |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
998 # TODO: whitelist access model |
245
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
999 |
422 | 1000 max_items = self._config.get(const.OPT_MAX_ITEMS, 'max') |
1001 | |
1002 if max_items != 'max': | |
1003 try: | |
1004 max_items = int(self._config[const.OPT_MAX_ITEMS]) | |
1005 except ValueError: | |
1006 log.err(f"Invalid max_items value: {max_items!r}") | |
1007 else: | |
1008 if max_items > 0: | |
1009 # we delete all items above the requested max | |
1010 cursor.execute( | |
1011 "DELETE FROM items WHERE node_id=%s and item_id in (SELECT " | |
1012 "item_id FROM items WHERE node_id=%s ORDER BY items.updated " | |
1013 "DESC, items.item_id DESC OFFSET %s)" , | |
1014 (self.nodeDbId, self.nodeDbId, max_items) | |
1015 ) | |
1016 | |
1017 | |
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1018 def _storeCategories(self, cursor, item_id, categories, update=False): |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1019 # TODO: handle canonical form |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1020 if update: |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1021 cursor.execute("""DELETE FROM item_categories |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1022 WHERE item_id=%s""", (item_id,)) |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1023 |
402
724e39d596a9
psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents:
393
diff
changeset
|
1024 # we use a set to avoid duplicates |
724e39d596a9
psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents:
393
diff
changeset
|
1025 for category in set(categories): |
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1026 cursor.execute("""INSERT INTO item_categories (item_id, category) |
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1027 VALUES (%s, %s)""", (item_id, category)) |
107 | 1028 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1029 def removeItems(self, itemIdentifiers): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
1030 return self.dbpool.runInteraction(self._removeItems, itemIdentifiers) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1031 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1032 def _removeItems(self, cursor, itemIdentifiers): |
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1033 self._checkNodeExists(cursor) |
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
1034 |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1035 deleted = [] |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1036 |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1037 for itemIdentifier in itemIdentifiers: |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1038 cursor.execute("""DELETE FROM items WHERE |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1039 node_id=%s AND |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1040 item=%s""", |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1041 (self.nodeDbId, |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1042 itemIdentifier)) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1043 |
142
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1044 if cursor.rowcount: |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1045 deleted.append(itemIdentifier) |
142
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1046 |
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1047 return deleted |
107 | 1048 |
278 | 1049 def getItems(self, authorized_groups, unrestricted, maxItems=None, ext_data=None): |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1050 """ Get all authorised items |
346 | 1051 |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1052 @param authorized_groups: we want to get items that these groups can access |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1053 @param unrestricted: if true, don't check permissions (i.e.: get all items) |
318 | 1054 @param maxItems: nb of items we want to get |
1055 @param ext_data: options for extra features like RSM and MAM | |
278 | 1056 |
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1057 @return: list of container.ItemData |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1058 if unrestricted is False, access_model and config will be None |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1059 """ |
278 | 1060 if ext_data is None: |
1061 ext_data = {} | |
347 | 1062 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=False) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1063 |
346 | 1064 def getItemsIds(self, authorized_groups, unrestricted, maxItems=None, ext_data=None): |
1065 """ Get all authorised items ids | |
1066 | |
1067 @param authorized_groups: we want to get items that these groups can access | |
1068 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
1069 @param maxItems: nb of items we want to get | |
1070 @param ext_data: options for extra features like RSM and MAM | |
1071 | |
1072 @return list(unicode): list of ids | |
1073 """ | |
1074 if ext_data is None: | |
1075 ext_data = {} | |
1076 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=True) | |
1077 | |
318 | 1078 def _appendSourcesAndFilters(self, query, args, authorized_groups, unrestricted, ext_data): |
1079 """append sources and filters to sql query requesting items and return ORDER BY | |
278 | 1080 |
318 | 1081 arguments query, args, authorized_groups, unrestricted and ext_data are the same as for |
1082 _getItems | |
1083 """ | |
1084 # SOURCES | |
323 | 1085 query.append("FROM nodes INNER JOIN items USING (node_id)") |
1086 | |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1087 if unrestricted: |
323 | 1088 query_filters = ["WHERE node_id=%s"] |
318 | 1089 args.append(self.nodeDbId) |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1090 else: |
323 | 1091 query.append("LEFT JOIN item_groups_authorized USING (item_id)") |
318 | 1092 args.append(self.nodeDbId) |
1093 if authorized_groups: | |
1094 get_groups = " or (items.access_model='roster' and groupname in %s)" | |
1095 args.append(authorized_groups) | |
1096 else: | |
1097 get_groups = "" | |
278 | 1098 |
323 | 1099 query_filters = ["WHERE node_id=%s AND (items.access_model='open'" + get_groups + ")"] |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1100 |
318 | 1101 # FILTERS |
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1102 if 'filters' in ext_data: # MAM filters |
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1103 for filter_ in ext_data['filters']: |
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1104 if filter_.var == 'start': |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1105 query_filters.append("AND created>=%s") |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1106 args.append(filter_.value) |
323 | 1107 elif filter_.var == 'end': |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1108 query_filters.append("AND created<=%s") |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1109 args.append(filter_.value) |
323 | 1110 elif filter_.var == 'with': |
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1111 jid_s = filter_.value |
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1112 if '/' in jid_s: |
323 | 1113 query_filters.append("AND publisher=%s") |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1114 args.append(filter_.value) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1115 else: |
323 | 1116 query_filters.append("AND publisher LIKE %s") |
414 | 1117 args.append("{}%".format(filter_.value)) |
430 | 1118 elif filter_.var == const.MAM_FILTER_FTS: |
1119 fts_cfg = self._config[const.OPT_FTS_LANGUAGE] | |
1120 if fts_cfg == const.VAL_FTS_GENERIC: | |
1121 fts_cfg = "simple" | |
1122 query_filters.append( | |
1123 "AND data_fts @@ websearch_to_tsquery(%s, %s)" | |
1124 ) | |
1125 args.append(fts_cfg) | |
1126 args.append(filter_.value) | |
323 | 1127 elif filter_.var == const.MAM_FILTER_CATEGORY: |
1128 query.append("LEFT JOIN item_categories USING (item_id)") | |
1129 query_filters.append("AND category=%s") | |
1130 args.append(filter_.value) | |
1131 else: | |
1132 log.msg("WARNING: unknown filter: {}".format(filter_.encode('utf-8'))) | |
1133 | |
1134 query.extend(query_filters) | |
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1135 |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
1136 return self.getOrderBy(ext_data) |
318 | 1137 |
346 | 1138 def _getItems(self, cursor, authorized_groups, unrestricted, maxItems, ext_data, ids_only): |
318 | 1139 self._checkNodeExists(cursor) |
1140 | |
1141 if maxItems == 0: | |
1142 return [] | |
1143 | |
1144 args = [] | |
1145 | |
1146 # SELECT | |
346 | 1147 if ids_only: |
1148 query = ["SELECT item"] | |
1149 else: | |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1150 query = ["SELECT data::text,items.access_model,item_id,created,updated"] |
318 | 1151 |
1152 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
278 | 1153 |
1154 if 'rsm' in ext_data: | |
1155 rsm = ext_data['rsm'] | |
1156 maxItems = rsm.max | |
1157 if rsm.index is not None: | |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1158 # We need to know the item_id of corresponding to the index (offset) of the current query |
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1159 # so we execute the query to look for the item_id |
318 | 1160 tmp_query = query[:] |
1161 tmp_args = args[:] | |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1162 tmp_query[0] = "SELECT item_id" |
318 | 1163 tmp_query.append("{} LIMIT 1 OFFSET %s".format(query_order)) |
1164 tmp_args.append(rsm.index) | |
1165 cursor.execute(' '.join(query), args) | |
1166 # FIXME: bad index is not managed yet | |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1167 item_id = cursor.fetchall()[0][0] |
318 | 1168 |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1169 # now that we have the id, we can use it |
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1170 query.append("AND item_id<=%s") |
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1171 args.append(item_id) |
278 | 1172 elif rsm.before is not None: |
1173 if rsm.before != '': | |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1174 query.append("AND item_id>(SELECT item_id FROM items WHERE item=%s LIMIT 1)") |
278 | 1175 args.append(rsm.before) |
318 | 1176 if maxItems is not None: |
1177 # if we have maxItems (i.e. a limit), we need to reverse order | |
1178 # in a first query to get the right items | |
1179 query.insert(0,"SELECT * from (") | |
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
1180 query.append(self.getOrderBy(ext_data, direction='ASC')) |
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
1181 query.append("LIMIT %s) as x") |
318 | 1182 args.append(maxItems) |
278 | 1183 elif rsm.after: |
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1184 query.append("AND item_id<(SELECT item_id FROM items WHERE item=%s LIMIT 1)") |
278 | 1185 args.append(rsm.after) |
1186 | |
318 | 1187 query.append(query_order) |
278 | 1188 |
297
4115999d85e9
fixes behavior when maxItems is 0
souliane <souliane@mailoo.org>
parents:
294
diff
changeset
|
1189 if maxItems is not None: |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1190 query.append("LIMIT %s") |
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1191 args.append(maxItems) |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1192 |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1193 cursor.execute(' '.join(query), args) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1194 |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1195 result = cursor.fetchall() |
346 | 1196 if unrestricted and not ids_only: |
318 | 1197 # with unrestricted query, we need to fill the access_list for a roster access items |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1198 ret = [] |
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1199 for item_data in result: |
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1200 item = generic.stripNamespace(parseXml(item_data.data)) |
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1201 access_model = item_data.access_model |
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1202 item_id = item_data.item_id |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1203 created = item_data.created |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1204 updated = item_data.updated |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1205 access_list = {} |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1206 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1207 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,)) |
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1208 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r.groupname for r in cursor.fetchall()] |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1209 |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1210 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated)) |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1211 # TODO: whitelist item access model |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1212 return ret |
318 | 1213 |
346 | 1214 if ids_only: |
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1215 return [r.item for r in result] |
346 | 1216 else: |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1217 items_data = [container.ItemData(generic.stripNamespace(parseXml(r.data)), r.access_model, created=r.created, updated=r.updated) for r in result] |
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1218 return items_data |
107 | 1219 |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1220 def getItemsById(self, authorized_groups, unrestricted, itemIdentifiers): |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1221 """Get items which are in the given list |
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1222 |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1223 @param authorized_groups: we want to get items that these groups can access |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1224 @param unrestricted: if true, don't check permissions |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1225 @param itemIdentifiers: list of ids of the items we want to get |
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1226 @return: list of container.ItemData |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1227 ItemData.config will contains access_list (managed as a dictionnary with same key as for item_config) |
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1228 if unrestricted is False, access_model and config will be None |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1229 """ |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1230 return self.dbpool.runInteraction(self._getItemsById, authorized_groups, unrestricted, itemIdentifiers) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1231 |
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1232 def _getItemsById(self, cursor, authorized_groups, unrestricted, itemIdentifiers): |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1233 self._checkNodeExists(cursor) |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1234 ret = [] |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1235 if unrestricted: #we get everything without checking permissions |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1236 for itemIdentifier in itemIdentifiers: |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1237 cursor.execute("""SELECT data::text,items.access_model,item_id,created,updated FROM nodes |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1238 INNER JOIN items USING (node_id) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1239 WHERE node_id=%s AND item=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1240 (self.nodeDbId, |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1241 itemIdentifier)) |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1242 result = cursor.fetchone() |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1243 if not result: |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1244 raise error.ItemNotFound() |
107 | 1245 |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1246 item = generic.stripNamespace(parseXml(result[0])) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1247 access_model = result[1] |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1248 item_id = result[2] |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1249 created= result[3] |
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1250 updated= result[4] |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1251 access_list = {} |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1252 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1253 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,)) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1254 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r[0] for r in cursor.fetchall()] |
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1255 #TODO: WHITELIST access_model |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1256 |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1257 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated)) |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1258 else: #we check permission before returning items |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1259 for itemIdentifier in itemIdentifiers: |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1260 args = [self.nodeDbId, itemIdentifier] |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1261 if authorized_groups: |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1262 args.append(authorized_groups) |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1263 cursor.execute("""SELECT data::text, created, updated FROM nodes |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1264 INNER JOIN items USING (node_id) |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1265 LEFT JOIN item_groups_authorized USING (item_id) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1266 WHERE node_id=%s AND item=%s AND |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1267 (items.access_model='open' """ + |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1268 ("or (items.access_model='roster' and groupname in %s)" if authorized_groups else '') + ")", |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1269 args) |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1270 |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1271 result = cursor.fetchone() |
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1272 if result: |
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1273 ret.append(container.ItemData(generic.stripNamespace(parseXml(result[0])), created=result[1], updated=result[2])) |
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1274 |
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1275 return ret |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1276 |
318 | 1277 def getItemsCount(self, authorized_groups, unrestricted, ext_data=None): |
1278 """Count expected number of items in a getItems query | |
1279 | |
1280 @param authorized_groups: we want to get items that these groups can access | |
1281 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
1282 @param ext_data: options for extra features like RSM and MAM | |
1283 """ | |
1284 if ext_data is None: | |
1285 ext_data = {} | |
1286 return self.dbpool.runInteraction(self._getItemsCount, authorized_groups, unrestricted, ext_data) | |
1287 | |
1288 def _getItemsCount(self, cursor, authorized_groups, unrestricted, ext_data): | |
1289 self._checkNodeExists(cursor) | |
1290 args = [] | |
1291 | |
1292 # SELECT | |
1293 query = ["SELECT count(1)"] | |
1294 | |
1295 self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
1296 | |
1297 cursor.execute(' '.join(query), args) | |
1298 return cursor.fetchall()[0][0] | |
1299 | |
1300 def getItemsIndex(self, item_id, authorized_groups, unrestricted, ext_data=None): | |
1301 """Get expected index of first item in the window of a getItems query | |
1302 | |
1303 @param item_id: id of the item | |
1304 @param authorized_groups: we want to get items that these groups can access | |
1305 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
1306 @param ext_data: options for extra features like RSM and MAM | |
1307 """ | |
1308 if ext_data is None: | |
1309 ext_data = {} | |
1310 return self.dbpool.runInteraction(self._getItemsIndex, item_id, authorized_groups, unrestricted, ext_data) | |
1311 | |
1312 def _getItemsIndex(self, cursor, item_id, authorized_groups, unrestricted, ext_data): | |
1313 self._checkNodeExists(cursor) | |
1314 args = [] | |
1315 | |
1316 # SELECT | |
1317 query = [] | |
1318 | |
1319 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
1320 | |
1321 query_select = "SELECT row_number from (SELECT row_number() OVER ({}), item".format(query_order) | |
1322 query.insert(0, query_select) | |
1323 query.append(") as x WHERE item=%s") | |
1324 args.append(item_id) | |
1325 | |
1326 cursor.execute(' '.join(query), args) | |
1327 # XXX: row_number start at 1, but we want that index start at 0 | |
324
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1328 try: |
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1329 return cursor.fetchall()[0][0] - 1 |
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1330 except IndexError: |
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1331 raise error.NodeNotFound() |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1332 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1333 def getItemsPublishers(self, itemIdentifiers): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1334 """Get the publishers for all given identifiers |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1335 |
308
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1336 @return (dict[unicode, jid.JID]): map of itemIdentifiers to publisher |
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1337 if item is not found, key is skipped in resulting dict |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1338 """ |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1339 return self.dbpool.runInteraction(self._getItemsPublishers, itemIdentifiers) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1340 |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1341 def _getItemsPublishers(self, cursor, itemIdentifiers): |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1342 self._checkNodeExists(cursor) |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1343 ret = {} |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1344 for itemIdentifier in itemIdentifiers: |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1345 cursor.execute("""SELECT publisher FROM items |
390
fba96e95e329
storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents:
384
diff
changeset
|
1346 WHERE node_id=%s AND item=%s""", |
fba96e95e329
storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents:
384
diff
changeset
|
1347 (self.nodeDbId, itemIdentifier,)) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1348 result = cursor.fetchone() |
308
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1349 if result: |
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1350 ret[itemIdentifier] = jid.JID(result[0]) |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1351 return ret |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1352 |
107 | 1353 def purge(self): |
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
1354 return self.dbpool.runInteraction(self._purge) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1355 |
127
d3689da18ed2
Don't use encode('utf-8') on serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents:
124
diff
changeset
|
1356 def _purge(self, cursor): |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1357 self._checkNodeExists(cursor) |
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1358 |
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1359 cursor.execute("""DELETE FROM items WHERE |
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1360 node_id=%s""", |
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1361 (self.nodeDbId,)) |
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1362 |
263
9dfd3890e646
added the constant FLAG_RETRACT_ALLOW_PUBLISHER to allow a publisher to retract an item he has published in a node of "open" publish model.
souliane <souliane@mailoo.org>
parents:
262
diff
changeset
|
1363 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
1364 class CollectionNode(Node): |
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
1365 |
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
1366 nodeType = 'collection' |