Mercurial > libervia-pubsub
annotate sat_pubsub/pgsql_storage.py @ 466:0d38c3529972
psql: schema update (9) to add `roster` table
| author | Goffi <goffi@goffi.org> |
|---|---|
| date | Fri, 15 Oct 2021 13:40:59 +0200 |
| parents | f520ac3164b0 |
| children | d86e0f8a1405 |
| rev | line source |
|---|---|
| 414 | 1 #!/usr/bin/env python3 |
| 233 | 2 |
| 440 | 3 # Copyright (c) 2012-2021 Jérôme Poisson |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
4 # Copyright (c) 2013-2016 Adrien Cossa |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
5 # Copyright (c) 2003-2011 Ralph Meijer |
| 233 | 6 |
| 7 | |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
8 # This program is free software: you can redistribute it and/or modify |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
9 # it under the terms of the GNU Affero General Public License as published by |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
10 # the Free Software Foundation, either version 3 of the License, or |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
11 # (at your option) any later version. |
| 233 | 12 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
13 # This program is distributed in the hope that it will be useful, |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
16 # GNU Affero General Public License for more details. |
| 233 | 17 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
18 # You should have received a copy of the GNU Affero General Public License |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
19 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
20 # -- |
| 233 | 21 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
22 # This program is based on Idavoll (http://idavoll.ik.nu/), |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
23 # originaly written by Ralph Meijer (http://ralphm.net/blog/) |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
24 # It is sublicensed under AGPL v3 (or any later version) as allowed by the original |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
25 # license. |
| 233 | 26 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
27 # -- |
| 233 | 28 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
29 # Here is a copy of the original license: |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
30 |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
31 # Copyright (c) 2003-2011 Ralph Meijer |
| 233 | 32 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
33 # Permission is hereby granted, free of charge, to any person obtaining |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
34 # a copy of this software and associated documentation files (the |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
35 # "Software"), to deal in the Software without restriction, including |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
36 # without limitation the rights to use, copy, modify, merge, publish, |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
37 # distribute, sublicense, and/or sell copies of the Software, and to |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
38 # permit persons to whom the Software is furnished to do so, subject to |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
39 # the following conditions: |
| 233 | 40 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
41 # The above copyright notice and this permission notice shall be |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
42 # included in all copies or substantial portions of the Software. |
| 233 | 43 |
|
312
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
44 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
45 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
46 # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
47 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
48 # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
49 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION |
|
5d7c3787672e
fixed copyright put in docstring instead of comments
Goffi <goffi@goffi.org>
parents:
311
diff
changeset
|
50 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| 233 | 51 |
|
155
5191ba7c4df8
Work towards first release 0.5.0.
Ralph Meijer <ralphm@ik.nu>
parents:
148
diff
changeset
|
52 |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
53 import copy, logging |
|
202
77c61e2b8c75
Use `domish.Element`s to represent items, instead of serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents:
198
diff
changeset
|
54 |
| 414 | 55 from zope.interface import implementer |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
56 |
|
344
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
57 from twisted.internet import reactor |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
58 from twisted.internet import defer |
| 107 | 59 from twisted.words.protocols.jabber import jid |
| 323 | 60 from twisted.python import log |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
61 |
|
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
62 from wokkel import generic |
|
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
63 from wokkel.pubsub import Subscription, BadRequest |
| 107 | 64 |
|
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
65 from sat_pubsub import error |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
66 from sat_pubsub import iidavoll |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
67 from sat_pubsub import const |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
68 from sat_pubsub import container |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
69 from sat_pubsub import exceptions |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
70 import uuid |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
71 import psycopg2 |
|
270
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
72 import psycopg2.extensions |
|
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
73 # we wants psycopg2 to return us unicode, not str |
|
270
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
74 psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) |
|
f17034e4cf4a
fixed unicode handling with psycopg2
Goffi <goffi@goffi.org>
parents:
265
diff
changeset
|
75 psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY) |
| 244 | 76 |
|
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
77 # parseXml manage str, but we get unicode |
|
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
78 parseXml = lambda unicode_data: generic.parseXml(unicode_data.encode('utf-8')) |
| 414 | 79 ITEMS_SEQ_NAME = 'node_{node_id}_seq' |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
80 PEP_COL_NAME = 'pep' |
|
466
0d38c3529972
psql: schema update (9) to add `roster` table
Goffi <goffi@goffi.org>
parents:
463
diff
changeset
|
81 CURRENT_VERSION = '9' |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
82 # retrieve the maximum integer item id + 1 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
83 NEXT_ITEM_ID_QUERY = r"SELECT COALESCE(max(item::integer)+1,1) as val from items where node_id={node_id} and item ~ E'^\\d+$'" |
|
271
232002e132db
pgsql_backend unicode fix: parseXml was expecting str and getting unicode
Goffi <goffi@goffi.org>
parents:
270
diff
changeset
|
84 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
85 |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
86 def withPEP(query, values, pep, recipient): |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
87 """Helper method to facilitate PEP management |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
88 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
89 @param query: SQL query basis |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
90 @param values: current values to replace in query |
|
364
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
91 @param pep(bool): True if we are in PEP mode |
|
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
92 @param recipient(jid.JID): jid of the recipient |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
93 @return: query + PEP AND check, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
94 recipient's bare jid is added to value if needed |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
95 """ |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
96 if pep: |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
97 pep_check="AND {}=%s".format(PEP_COL_NAME) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
98 values=list(values) + [recipient.userhost()] |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
99 else: |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
100 pep_check="AND {} IS NULL".format(PEP_COL_NAME) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
101 return "{} {}".format(query, pep_check), values |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
102 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
103 |
| 414 | 104 @implementer(iidavoll.IStorage) |
| 107 | 105 class Storage: |
| 106 | |
| 430 | 107 fts_languages = ['generic'] |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
108 defaultConfig = { |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
109 'leaf': { |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
110 const.OPT_PERSIST_ITEMS: True, |
| 422 | 111 const.OPT_MAX_ITEMS: 'max', |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
112 const.OPT_DELIVER_PAYLOADS: True, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
113 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub', |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
114 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT, |
| 260 | 115 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT, |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
116 const.OPT_OVERWRITE_POLICY: const.VAL_OWPOL_DEFAUT, |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
117 const.OPT_SERIAL_IDS: False, |
|
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
118 const.OPT_CONSISTENT_PUBLISHER: False, |
| 430 | 119 const.OPT_FTS_LANGUAGE: const.VAL_FTS_GENERIC, |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
120 }, |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
121 'collection': { |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
122 const.OPT_DELIVER_PAYLOADS: True, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
123 const.OPT_SEND_LAST_PUBLISHED_ITEM: 'on_sub', |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
124 const.OPT_ACCESS_MODEL: const.VAL_AMODEL_DEFAULT, |
| 260 | 125 const.OPT_PUBLISH_MODEL: const.VAL_PMODEL_DEFAULT, |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
126 } |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
127 } |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
128 |
|
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
129 def __init__(self, dbpool): |
|
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
130 self.dbpool = dbpool |
|
344
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
131 d = self.dbpool.runQuery("SELECT value FROM metadata WHERE key='version'") |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
132 d.addCallbacks(self._checkVersion, self._versionEb) |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
133 |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
134 def _checkVersion(self, row): |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
135 version = row[0].value |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
136 if version != CURRENT_VERSION: |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
137 logging.error("Bad database schema version ({current}), please upgrade to {needed}".format( |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
138 current=version, needed=CURRENT_VERSION)) |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
139 reactor.stop() |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
140 |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
141 def _versionEb(self, failure): |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
142 logging.error("Can't check schema version: {reason}".format(reason=failure)) |
|
8cf1be9572f8
pgsql: check schema version and exit with a message asking to upgrade if it's not the current one
Goffi <goffi@goffi.org>
parents:
341
diff
changeset
|
143 reactor.stop() |
| 107 | 144 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
145 def _buildNode(self, row): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
146 """Build a note class from database result row""" |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
147 if not row: |
|
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
148 raise error.NodeNotFound() |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
149 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
150 if row[2] == 'leaf': |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
151 configuration = { |
| 422 | 152 const.OPT_PERSIST_ITEMS: row[3], |
| 153 const.OPT_MAX_ITEMS: 'max' if row[4] == 0 else str(row[4]), | |
| 154 const.OPT_DELIVER_PAYLOADS: row[5], | |
| 155 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6], | |
| 156 const.OPT_ACCESS_MODEL:row[7], | |
| 157 const.OPT_PUBLISH_MODEL:row[8], | |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
158 const.OPT_OVERWRITE_POLICY: row[9], |
|
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
159 const.OPT_SERIAL_IDS:row[10], |
|
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
160 const.OPT_CONSISTENT_PUBLISHER:row[11], |
|
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
161 const.OPT_FTS_LANGUAGE: row[12], |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
162 } |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
163 schema = row[13] |
| 352 | 164 if schema is not None: |
| 165 schema = parseXml(schema) | |
| 166 node = LeafNode(row[0], row[1], configuration, schema) | |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
167 node.dbpool = self.dbpool |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
168 return node |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
169 elif row[2] == 'collection': |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
170 configuration = { |
| 422 | 171 const.OPT_DELIVER_PAYLOADS: row[5], |
| 172 const.OPT_SEND_LAST_PUBLISHED_ITEM: row[6], | |
| 173 const.OPT_ACCESS_MODEL: row[7], | |
| 174 const.OPT_PUBLISH_MODEL:row[8], | |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
175 } |
| 352 | 176 node = CollectionNode(row[0], row[1], configuration, None) |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
177 node.dbpool = self.dbpool |
| 107 | 178 return node |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
179 else: |
|
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
180 raise BadRequest(text="Unknown node type !") |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
181 |
| 430 | 182 def getFTSLanguages(self): |
| 183 """Get list of available languages for full text search""" | |
| 184 return self.dbpool.runInteraction(self._getFTSLanguages) | |
| 185 | |
| 186 def _getFTSLanguages(self, cursor): | |
| 187 cursor.execute("SELECT cfgname FROM pg_ts_config") | |
| 188 result = [r.cfgname for r in cursor.fetchall()] | |
| 189 result.remove("simple") | |
| 190 result.insert(0, "generic") | |
| 191 Node.fts_languages = self.fts_languages = result | |
| 192 return result | |
| 193 | |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
194 def getNodeById(self, nodeDbId): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
195 """Get node using database ID insted of pubsub identifier |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
196 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
197 @param nodeDbId(unicode): database ID |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
198 """ |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
199 return self.dbpool.runInteraction(self._getNodeById, nodeDbId) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
200 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
201 def _getNodeById(self, cursor, nodeDbId): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
202 cursor.execute("""SELECT node_id, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
203 node, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
204 node_type, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
205 persist_items, |
| 422 | 206 max_items, |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
207 deliver_payloads, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
208 send_last_published_item, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
209 access_model, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
210 publish_model, |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
211 overwrite_policy, |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
212 serial_ids, |
|
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
213 consistent_publisher, |
| 430 | 214 fts_language, |
| 352 | 215 schema::text, |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
216 pep |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
217 FROM nodes |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
218 WHERE node_id=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
219 (nodeDbId,)) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
220 row = cursor.fetchone() |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
221 return self._buildNode(row) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
222 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
223 def getNode(self, nodeIdentifier, pep, recipient=None): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
224 return self.dbpool.runInteraction(self._getNode, nodeIdentifier, pep, recipient) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
225 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
226 def _getNode(self, cursor, nodeIdentifier, pep, recipient): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
227 cursor.execute(*withPEP("""SELECT node_id, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
228 node, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
229 node_type, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
230 persist_items, |
| 422 | 231 max_items, |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
232 deliver_payloads, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
233 send_last_published_item, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
234 access_model, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
235 publish_model, |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
236 overwrite_policy, |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
237 serial_ids, |
|
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
238 consistent_publisher, |
| 430 | 239 fts_language, |
| 352 | 240 schema::text, |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
241 pep |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
242 FROM nodes |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
243 WHERE node=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
244 (nodeIdentifier,), pep, recipient)) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
245 row = cursor.fetchone() |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
246 return self._buildNode(row) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
247 |
|
349
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
248 def getNodeIds(self, pep, recipient, allowed_accesses=None): |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
249 """retrieve ids of existing nodes |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
250 |
|
364
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
251 @param pep(bool): True if it's a PEP request |
|
8bd8be6815ab
completed docstrings + use short notation for sets
Arnaud Joset <info@agayon.be>
parents:
357
diff
changeset
|
252 @param recipient(jid.JID, None): recipient of the PEP request |
|
349
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
253 @param allowed_accesses(None, set): only nodes with access |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
254 in this set will be returned |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
255 None to return all nodes |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
256 @return (list[unicode]): ids of nodes |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
257 """ |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
258 if not pep: |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
259 query = "SELECT node from nodes WHERE pep is NULL" |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
260 values = [] |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
261 else: |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
262 query = "SELECT node from nodes WHERE pep=%s" |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
263 values = [recipient.userhost()] |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
264 |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
265 if allowed_accesses is not None: |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
266 query += "AND access_model IN %s" |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
267 values.append(tuple(allowed_accesses)) |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
268 |
|
20b82fb8de02
backend: check nodes/items permission on disco#items:
Goffi <goffi@goffi.org>
parents:
347
diff
changeset
|
269 d = self.dbpool.runQuery(query, values) |
| 107 | 270 d.addCallback(lambda results: [r[0] for r in results]) |
| 271 return d | |
| 272 | |
| 352 | 273 def createNode(self, nodeIdentifier, owner, config, schema, pep, recipient=None): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
274 return self.dbpool.runInteraction(self._createNode, nodeIdentifier, |
| 352 | 275 owner, config, schema, pep, recipient) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
276 |
| 352 | 277 def _createNode(self, cursor, nodeIdentifier, owner, config, schema, pep, recipient): |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
278 if config['pubsub#node_type'] != 'leaf': |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
279 raise error.NoCollections() |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
280 |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
281 owner = owner.userhost() |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
282 |
| 107 | 283 try: |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
284 cursor.execute("""INSERT INTO nodes |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
285 (node, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
286 node_type, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
287 persist_items, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
288 deliver_payloads, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
289 send_last_published_item, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
290 access_model, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
291 publish_model, |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
292 overwrite_policy, |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
293 serial_ids, |
|
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
294 consistent_publisher, |
| 430 | 295 fts_language, |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
296 schema, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
297 pep) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
298 VALUES |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
299 (%s, 'leaf', %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)""", |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
300 (nodeIdentifier, |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
301 config['pubsub#persist_items'], |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
302 config['pubsub#deliver_payloads'], |
|
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
303 config['pubsub#send_last_published_item'], |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
304 config[const.OPT_ACCESS_MODEL], |
| 260 | 305 config[const.OPT_PUBLISH_MODEL], |
|
438
b5e1e8d93dd4
backend, pgsql: new `overwrite_policy` node setting:
Goffi <goffi@goffi.org>
parents:
434
diff
changeset
|
306 config[const.OPT_OVERWRITE_POLICY], |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
307 config[const.OPT_SERIAL_IDS], |
|
403
1dc606612405
implemented experimental "consistent_publisher" option:
Goffi <goffi@goffi.org>
parents:
402
diff
changeset
|
308 config[const.OPT_CONSISTENT_PUBLISHER], |
| 430 | 309 config[const.OPT_FTS_LANGUAGE], |
| 352 | 310 schema, |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
311 recipient.userhost() if pep else None |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
312 ) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
313 ) |
|
332
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
314 except cursor._pool.dbapi.IntegrityError as e: |
|
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
315 if e.pgcode == "23505": |
|
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
316 # unique_violation |
|
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
317 raise error.NodeExists() |
|
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
318 else: |
|
31cbd8b9fa7f
pgsql: node creation now return error.NodeExists in case of unique violation, and InvalidConfigurationOption else
Goffi <goffi@goffi.org>
parents:
331
diff
changeset
|
319 raise error.InvalidConfigurationOption() |
|
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
320 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
321 cursor.execute(*withPEP("""SELECT node_id FROM nodes WHERE node=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
322 (nodeIdentifier,), pep, recipient)); |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
323 node_id = cursor.fetchone()[0] |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
324 |
|
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
325 cursor.execute("""SELECT 1 as bool from entities where jid=%s""", |
|
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
326 (owner,)) |
| 107 | 327 |
| 328 if not cursor.fetchone(): | |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
329 # XXX: we can NOT rely on the previous query! Commit is needed now because |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
330 # if the entry exists the next query will leave the database in a corrupted |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
331 # state: the solution is to rollback. I tried with other methods like |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
332 # "WHERE NOT EXISTS" but none of them worked, so the following solution |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
333 # looks like the sole - unless you have auto-commit on. More info |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
334 # about this issue: http://cssmay.com/question/tag/tag-psycopg2 |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
335 cursor.connection.commit() |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
336 try: |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
337 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""", |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
338 (owner,)) |
|
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
339 except psycopg2.IntegrityError as e: |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
340 cursor.connection.rollback() |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
341 logging.warning("during node creation: %s" % e.message) |
| 107 | 342 |
| 343 cursor.execute("""INSERT INTO affiliations | |
| 344 (node_id, entity_id, affiliation) | |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
345 SELECT %s, entity_id, 'owner' FROM |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
346 (SELECT entity_id FROM entities |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
347 WHERE jid=%s) as e""", |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
348 (node_id, owner)) |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
349 |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
350 if config[const.OPT_ACCESS_MODEL] == const.VAL_AMODEL_PUBLISHER_ROSTER: |
|
259
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
351 if const.OPT_ROSTER_GROUPS_ALLOWED in config: |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
352 allowed_groups = config[const.OPT_ROSTER_GROUPS_ALLOWED] |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
353 else: |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
354 allowed_groups = [] |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
355 for group in allowed_groups: |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
356 #TODO: check that group are actually in roster |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
357 cursor.execute("""INSERT INTO node_groups_authorized (node_id, groupname) |
|
6fe7da6b4b32
node "roster" access model management
Goffi <goffi@goffi.org>
parents:
255
diff
changeset
|
358 VALUES (%s,%s)""" , (node_id, group)) |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
359 # XXX: affiliations can't be set on during node creation (at least not with XEP-0060 alone) |
|
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
360 # so whitelist affiliations need to be done afterward |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
361 |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
362 # no we may have to do extra things according to config options |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
363 default_conf = self.defaultConfig['leaf'] |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
364 # XXX: trigger works on node creation because OPT_SERIAL_IDS is False in defaultConfig |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
365 # if this value is changed, the _configurationTriggers method should be adapted. |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
366 Node._configurationTriggers(cursor, node_id, default_conf, config) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
367 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
368 def deleteNodeByDbId(self, db_id): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
369 """Delete a node using directly its database id""" |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
370 return self.dbpool.runInteraction(self._deleteNodeByDbId, db_id) |
| 107 | 371 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
372 def _deleteNodeByDbId(self, cursor, db_id): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
373 cursor.execute("""DELETE FROM nodes WHERE node_id=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
374 (db_id,)) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
375 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
376 if cursor.rowcount != 1: |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
377 raise error.NodeNotFound() |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
378 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
379 def deleteNode(self, nodeIdentifier, pep, recipient=None): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
380 return self.dbpool.runInteraction(self._deleteNode, nodeIdentifier, pep, recipient) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
381 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
382 def _deleteNode(self, cursor, nodeIdentifier, pep, recipient): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
383 cursor.execute(*withPEP("""DELETE FROM nodes WHERE node=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
384 (nodeIdentifier,), pep, recipient)) |
| 107 | 385 |
| 386 if cursor.rowcount != 1: | |
|
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
387 raise error.NodeNotFound() |
| 107 | 388 |
| 331 | 389 def getAffiliations(self, entity, nodeIdentifier, pep, recipient=None): |
| 390 return self.dbpool.runInteraction(self._getAffiliations, entity, nodeIdentifier, pep, recipient) | |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
391 |
| 331 | 392 def _getAffiliations(self, cursor, entity, nodeIdentifier, pep, recipient=None): |
| 393 query = ["""SELECT node, affiliation FROM entities | |
| 394 NATURAL JOIN affiliations | |
| 395 NATURAL JOIN nodes | |
| 396 WHERE jid=%s"""] | |
| 397 args = [entity.userhost()] | |
| 398 | |
| 399 if nodeIdentifier is not None: | |
| 400 query.append("AND node=%s") | |
| 401 args.append(nodeIdentifier) | |
| 402 | |
| 403 cursor.execute(*withPEP(' '.join(query), args, pep, recipient)) | |
| 404 rows = cursor.fetchall() | |
| 405 return [tuple(r) for r in rows] | |
| 107 | 406 |
|
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
407 def getSubscriptions(self, entity, nodeIdentifier=None, pep=False, recipient=None): |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
408 """retrieve subscriptions of an entity |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
409 |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
410 @param entity(jid.JID): entity to check |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
411 @param nodeIdentifier(unicode, None): node identifier |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
412 None to retrieve all subscriptions |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
413 @param pep: True if we are in PEP mode |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
414 @param recipient: jid of the recipient |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
415 """ |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
416 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
417 def toSubscriptions(rows): |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
418 subscriptions = [] |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
419 for row in rows: |
|
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
420 subscriber = jid.internJID('%s/%s' % (row.jid, |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
421 row.resource)) |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
422 subscription = Subscription(row.node, subscriber, row.state) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
423 subscriptions.append(subscription) |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
424 return subscriptions |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
425 |
|
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
426 query = ["""SELECT node, |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
427 jid, |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
428 resource, |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
429 state |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
430 FROM entities |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
431 NATURAL JOIN subscriptions |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
432 NATURAL JOIN nodes |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
433 WHERE jid=%s"""] |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
434 |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
435 args = [entity.userhost()] |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
436 |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
437 if nodeIdentifier is not None: |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
438 query.append("AND node=%s") |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
439 args.append(nodeIdentifier) |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
440 |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
441 d = self.dbpool.runQuery(*withPEP(' '.join(query), args, pep, recipient)) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
442 d.addCallback(toSubscriptions) |
| 107 | 443 return d |
| 444 | |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
445 def getDefaultConfiguration(self, nodeType): |
|
366
81e6d4a516c3
storage (pgsql): getDefaultConfiguration now returns a shallow copy of default conf dict, to avoid modification.
Goffi <goffi@goffi.org>
parents:
365
diff
changeset
|
446 return self.defaultConfig[nodeType].copy() |
| 107 | 447 |
|
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
448 def formatLastItems(self, result): |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
449 last_items = [] |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
450 for pep_jid_s, node, data, item_access_model in result: |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
451 pep_jid = jid.JID(pep_jid_s) |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
452 item = generic.stripNamespace(parseXml(data)) |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
453 last_items.append((pep_jid, node, item, item_access_model)) |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
454 return last_items |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
455 |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
456 def getLastItems(self, entities, nodes, node_accesses, item_accesses, pep): |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
457 """get last item for several nodes and entities in a single request""" |
|
463
f520ac3164b0
privilege: improvment on last message sending on presence with `+notify`:
Goffi <goffi@goffi.org>
parents:
457
diff
changeset
|
458 # TODO: manage other access model (whitelist, …) |
|
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
459 if not entities or not nodes or not node_accesses or not item_accesses: |
|
429
0526073ff2ab
pgsql: replaced ValueError by BadRequest:
Goffi <goffi@goffi.org>
parents:
424
diff
changeset
|
460 raise BadRequest(text="entities, nodes and accesses must not be empty") |
|
463
f520ac3164b0
privilege: improvment on last message sending on presence with `+notify`:
Goffi <goffi@goffi.org>
parents:
457
diff
changeset
|
461 if any(not {"open", "presence"}.issuperset(a) |
|
f520ac3164b0
privilege: improvment on last message sending on presence with `+notify`:
Goffi <goffi@goffi.org>
parents:
457
diff
changeset
|
462 for a in (node_accesses, item_accesses)): |
|
f520ac3164b0
privilege: improvment on last message sending on presence with `+notify`:
Goffi <goffi@goffi.org>
parents:
457
diff
changeset
|
463 raise NotImplementedError('only "open" and "presence" access model are handled for now') |
|
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
464 if not pep: |
| 414 | 465 raise NotImplementedError("getLastItems is only implemented for PEP at the moment") |
|
351
2098295747fd
pgsql: cast items.data to text when needed + db schema version bump.
Goffi <goffi@goffi.org>
parents:
349
diff
changeset
|
466 d = self.dbpool.runQuery("""SELECT DISTINCT ON (node_id) pep, node, data::text, items.access_model |
|
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
467 FROM items |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
468 NATURAL JOIN nodes |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
469 WHERE nodes.pep IN %s |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
470 AND node IN %s |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
471 AND nodes.access_model in %s |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
472 AND items.access_model in %s |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
473 ORDER BY node_id DESC, items.updated DESC, items.item_id DESC""", |
|
337
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
474 (tuple([e.userhost() for e in entities]), |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
475 nodes, |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
476 node_accesses, |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
477 item_accesses)) |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
478 d.addCallback(self.formatLastItems) |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
479 return d |
|
57a3051ee435
storage (pgsql): added getLastItems method, to get last items from a series of nodes and entities (needed for PEP compliance).
Goffi <goffi@goffi.org>
parents:
332
diff
changeset
|
480 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
481 |
| 414 | 482 @implementer(iidavoll.INode) |
| 107 | 483 class Node: |
| 484 | |
| 485 | |
| 352 | 486 def __init__(self, nodeDbId, nodeIdentifier, config, schema): |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
487 self.nodeDbId = nodeDbId |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
488 self.nodeIdentifier = nodeIdentifier |
| 107 | 489 self._config = config |
| 352 | 490 self._schema = schema |
| 107 | 491 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
492 def _checkNodeExists(self, cursor): |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
493 cursor.execute("""SELECT 1 as exist FROM nodes WHERE node_id=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
494 (self.nodeDbId,)) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
495 if not cursor.fetchone(): |
|
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
496 raise error.NodeNotFound() |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
497 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
498 def getType(self): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
499 return self.nodeType |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
500 |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
501 def getOwners(self): |
|
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
502 d = self.dbpool.runQuery("""SELECT jid FROM nodes NATURAL JOIN affiliations NATURAL JOIN entities WHERE node_id=%s and affiliation='owner'""", (self.nodeDbId,)) |
|
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
503 d.addCallback(lambda rows: [jid.JID(r[0]) for r in rows]) |
|
243
42048e37699e
added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents:
240
diff
changeset
|
504 return d |
|
42048e37699e
added experimental roster access_model (use remote_roster)
Goffi <goffi@goffi.org>
parents:
240
diff
changeset
|
505 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
506 def getConfiguration(self): |
| 107 | 507 return self._config |
| 508 | |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
509 def getNextId(self): |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
510 """return XMPP item id usable for next item to publish |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
511 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
512 the return value will be next int if serila_ids is set, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
513 else an UUID will be returned |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
514 """ |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
515 if self._config[const.OPT_SERIAL_IDS]: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
516 d = self.dbpool.runQuery("SELECT nextval('{seq_name}')".format( |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
517 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId))) |
| 414 | 518 d.addCallback(lambda rows: str(rows[0][0])) |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
519 return d |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
520 else: |
| 414 | 521 return defer.succeed(str(uuid.uuid4())) |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
522 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
523 @staticmethod |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
524 def _configurationTriggers(cursor, node_id, old_config, new_config): |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
525 """trigger database relative actions needed when a config is changed |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
526 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
527 @param cursor(): current db cursor |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
528 @param node_id(unicode): database ID of the node |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
529 @param old_config(dict): config of the node before the change |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
530 @param new_config(dict): new options that will be changed |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
531 """ |
| 430 | 532 if const.OPT_SERIAL_IDS not in new_config: |
| 533 return | |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
534 serial_ids = new_config[const.OPT_SERIAL_IDS] |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
535 if serial_ids != old_config[const.OPT_SERIAL_IDS]: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
536 # serial_ids option has been modified, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
537 # we need to handle corresponding sequence |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
538 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
539 # XXX: we use .format in following queries because values |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
540 # are generated by ourself |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
541 seq_name = ITEMS_SEQ_NAME.format(node_id=node_id) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
542 if serial_ids: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
543 # the next query get the max value +1 of all XMPP items ids |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
544 # which are integers, and default to 1 |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
545 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=node_id)) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
546 next_val = cursor.fetchone()[0] |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
547 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name)) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
548 cursor.execute("CREATE SEQUENCE {seq_name} START {next_val} OWNED BY nodes.node_id".format( |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
549 seq_name = seq_name, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
550 next_val = next_val)) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
551 else: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
552 cursor.execute("DROP SEQUENCE IF EXISTS {seq_name}".format(seq_name = seq_name)) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
553 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
554 def setConfiguration(self, options): |
| 430 | 555 to_delete = [] |
| 556 for option, value in options.items(): | |
| 557 try: | |
| 558 if self._config[option] == value: | |
| 559 to_delete.append(option) | |
| 560 except KeyError: | |
| 561 raise BadRequest(text=f"Invalid option: {option!r}") | |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
562 |
| 430 | 563 for option in to_delete: |
|
434
f6a225476165
pgsql: fixed `setConfiguration` when an updated field value is the same as current one
Goffi <goffi@goffi.org>
parents:
433
diff
changeset
|
564 del options.fields[option] |
|
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
565 |
|
434
f6a225476165
pgsql: fixed `setConfiguration` when an updated field value is the same as current one
Goffi <goffi@goffi.org>
parents:
433
diff
changeset
|
566 if not options.fields: |
| 430 | 567 return |
| 568 | |
| 569 if options.get(const.OPT_MAX_ITEMS) == "max": | |
|
424
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
570 # XXX: "max" is default value for config we must convert |
|
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
571 # it to an interger. See backend's _doSetNodeConfiguration comment |
| 430 | 572 options[const.OPT_MAX_ITEMS] = "0" |
|
424
3fce48c0a44d
psql: fixed use of default value for max items in `setConfiguration`
Goffi <goffi@goffi.org>
parents:
422
diff
changeset
|
573 |
| 430 | 574 if ((const.OPT_FTS_LANGUAGE in options |
| 575 and options[const.OPT_FTS_LANGUAGE] not in self.fts_languages)): | |
| 576 raise BadRequest(text= | |
| 577 f"invalid {const.OPT_FTS_LANGUAGE} value: " | |
| 578 f"{options[const.OPT_FTS_LANGUAGE]!r}" | |
| 579 ) | |
| 580 | |
| 581 | |
| 582 d = self.dbpool.runInteraction(self._setConfiguration, options) | |
| 583 d.addCallback(self._updateCachedConfiguration, options) | |
|
124
c4ee16bc48e5
Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents:
121
diff
changeset
|
584 return d |
|
c4ee16bc48e5
Change Node.set_configuration() to set cached configuration in a callback.
Ralph Meijer <ralphm@ik.nu>
parents:
121
diff
changeset
|
585 |
| 430 | 586 def _setConfiguration(self, cursor, options): |
| 587 self._configurationTriggers(cursor, self.nodeDbId, self._config, options) | |
| 588 # options names all follow the scheme "pubsub#{col_name}" | |
| 589 col_names = (o[7:] for o in options) | |
| 590 values = ','.join(f"{name}=%s" for name in col_names) | |
| 591 cursor.execute(f"UPDATE nodes SET {values} WHERE node_id=%s", | |
| 592 (*options.values(), self.nodeDbId)) | |
| 593 if cursor.rowcount == 0: | |
| 594 raise error.NodeNotFound() | |
| 107 | 595 |
| 430 | 596 def _updateCachedConfiguration(self, __, options): |
| 597 self._config.update(options) | |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
598 |
| 352 | 599 def getSchema(self): |
| 600 return self._schema | |
| 601 | |
| 602 def setSchema(self, schema): | |
| 603 d = self.dbpool.runInteraction(self._setSchema, schema) | |
| 604 d.addCallback(self._setCachedSchema, schema) | |
| 605 return d | |
| 606 | |
| 607 def _setSchema(self, cursor, schema): | |
| 608 self._checkNodeExists(cursor) | |
| 609 cursor.execute("""UPDATE nodes SET schema=%s | |
| 610 WHERE node_id=%s""", | |
| 611 (schema.toXml() if schema else None, | |
| 612 self.nodeDbId)) | |
| 613 | |
| 430 | 614 def _setCachedSchema(self, __, schema): |
| 352 | 615 self._schema = schema |
| 616 | |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
617 def getMetaData(self): |
| 107 | 618 config = copy.copy(self._config) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
619 config["pubsub#node_type"] = self.nodeType |
| 107 | 620 return config |
| 621 | |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
622 def getAffiliation(self, entity): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
623 return self.dbpool.runInteraction(self._getAffiliation, entity) |
| 107 | 624 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
625 def _getAffiliation(self, cursor, entity): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
626 self._checkNodeExists(cursor) |
| 107 | 627 cursor.execute("""SELECT affiliation FROM affiliations |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
628 NATURAL JOIN nodes |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
629 NATURAL JOIN entities |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
630 WHERE node_id=%s AND jid=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
631 (self.nodeDbId, |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
632 entity.userhost())) |
| 107 | 633 |
| 634 try: | |
| 635 return cursor.fetchone()[0] | |
| 636 except TypeError: | |
| 637 return None | |
| 638 | |
|
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
639 def getAccessModel(self): |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
640 return self._config[const.OPT_ACCESS_MODEL] |
|
240
70c8bb90d75f
added access_model to config, default to 'open'
Goffi <goffi@goffi.org>
parents:
235
diff
changeset
|
641 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
642 def getSubscription(self, subscriber): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
643 return self.dbpool.runInteraction(self._getSubscription, subscriber) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
644 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
645 def _getSubscription(self, cursor, subscriber): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
646 self._checkNodeExists(cursor) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
647 |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
648 userhost = subscriber.userhost() |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
649 resource = subscriber.resource or '' |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
650 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
651 cursor.execute("""SELECT state FROM subscriptions |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
652 NATURAL JOIN nodes |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
653 NATURAL JOIN entities |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
654 WHERE node_id=%s AND jid=%s AND resource=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
655 (self.nodeDbId, |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
656 userhost, |
|
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
657 resource)) |
|
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
658 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
659 row = cursor.fetchone() |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
660 if not row: |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
661 return None |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
662 else: |
|
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
663 return Subscription(self.nodeIdentifier, subscriber, row[0]) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
664 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
665 def getSubscriptions(self, state=None): |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
666 return self.dbpool.runInteraction(self._getSubscriptions, state) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
667 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
668 def _getSubscriptions(self, cursor, state): |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
669 self._checkNodeExists(cursor) |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
670 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
671 query = """SELECT node, jid, resource, state, |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
672 subscription_type, subscription_depth |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
673 FROM subscriptions |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
674 NATURAL JOIN nodes |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
675 NATURAL JOIN entities |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
676 WHERE node_id=%s""" |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
677 values = [self.nodeDbId] |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
678 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
679 if state: |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
680 query += " AND state=%s" |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
681 values.append(state) |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
682 |
|
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
683 cursor.execute(query, values) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
684 rows = cursor.fetchall() |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
685 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
686 subscriptions = [] |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
687 for row in rows: |
| 414 | 688 subscriber = jid.JID('%s/%s' % (row.jid, row.resource)) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
689 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
690 options = {} |
|
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
691 if row.subscription_type: |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
692 options['pubsub#subscription_type'] = row.subscription_type; |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
693 if row.subscription_depth: |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
694 options['pubsub#subscription_depth'] = row.subscription_depth; |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
695 |
|
357
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
696 subscriptions.append(Subscription(row.node, subscriber, |
|
1167e48e5f52
handle single node on subscriptions request
Goffi <goffi@goffi.org>
parents:
356
diff
changeset
|
697 row.state, options)) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
698 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
699 return subscriptions |
| 107 | 700 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
701 def addSubscription(self, subscriber, state, config): |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
702 return self.dbpool.runInteraction(self._addSubscription, subscriber, |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
703 state, config) |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
704 |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
705 def _addSubscription(self, cursor, subscriber, state, config): |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
706 self._checkNodeExists(cursor) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
707 |
| 107 | 708 userhost = subscriber.userhost() |
| 709 resource = subscriber.resource or '' | |
| 710 | |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
711 subscription_type = config.get('pubsub#subscription_type') |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
712 subscription_depth = config.get('pubsub#subscription_depth') |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
713 |
| 107 | 714 try: |
| 715 cursor.execute("""INSERT INTO entities (jid) VALUES (%s)""", | |
|
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
716 (userhost,)) |
|
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
717 except cursor._pool.dbapi.IntegrityError: |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
718 cursor.connection.rollback() |
| 107 | 719 |
| 720 try: | |
| 721 cursor.execute("""INSERT INTO subscriptions | |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
722 (node_id, entity_id, resource, state, |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
723 subscription_type, subscription_depth) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
724 SELECT %s, entity_id, %s, %s, %s, %s FROM |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
725 (SELECT entity_id FROM entities |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
726 WHERE jid=%s) AS ent_id""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
727 (self.nodeDbId, |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
728 resource, |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
729 state, |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
730 subscription_type, |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
731 subscription_depth, |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
732 userhost)) |
|
227
8540825f85e0
Replaced unmaintained pyPgSQL by Psycopg 2
Goffi <goffi@goffi.org>
parents:
206
diff
changeset
|
733 except cursor._pool.dbapi.IntegrityError: |
|
167
ef22e4150caa
Move protocol implementations (pubsub, disco, forms) to and depend on wokkel.
Ralph Meijer <ralphm@ik.nu>
parents:
163
diff
changeset
|
734 raise error.SubscriptionExists() |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
735 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
736 def removeSubscription(self, subscriber): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
737 return self.dbpool.runInteraction(self._removeSubscription, |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
738 subscriber) |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
739 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
740 def _removeSubscription(self, cursor, subscriber): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
741 self._checkNodeExists(cursor) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
742 |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
743 userhost = subscriber.userhost() |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
744 resource = subscriber.resource or '' |
| 107 | 745 |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
746 cursor.execute("""DELETE FROM subscriptions WHERE |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
747 node_id=%s AND |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
748 entity_id=(SELECT entity_id FROM entities |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
749 WHERE jid=%s) AND |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
750 resource=%s""", |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
751 (self.nodeDbId, |
|
136
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
752 userhost, |
|
327de183f48d
Discover client_encoding parameter to pyPgSQL, removing all encode() calls.
Ralph Meijer <ralphm@ik.nu>
parents:
127
diff
changeset
|
753 resource)) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
754 if cursor.rowcount != 1: |
|
171
bc269696ef42
Reply with the correct error condition on subscription when not subscribed.
Ralph Meijer <ralphm@ik.nu>
parents:
170
diff
changeset
|
755 raise error.NotSubscribed() |
| 107 | 756 |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
757 return None |
| 107 | 758 |
|
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
759 def setSubscriptions(self, subscriptions): |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
760 return self.dbpool.runInteraction(self._setSubscriptions, subscriptions) |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
761 |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
762 def _setSubscriptions(self, cursor, subscriptions): |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
763 self._checkNodeExists(cursor) |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
764 |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
765 entities = self.getOrCreateEntities(cursor, [s.subscriber for s in subscriptions]) |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
766 entities_map = {jid.JID(e.jid): e for e in entities} |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
767 |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
768 # then we construct values for subscriptions update according to entity_id we just got |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
769 placeholders = ','.join(len(subscriptions) * ["%s"]) |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
770 values = [] |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
771 for subscription in subscriptions: |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
772 entity_id = entities_map[subscription.subscriber].entity_id |
| 414 | 773 resource = subscription.subscriber.resource or '' |
|
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
774 values.append((self.nodeDbId, entity_id, resource, subscription.state, None, None)) |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
775 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5 |
|
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
776 cursor.execute("INSERT INTO subscriptions(node_id, entity_id, resource, state, subscription_type, subscription_depth) VALUES " + placeholders + " ON CONFLICT (entity_id, resource, node_id) DO UPDATE SET state=EXCLUDED.state", [v for v in values]) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
777 |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
778 def isSubscribed(self, entity): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
779 return self.dbpool.runInteraction(self._isSubscribed, entity) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
780 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
781 def _isSubscribed(self, cursor, entity): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
782 self._checkNodeExists(cursor) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
783 |
|
276
b757c29b20d7
import changes from idavoll changeset 233 (24be3a11ddbc), by Ralph Meijer and based on a patch by Nuno Santos
souliane <souliane@mailoo.org>
parents:
271
diff
changeset
|
784 cursor.execute("""SELECT 1 as bool FROM entities |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
785 NATURAL JOIN subscriptions |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
786 NATURAL JOIN nodes |
|
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
787 WHERE entities.jid=%s |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
788 AND node_id=%s AND state='subscribed'""", |
|
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
789 (entity.userhost(), |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
790 self.nodeDbId)) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
791 |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
792 return cursor.fetchone() is not None |
| 107 | 793 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
794 def getAffiliations(self): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
795 return self.dbpool.runInteraction(self._getAffiliations) |
|
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
796 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
797 def _getAffiliations(self, cursor): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
798 self._checkNodeExists(cursor) |
|
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
799 |
|
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
800 cursor.execute("""SELECT jid, affiliation FROM nodes |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
801 NATURAL JOIN affiliations |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
802 NATURAL JOIN entities |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
803 WHERE node_id=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
804 (self.nodeDbId,)) |
|
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
805 result = cursor.fetchall() |
|
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
806 |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
807 return {jid.internJID(r[0]): r[1] for r in result} |
|
145
f393bccec4bc
Add get_affiliations to Node class in storage facilities in preparation of
Ralph Meijer <ralphm@ik.nu>
parents:
142
diff
changeset
|
808 |
| 331 | 809 def getOrCreateEntities(self, cursor, entities_jids): |
| 810 """Get entity_id from entities in entities table | |
| 811 | |
| 812 Entities will be inserted it they don't exist | |
| 813 @param entities_jid(list[jid.JID]): entities to get or create | |
|
341
b49f75a26156
backend, pgsql: implemented subscriptionsGet and subscriptionsSet
Goffi <goffi@goffi.org>
parents:
337
diff
changeset
|
814 @return list[record(entity_id,jid)]]: list of entity_id and jid (as plain string) |
| 331 | 815 both existing and inserted entities are returned |
| 816 """ | |
| 817 # cf. http://stackoverflow.com/a/35265559 | |
| 818 placeholders = ','.join(len(entities_jids) * ["(%s)"]) | |
| 819 query = ( | |
| 820 """ | |
| 821 WITH | |
| 822 jid_values (jid) AS ( | |
| 823 VALUES {placeholders} | |
| 824 ), | |
| 825 inserted (entity_id, jid) AS ( | |
| 826 INSERT INTO entities (jid) | |
| 827 SELECT jid | |
| 828 FROM jid_values | |
| 829 ON CONFLICT DO NOTHING | |
| 830 RETURNING entity_id, jid | |
| 831 ) | |
| 832 SELECT e.entity_id, e.jid | |
| 833 FROM entities e JOIN jid_values jv ON jv.jid = e.jid | |
| 834 UNION ALL | |
| 835 SELECT entity_id, jid | |
| 836 FROM inserted""".format(placeholders=placeholders)) | |
| 837 cursor.execute(query, [j.userhost() for j in entities_jids]) | |
| 838 return cursor.fetchall() | |
| 839 | |
| 840 def setAffiliations(self, affiliations): | |
| 841 return self.dbpool.runInteraction(self._setAffiliations, affiliations) | |
| 842 | |
| 843 def _setAffiliations(self, cursor, affiliations): | |
| 844 self._checkNodeExists(cursor) | |
| 845 | |
| 846 entities = self.getOrCreateEntities(cursor, affiliations) | |
| 847 | |
| 848 # then we construct values for affiliations update according to entity_id we just got | |
| 849 placeholders = ','.join(len(affiliations) * ["(%s,%s,%s)"]) | |
| 850 values = [] | |
| 414 | 851 for e in entities: |
| 852 values.extend((e.entity_id, affiliations[jid.JID(e.jid)], self.nodeDbId)) | |
| 331 | 853 |
| 854 # we use upsert so new values are inserted and existing one updated. This feature is only available for PostgreSQL >= 9.5 | |
| 855 cursor.execute("INSERT INTO affiliations(entity_id,affiliation,node_id) VALUES " + placeholders + " ON CONFLICT (entity_id,node_id) DO UPDATE SET affiliation=EXCLUDED.affiliation", values) | |
| 856 | |
| 857 def deleteAffiliations(self, entities): | |
| 858 return self.dbpool.runInteraction(self._deleteAffiliations, entities) | |
| 859 | |
| 860 def _deleteAffiliations(self, cursor, entities): | |
| 861 """delete affiliations and subscriptions for this entity""" | |
| 862 self._checkNodeExists(cursor) | |
| 863 placeholders = ','.join(len(entities) * ["%s"]) | |
| 864 cursor.execute("DELETE FROM affiliations WHERE node_id=%s AND entity_id in (SELECT entity_id FROM entities WHERE jid IN (" + placeholders + ")) RETURNING entity_id", [self.nodeDbId] + [e.userhost() for e in entities]) | |
| 865 | |
| 866 rows = cursor.fetchall() | |
| 867 placeholders = ','.join(len(rows) * ["%s"]) | |
| 868 cursor.execute("DELETE FROM subscriptions WHERE node_id=%s AND entity_id in (" + placeholders + ")", [self.nodeDbId] + [r[0] for r in rows]) | |
| 869 | |
| 870 def getAuthorizedGroups(self): | |
| 871 return self.dbpool.runInteraction(self._getNodeGroups) | |
| 872 | |
| 873 def _getAuthorizedGroups(self, cursor): | |
| 874 cursor.execute("SELECT groupname FROM node_groups_authorized NATURAL JOIN nodes WHERE node=%s", | |
| 875 (self.nodeDbId,)) | |
| 876 rows = cursor.fetchall() | |
| 877 return [row[0] for row in rows] | |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
878 |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
879 |
| 414 | 880 @implementer(iidavoll.ILeafNode) |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
881 class LeafNode(Node): |
|
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
882 |
| 107 | 883 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
884 nodeType = 'leaf' |
| 107 | 885 |
| 451 | 886 def getOrderBy(self, ext_data, direction='ASC'): |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
887 """Return ORDER BY clause corresponding to Order By key in ext_data |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
888 |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
889 @param ext_data (dict): extra data as used in getItems |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
890 @param direction (unicode): ORDER BY direction (ASC or DESC) |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
891 @return (unicode): ORDER BY clause to use |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
892 """ |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
893 keys = ext_data.get('order_by') |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
894 if not keys: |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
895 return f"ORDER BY updated {direction}, item_id {direction}" |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
896 cols_statmnt = [] |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
897 for key in keys: |
|
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
898 if key == 'creation': |
|
457
7c9792f934a2
psql: use `created` colum to sort by creation:
Goffi <goffi@goffi.org>
parents:
456
diff
changeset
|
899 column = 'created' |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
900 elif key == 'modification': |
|
378
22832c1d2827
storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents:
375
diff
changeset
|
901 column = 'updated' |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
902 else: |
| 414 | 903 log.msg("WARNING: Unknown order by key: {key}".format(key=key)) |
|
378
22832c1d2827
storage (pgsql): don't specify table name in ORDER BY clause as it can cause troubles in some requests.
Goffi <goffi@goffi.org>
parents:
375
diff
changeset
|
904 column = 'updated' |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
905 cols_statmnt.append(f"{column} {direction}") |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
906 |
|
457
7c9792f934a2
psql: use `created` colum to sort by creation:
Goffi <goffi@goffi.org>
parents:
456
diff
changeset
|
907 cols_statmnt.append(f"item_id {direction}") |
| 414 | 908 return "ORDER BY " + ",".join([col for col in cols_statmnt]) |
|
375
9a787881b824
implemented Order-By ProtoXEP (MAM + PubSub)
Goffi <goffi@goffi.org>
parents:
374
diff
changeset
|
909 |
|
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
910 @defer.inlineCallbacks |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
911 def storeItems(self, items_data, publisher): |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
912 # XXX: runInteraction doesn't seem to work when there are several "insert" |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
913 # or "update". |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
914 # Before the unpacking was done in _storeItems, but this was causing trouble |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
915 # in case of multiple items_data. So this has now be moved here. |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
916 # FIXME: investigate the issue with runInteraction |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
917 for item_data in items_data: |
|
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
918 yield self.dbpool.runInteraction(self._storeItems, item_data, publisher) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
919 |
|
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
920 def _storeItems(self, cursor, item_data, publisher): |
| 250 | 921 self._checkNodeExists(cursor) |
|
393
728b08c0d000
storage (pgsql): fixed storage of multiple items
Goffi <goffi@goffi.org>
parents:
390
diff
changeset
|
922 self._storeItem(cursor, item_data, publisher) |
| 244 | 923 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
924 def _storeItem(self, cursor, item_data, publisher): |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
925 # first try to insert the item |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
926 # - if it fails (conflict), and the item is new and we have serial_ids options, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
927 # current id will be recomputed using next item id query (note that is not perfect, as |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
928 # table is not locked and this can fail if two items are added at the same time |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
929 # but this can only happen with serial_ids and if future ids have been set by a client, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
930 # this case should be rare enough to consider this situation acceptable) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
931 # - if item insertion fail and the item is not new, we do an update |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
932 # - in other cases, exception is raised |
|
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
933 item, access_model, item_config = item_data.item, item_data.access_model, item_data.config |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
934 data = item.toXml() |
| 430 | 935 data_fts_cfg = self._config[const.OPT_FTS_LANGUAGE] |
| 936 if data_fts_cfg == const.VAL_FTS_GENERIC: | |
| 937 data_fts_cfg = "simple" | |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
938 |
| 430 | 939 insert_query = ( |
| 940 "INSERT INTO items(node_id, item, publisher, data, access_model, " | |
| 941 "data_fts_cfg) VALUES (%s, %s, %s, %s, %s, %s) RETURNING item_id" | |
| 942 ) | |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
943 insert_data = [self.nodeDbId, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
944 item["id"], |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
945 publisher.full(), |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
946 data, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
947 access_model, |
| 430 | 948 data_fts_cfg, |
| 949 ] | |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
950 |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
951 try: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
952 cursor.execute(insert_query, insert_data) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
953 except cursor._pool.dbapi.IntegrityError as e: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
954 if e.pgcode != "23505": |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
955 # we only handle unique_violation, every other exception must be raised |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
956 raise e |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
957 cursor.connection.rollback() |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
958 # the item already exist |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
959 if item_data.new: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
960 # the item is new |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
961 if self._config[const.OPT_SERIAL_IDS]: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
962 # this can happen with serial_ids, if a item has been stored |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
963 # with a future id (generated by XMPP client) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
964 cursor.execute(NEXT_ITEM_ID_QUERY.format(node_id=self.nodeDbId)) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
965 next_id = cursor.fetchone()[0] |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
966 # we update the sequence, so we can skip conflicting ids |
| 414 | 967 cursor.execute("SELECT setval('{seq_name}', %s)".format( |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
968 seq_name = ITEMS_SEQ_NAME.format(node_id=self.nodeDbId)), [next_id]) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
969 # and now we can retry the query with the new id |
| 414 | 970 item['id'] = insert_data[1] = str(next_id) |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
971 # item saved in DB must also be updated with the new id |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
972 insert_data[3] = item.toXml() |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
973 cursor.execute(insert_query, insert_data) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
974 else: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
975 # but if we have not serial_ids, we have a real problem |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
976 raise e |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
977 else: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
978 # this is an update |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
979 cursor.execute("""UPDATE items SET updated=now(), publisher=%s, data=%s |
| 430 | 980 WHERE node_id=%s AND items.item=%s |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
981 RETURNING item_id""", |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
982 (publisher.full(), |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
983 data, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
984 self.nodeDbId, |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
985 item["id"])) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
986 if cursor.rowcount != 1: |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
987 raise exceptions.InternalError("item has not been updated correctly") |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
988 item_id = cursor.fetchone()[0]; |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
989 self._storeCategories(cursor, item_id, item_data.categories, update=True) |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
990 return |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
991 |
|
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
992 item_id = cursor.fetchone()[0]; |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
993 self._storeCategories(cursor, item_id, item_data.categories) |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
994 |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
995 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
| 250 | 996 if const.OPT_ROSTER_GROUPS_ALLOWED in item_config: |
| 422 | 997 # XXX: needed to force list if there is only one value |
| 998 item_config.fields[const.OPT_ROSTER_GROUPS_ALLOWED].fieldType='list-multi' | |
| 250 | 999 allowed_groups = item_config[const.OPT_ROSTER_GROUPS_ALLOWED] |
|
245
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1000 else: |
|
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1001 allowed_groups = [] |
|
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1002 for group in allowed_groups: |
|
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1003 #TODO: check that group are actually in roster |
|
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1004 cursor.execute("""INSERT INTO item_groups_authorized (item_id, groupname) |
|
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1005 VALUES (%s,%s)""" , (item_id, group)) |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1006 # TODO: whitelist access model |
|
245
e11e99246be5
allowed groups from item_config are now stored
Goffi <goffi@goffi.org>
parents:
244
diff
changeset
|
1007 |
| 422 | 1008 max_items = self._config.get(const.OPT_MAX_ITEMS, 'max') |
| 1009 | |
| 1010 if max_items != 'max': | |
| 1011 try: | |
| 1012 max_items = int(self._config[const.OPT_MAX_ITEMS]) | |
| 1013 except ValueError: | |
| 1014 log.err(f"Invalid max_items value: {max_items!r}") | |
| 1015 else: | |
| 1016 if max_items > 0: | |
| 1017 # we delete all items above the requested max | |
| 1018 cursor.execute( | |
| 1019 "DELETE FROM items WHERE node_id=%s and item_id in (SELECT " | |
| 1020 "item_id FROM items WHERE node_id=%s ORDER BY items.updated " | |
| 1021 "DESC, items.item_id DESC OFFSET %s)" , | |
| 1022 (self.nodeDbId, self.nodeDbId, max_items) | |
| 1023 ) | |
| 1024 | |
| 1025 | |
|
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1026 def _storeCategories(self, cursor, item_id, categories, update=False): |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1027 # TODO: handle canonical form |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1028 if update: |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1029 cursor.execute("""DELETE FROM item_categories |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1030 WHERE item_id=%s""", (item_id,)) |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1031 |
|
402
724e39d596a9
psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents:
393
diff
changeset
|
1032 # we use a set to avoid duplicates |
|
724e39d596a9
psql: use a set for categories, to avoid duplicates (and constraint failure)
Goffi <goffi@goffi.org>
parents:
393
diff
changeset
|
1033 for category in set(categories): |
|
301
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1034 cursor.execute("""INSERT INTO item_categories (item_id, category) |
|
05c875a13a62
categories are now stored in a dedicated table if item contain an atom entry:
Goffi <goffi@goffi.org>
parents:
297
diff
changeset
|
1035 VALUES (%s, %s)""", (item_id, category)) |
| 107 | 1036 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1037 def removeItems(self, itemIdentifiers): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
1038 return self.dbpool.runInteraction(self._removeItems, itemIdentifiers) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1039 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1040 def _removeItems(self, cursor, itemIdentifiers): |
|
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1041 self._checkNodeExists(cursor) |
|
180
fc687620599b
Properly add send_last_published_item configuration item.
Ralph Meijer <ralphm@ik.nu>
parents:
173
diff
changeset
|
1042 |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1043 deleted = [] |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1044 |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1045 for itemIdentifier in itemIdentifiers: |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1046 cursor.execute("""DELETE FROM items WHERE |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1047 node_id=%s AND |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1048 item=%s""", |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1049 (self.nodeDbId, |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1050 itemIdentifier)) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1051 |
|
142
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1052 if cursor.rowcount: |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1053 deleted.append(itemIdentifier) |
|
142
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1054 |
|
812300cdbc22
Changed behaviour of retraction of items so that only the actually deleted
Ralph Meijer <ralphm@ik.nu>
parents:
136
diff
changeset
|
1055 return deleted |
| 107 | 1056 |
| 278 | 1057 def getItems(self, authorized_groups, unrestricted, maxItems=None, ext_data=None): |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1058 """ Get all authorised items |
| 346 | 1059 |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1060 @param authorized_groups: we want to get items that these groups can access |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1061 @param unrestricted: if true, don't check permissions (i.e.: get all items) |
| 318 | 1062 @param maxItems: nb of items we want to get |
| 1063 @param ext_data: options for extra features like RSM and MAM | |
| 278 | 1064 |
|
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1065 @return: list of container.ItemData |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1066 if unrestricted is False, access_model and config will be None |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1067 """ |
| 278 | 1068 if ext_data is None: |
| 1069 ext_data = {} | |
| 347 | 1070 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=False) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1071 |
| 346 | 1072 def getItemsIds(self, authorized_groups, unrestricted, maxItems=None, ext_data=None): |
| 1073 """ Get all authorised items ids | |
| 1074 | |
| 1075 @param authorized_groups: we want to get items that these groups can access | |
| 1076 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
| 1077 @param maxItems: nb of items we want to get | |
| 1078 @param ext_data: options for extra features like RSM and MAM | |
| 1079 | |
| 1080 @return list(unicode): list of ids | |
| 1081 """ | |
| 1082 if ext_data is None: | |
| 1083 ext_data = {} | |
| 1084 return self.dbpool.runInteraction(self._getItems, authorized_groups, unrestricted, maxItems, ext_data, ids_only=True) | |
| 1085 | |
| 318 | 1086 def _appendSourcesAndFilters(self, query, args, authorized_groups, unrestricted, ext_data): |
| 1087 """append sources and filters to sql query requesting items and return ORDER BY | |
| 278 | 1088 |
| 318 | 1089 arguments query, args, authorized_groups, unrestricted and ext_data are the same as for |
| 1090 _getItems | |
| 1091 """ | |
| 1092 # SOURCES | |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1093 query.append("FROM items") |
| 323 | 1094 |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1095 if unrestricted: |
| 323 | 1096 query_filters = ["WHERE node_id=%s"] |
| 318 | 1097 args.append(self.nodeDbId) |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1098 else: |
| 323 | 1099 query.append("LEFT JOIN item_groups_authorized USING (item_id)") |
| 318 | 1100 args.append(self.nodeDbId) |
| 1101 if authorized_groups: | |
| 1102 get_groups = " or (items.access_model='roster' and groupname in %s)" | |
| 1103 args.append(authorized_groups) | |
| 1104 else: | |
| 1105 get_groups = "" | |
| 278 | 1106 |
| 323 | 1107 query_filters = ["WHERE node_id=%s AND (items.access_model='open'" + get_groups + ")"] |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1108 |
| 318 | 1109 # FILTERS |
|
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1110 if 'filters' in ext_data: # MAM filters |
|
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1111 for filter_ in ext_data['filters']: |
|
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1112 if filter_.var == 'start': |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1113 query_filters.append("AND created>=%s") |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1114 args.append(filter_.value) |
| 323 | 1115 elif filter_.var == 'end': |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1116 query_filters.append("AND created<=%s") |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1117 args.append(filter_.value) |
| 323 | 1118 elif filter_.var == 'with': |
|
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1119 jid_s = filter_.value |
|
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1120 if '/' in jid_s: |
| 323 | 1121 query_filters.append("AND publisher=%s") |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1122 args.append(filter_.value) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1123 else: |
| 323 | 1124 query_filters.append("AND publisher LIKE %s") |
| 414 | 1125 args.append("{}%".format(filter_.value)) |
| 430 | 1126 elif filter_.var == const.MAM_FILTER_FTS: |
| 1127 fts_cfg = self._config[const.OPT_FTS_LANGUAGE] | |
| 1128 if fts_cfg == const.VAL_FTS_GENERIC: | |
| 1129 fts_cfg = "simple" | |
| 1130 query_filters.append( | |
| 1131 "AND data_fts @@ websearch_to_tsquery(%s, %s)" | |
| 1132 ) | |
| 1133 args.append(fts_cfg) | |
| 1134 args.append(filter_.value) | |
| 323 | 1135 elif filter_.var == const.MAM_FILTER_CATEGORY: |
| 1136 query.append("LEFT JOIN item_categories USING (item_id)") | |
| 1137 query_filters.append("AND category=%s") | |
| 1138 args.append(filter_.value) | |
| 1139 else: | |
| 1140 log.msg("WARNING: unknown filter: {}".format(filter_.encode('utf-8'))) | |
| 1141 | |
| 1142 query.extend(query_filters) | |
|
282
7d54ff2eeaf2
actually retrieve the MAM messages
souliane <souliane@mailoo.org>
parents:
278
diff
changeset
|
1143 |
| 451 | 1144 return self.getOrderBy(ext_data, direction="ASC") |
| 318 | 1145 |
| 346 | 1146 def _getItems(self, cursor, authorized_groups, unrestricted, maxItems, ext_data, ids_only): |
| 318 | 1147 self._checkNodeExists(cursor) |
| 1148 | |
| 1149 if maxItems == 0: | |
| 1150 return [] | |
| 1151 | |
| 1152 args = [] | |
| 1153 | |
| 1154 # SELECT | |
| 346 | 1155 if ids_only: |
| 1156 query = ["SELECT item"] | |
| 1157 else: | |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1158 query = ["SELECT data::text,items.access_model,item_id,created,updated"] |
| 318 | 1159 |
| 1160 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
| 278 | 1161 |
| 1162 if 'rsm' in ext_data: | |
| 1163 rsm = ext_data['rsm'] | |
| 1164 maxItems = rsm.max | |
| 1165 if rsm.index is not None: | |
|
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1166 # We need to know the item_id of corresponding to the index (offset) of the current query |
|
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1167 # so we execute the query to look for the item_id |
| 318 | 1168 tmp_query = query[:] |
| 1169 tmp_args = args[:] | |
|
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1170 tmp_query[0] = "SELECT item_id" |
| 318 | 1171 tmp_query.append("{} LIMIT 1 OFFSET %s".format(query_order)) |
| 1172 tmp_args.append(rsm.index) | |
| 1173 cursor.execute(' '.join(query), args) | |
| 1174 # FIXME: bad index is not managed yet | |
|
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1175 item_id = cursor.fetchall()[0][0] |
| 318 | 1176 |
|
319
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1177 # now that we have the id, we can use it |
|
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1178 query.append("AND item_id<=%s") |
|
a51947371625
pgsql storage: use item_id instead of date to order and select items
Goffi <goffi@goffi.org>
parents:
318
diff
changeset
|
1179 args.append(item_id) |
| 278 | 1180 elif rsm.before is not None: |
| 1181 if rsm.before != '': | |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1182 query.insert(0,"SELECT * from (") |
|
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1183 query[1] += f",row_number() OVER ({query_order}) as row_number" |
|
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1184 query.append(f"{query_order}) as x") |
|
450
96059231d58f
pgsql: fix RSM before and after subqueries
Goffi <goffi@goffi.org>
parents:
440
diff
changeset
|
1185 query.append( |
| 456 | 1186 "WHERE row_number<(SELECT row_number FROM (SELECT row_number() " |
| 1187 f"OVER ({query_order}) as row_number,item" | |
|
450
96059231d58f
pgsql: fix RSM before and after subqueries
Goffi <goffi@goffi.org>
parents:
440
diff
changeset
|
1188 ) |
| 456 | 1189 self._appendSourcesAndFilters( |
| 1190 query, args, authorized_groups, unrestricted, ext_data | |
| 1191 ) | |
| 1192 query.append(") as row_num_sub WHERE item=%s)") | |
| 1193 args.append(rsm.before) | |
| 451 | 1194 # we need to reverse order in a first query to get the right |
| 1195 # items | |
| 1196 query.insert(0,"SELECT * from (") | |
| 1197 query.append(self.getOrderBy(ext_data, direction='DESC')) | |
| 1198 query.append("LIMIT %s) as x") | |
| 1199 if maxItems is None: | |
| 1200 # we default to 10 max when we request last page | |
| 1201 log.msg( | |
| 1202 "WARNING: getting last page with RSM without max, " | |
| 1203 "we default to max=10" | |
| 1204 ) | |
| 1205 args.append(10) | |
| 1206 else: | |
| 318 | 1207 args.append(maxItems) |
| 278 | 1208 elif rsm.after: |
|
454
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1209 query.insert(0,"SELECT * from (") |
|
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1210 query[1] += f",row_number() OVER ({query_order}) as row_number" |
|
7f1394bb96db
psql: fix ordering by using windowed function `row_number`
Goffi <goffi@goffi.org>
parents:
451
diff
changeset
|
1211 query.append(f"{query_order}) as x") |
|
450
96059231d58f
pgsql: fix RSM before and after subqueries
Goffi <goffi@goffi.org>
parents:
440
diff
changeset
|
1212 query.append( |
| 456 | 1213 "WHERE row_number>(SELECT row_number FROM (SELECT row_number() " |
| 1214 f"OVER ({query_order}) as row_number,item" | |
|
450
96059231d58f
pgsql: fix RSM before and after subqueries
Goffi <goffi@goffi.org>
parents:
440
diff
changeset
|
1215 ) |
| 456 | 1216 self._appendSourcesAndFilters( |
| 1217 query, args, authorized_groups, unrestricted, ext_data | |
| 1218 ) | |
| 1219 query.append(") as row_num_sub WHERE item=%s)") | |
| 1220 args.append(rsm.after) | |
| 451 | 1221 else: |
| 1222 rsm = False | |
| 278 | 1223 |
| 318 | 1224 query.append(query_order) |
| 278 | 1225 |
|
297
4115999d85e9
fixes behavior when maxItems is 0
souliane <souliane@mailoo.org>
parents:
294
diff
changeset
|
1226 if maxItems is not None: |
| 451 | 1227 if rsm == False: |
| 1228 # if we have legacy maxItems (i.e. not RSM max), we need to | |
| 1229 # reverse order (maxItems get last items, cf. | |
| 1230 # https://xmpp.org/extensions/xep-0060.html#subscriber-retrieve-requestrecent) | |
| 1231 # in a first query to get the right items | |
| 1232 query.insert(0,"SELECT * from (") | |
| 1233 query.insert(len(query)-1, self.getOrderBy(ext_data, direction='DESC')) | |
| 1234 query.insert(len(query)-1, "LIMIT %s) as x") | |
| 1235 args.append(maxItems) | |
| 1236 else: | |
| 1237 query.append("LIMIT %s") | |
| 1238 args.append(maxItems) | |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1239 |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1240 cursor.execute(' '.join(query), args) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1241 |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1242 result = cursor.fetchall() |
| 346 | 1243 if unrestricted and not ids_only: |
| 318 | 1244 # with unrestricted query, we need to fill the access_list for a roster access items |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1245 ret = [] |
|
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1246 for item_data in result: |
|
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1247 item = generic.stripNamespace(parseXml(item_data.data)) |
|
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1248 access_model = item_data.access_model |
|
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1249 item_id = item_data.item_id |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1250 created = item_data.created |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1251 updated = item_data.updated |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1252 access_list = {} |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1253 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1254 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,)) |
|
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1255 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r.groupname for r in cursor.fetchall()] |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1256 |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1257 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated)) |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1258 # TODO: whitelist item access model |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1259 return ret |
| 318 | 1260 |
| 346 | 1261 if ids_only: |
|
356
95c83899b5e9
pgsql: fixed bad data filling in getItemsData:
Goffi <goffi@goffi.org>
parents:
352
diff
changeset
|
1262 return [r.item for r in result] |
| 346 | 1263 else: |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1264 items_data = [container.ItemData(generic.stripNamespace(parseXml(r.data)), r.access_model, created=r.created, updated=r.updated) for r in result] |
|
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1265 return items_data |
| 107 | 1266 |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1267 def getItemsById(self, authorized_groups, unrestricted, itemIdentifiers): |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1268 """Get items which are in the given list |
|
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1269 |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1270 @param authorized_groups: we want to get items that these groups can access |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1271 @param unrestricted: if true, don't check permissions |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1272 @param itemIdentifiers: list of ids of the items we want to get |
|
317
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1273 @return: list of container.ItemData |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1274 ItemData.config will contains access_list (managed as a dictionnary with same key as for item_config) |
|
34adc4a8aa64
new container module, with an ItemData container:
Goffi <goffi@goffi.org>
parents:
312
diff
changeset
|
1275 if unrestricted is False, access_model and config will be None |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1276 """ |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1277 return self.dbpool.runInteraction(self._getItemsById, authorized_groups, unrestricted, itemIdentifiers) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1278 |
|
248
50f6ee966da8
item are gotten according to item's access model in getItems
Goffi <goffi@goffi.org>
parents:
247
diff
changeset
|
1279 def _getItemsById(self, cursor, authorized_groups, unrestricted, itemIdentifiers): |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1280 self._checkNodeExists(cursor) |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1281 ret = [] |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1282 if unrestricted: #we get everything without checking permissions |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1283 for itemIdentifier in itemIdentifiers: |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1284 cursor.execute("""SELECT data::text,items.access_model,item_id,created,updated FROM nodes |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1285 INNER JOIN items USING (node_id) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1286 WHERE node_id=%s AND item=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1287 (self.nodeDbId, |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1288 itemIdentifier)) |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1289 result = cursor.fetchone() |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1290 if not result: |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1291 raise error.ItemNotFound() |
| 107 | 1292 |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1293 item = generic.stripNamespace(parseXml(result[0])) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1294 access_model = result[1] |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1295 item_id = result[2] |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1296 created= result[3] |
|
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1297 updated= result[4] |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1298 access_list = {} |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1299 if access_model == const.VAL_AMODEL_PUBLISHER_ROSTER: |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1300 cursor.execute('SELECT groupname FROM item_groups_authorized WHERE item_id=%s', (item_id,)) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1301 access_list[const.OPT_ROSTER_GROUPS_ALLOWED] = [r[0] for r in cursor.fetchall()] |
|
330
82d1259b3e36
backend, pgsql storage: better items/notification handling, various fixes:
Goffi <goffi@goffi.org>
parents:
324
diff
changeset
|
1302 #TODO: WHITELIST access_model |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1303 |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1304 ret.append(container.ItemData(item, access_model, access_list, created=created, updated=updated)) |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1305 else: #we check permission before returning items |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1306 for itemIdentifier in itemIdentifiers: |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1307 args = [self.nodeDbId, itemIdentifier] |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1308 if authorized_groups: |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1309 args.append(authorized_groups) |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1310 cursor.execute("""SELECT data::text, created, updated FROM nodes |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1311 INNER JOIN items USING (node_id) |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1312 LEFT JOIN item_groups_authorized USING (item_id) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1313 WHERE node_id=%s AND item=%s AND |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1314 (items.access_model='open' """ + |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1315 ("or (items.access_model='roster' and groupname in %s)" if authorized_groups else '') + ")", |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1316 args) |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1317 |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1318 result = cursor.fetchone() |
|
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1319 if result: |
|
367
a772f7dac930
backend, storage(pgsql): creation/update date + serial ids:
Goffi <goffi@goffi.org>
parents:
366
diff
changeset
|
1320 ret.append(container.ItemData(generic.stripNamespace(parseXml(result[0])), created=result[1], updated=result[2])) |
|
262
7b821432d012
fixed node auto-create feature
souliane <souliane@mailoo.org>
parents:
260
diff
changeset
|
1321 |
|
252
25a1dc7181cc
full items, with item-configuration, are returned if items are asked by the owner
Goffi <goffi@goffi.org>
parents:
250
diff
changeset
|
1322 return ret |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1323 |
| 318 | 1324 def getItemsCount(self, authorized_groups, unrestricted, ext_data=None): |
| 1325 """Count expected number of items in a getItems query | |
| 1326 | |
| 1327 @param authorized_groups: we want to get items that these groups can access | |
| 1328 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
| 1329 @param ext_data: options for extra features like RSM and MAM | |
| 1330 """ | |
| 1331 if ext_data is None: | |
| 1332 ext_data = {} | |
| 1333 return self.dbpool.runInteraction(self._getItemsCount, authorized_groups, unrestricted, ext_data) | |
| 1334 | |
| 1335 def _getItemsCount(self, cursor, authorized_groups, unrestricted, ext_data): | |
| 1336 self._checkNodeExists(cursor) | |
| 1337 args = [] | |
| 1338 | |
| 1339 # SELECT | |
| 1340 query = ["SELECT count(1)"] | |
| 1341 | |
| 1342 self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
| 1343 | |
| 1344 cursor.execute(' '.join(query), args) | |
| 1345 return cursor.fetchall()[0][0] | |
| 1346 | |
| 1347 def getItemsIndex(self, item_id, authorized_groups, unrestricted, ext_data=None): | |
| 1348 """Get expected index of first item in the window of a getItems query | |
| 1349 | |
| 1350 @param item_id: id of the item | |
| 1351 @param authorized_groups: we want to get items that these groups can access | |
| 1352 @param unrestricted: if true, don't check permissions (i.e.: get all items) | |
| 1353 @param ext_data: options for extra features like RSM and MAM | |
| 1354 """ | |
| 1355 if ext_data is None: | |
| 1356 ext_data = {} | |
| 1357 return self.dbpool.runInteraction(self._getItemsIndex, item_id, authorized_groups, unrestricted, ext_data) | |
| 1358 | |
| 1359 def _getItemsIndex(self, cursor, item_id, authorized_groups, unrestricted, ext_data): | |
| 1360 self._checkNodeExists(cursor) | |
| 1361 args = [] | |
| 1362 | |
| 1363 # SELECT | |
| 1364 query = [] | |
| 1365 | |
| 1366 query_order = self._appendSourcesAndFilters(query, args, authorized_groups, unrestricted, ext_data) | |
| 1367 | |
| 1368 query_select = "SELECT row_number from (SELECT row_number() OVER ({}), item".format(query_order) | |
| 1369 query.insert(0, query_select) | |
| 1370 query.append(") as x WHERE item=%s") | |
| 1371 args.append(item_id) | |
| 1372 | |
| 1373 cursor.execute(' '.join(query), args) | |
| 1374 # XXX: row_number start at 1, but we want that index start at 0 | |
|
324
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1375 try: |
|
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1376 return cursor.fetchall()[0][0] - 1 |
|
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1377 except IndexError: |
|
e73e42b4f6ff
pgsql storage: getItemsIndex raise NodeNotFound in this make sense
Goffi <goffi@goffi.org>
parents:
323
diff
changeset
|
1378 raise error.NodeNotFound() |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1379 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1380 def getItemsPublishers(self, itemIdentifiers): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1381 """Get the publishers for all given identifiers |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1382 |
|
308
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1383 @return (dict[unicode, jid.JID]): map of itemIdentifiers to publisher |
|
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1384 if item is not found, key is skipped in resulting dict |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1385 """ |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1386 return self.dbpool.runInteraction(self._getItemsPublishers, itemIdentifiers) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1387 |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1388 def _getItemsPublishers(self, cursor, itemIdentifiers): |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1389 self._checkNodeExists(cursor) |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1390 ret = {} |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1391 for itemIdentifier in itemIdentifiers: |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1392 cursor.execute("""SELECT publisher FROM items |
|
390
fba96e95e329
storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents:
384
diff
changeset
|
1393 WHERE node_id=%s AND item=%s""", |
|
fba96e95e329
storage (pgsql): fixed filtering in getItemsPublishers
Goffi <goffi@goffi.org>
parents:
384
diff
changeset
|
1394 (self.nodeDbId, itemIdentifier,)) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1395 result = cursor.fetchone() |
|
308
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1396 if result: |
|
087b705493a6
fixed publisher check on item publishing
Goffi <goffi@goffi.org>
parents:
301
diff
changeset
|
1397 ret[itemIdentifier] = jid.JID(result[0]) |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1398 return ret |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1399 |
| 107 | 1400 def purge(self): |
|
204
b4bf0a5ce50d
Implement storage facilities for the HTTP gateway.
Ralph Meijer <ralphm@ik.nu>
parents:
202
diff
changeset
|
1401 return self.dbpool.runInteraction(self._purge) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1402 |
|
127
d3689da18ed2
Don't use encode('utf-8') on serialized XML.
Ralph Meijer <ralphm@ik.nu>
parents:
124
diff
changeset
|
1403 def _purge(self, cursor): |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1404 self._checkNodeExists(cursor) |
|
121
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1405 |
|
4f0113adb7ed
Add Node._check_node_exists() calls to all Node methods, because nodes could
Ralph Meijer <ralphm@ik.nu>
parents:
113
diff
changeset
|
1406 cursor.execute("""DELETE FROM items WHERE |
|
294
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1407 node_id=%s""", |
|
df1edebb0466
PEP implementation, draft (huge patch sorry):
Goffi <goffi@goffi.org>
parents:
283
diff
changeset
|
1408 (self.nodeDbId,)) |
|
198
e404775b12df
Change naming and spacing conventions to match Twisted's.
Ralph Meijer <ralphm@ik.nu>
parents:
181
diff
changeset
|
1409 |
|
263
9dfd3890e646
added the constant FLAG_RETRACT_ALLOW_PUBLISHER to allow a publisher to retract an item he has published in a node of "open" publish model.
souliane <souliane@mailoo.org>
parents:
262
diff
changeset
|
1410 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
1411 class CollectionNode(Node): |
|
146
b4490bdc77e5
Change semantics of Node.is_subscribed() to match all subscriptions for an
Ralph Meijer <ralphm@ik.nu>
parents:
145
diff
changeset
|
1412 |
|
206
274a45d2a5ab
Implement root collection that includes all leaf nodes.
Ralph Meijer <ralphm@ik.nu>
parents:
204
diff
changeset
|
1413 nodeType = 'collection' |