Mercurial > libervia-pubsub

diff sat_pubsub/backend.py @ 430:5a0ada3b61ca

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Full-Text Search implementation: /!\ pgsql schema needs to be updated /!\ /!\ Minimal PostgreSQL required version is now 12 /!\ A new options is available to specify main language of a node. By default a `generic` language is used (which uses the `simple` configuration in PostgreSQL). When a node owner changes the language, the index is rebuilt accordingly. It is possible to have item specific language for multilingual nodes (but for the moment the search is done with node language, so the results won't be good). If an item language is explicitely set in `item_languages`, the FTS configuration won't be affected by node FTS language setting. Search is parsed with `websearch_to_tsquery` for now, but this parser doesn't handle prefix matching, so it may be replaced in the future. SetConfiguration now only updates the modified values, this avoid triggering the FTS re-indexing on each config change. `_checkNodeExists` is not called anymore as we can check if a row has been modified to see if the node exists, this avoid a useless query. Item storing has been slighly improved with a useless SELECT and condition removed. To avoid 2 schema updates in a row, the `sat_pubsub_update_5_6.sql` file also prepares the implementation of XEP-0346 by updating nodes with a schema and creating the suitable template nodes.
author Goffi <goffi@goffi.org>
date Fri, 11 Dec 2020 17:18:52 +0100
parents c21f31355ab9
children 5e8b8ef5c862
line wrap: on
line diff
--- a/sat_pubsub/backend.py	Thu Dec 10 10:46:34 2020 +0100
+++ b/sat_pubsub/backend.py	Fri Dec 11 17:18:52 2020 +0100
@@ -83,6 +83,9 @@
 from sat_pubsub import const
 from sat_pubsub import container
 
+# TODO: modernise the code, get rid of legacy Twisted logging, use coroutines,
+#   better formatting
+
 
 def _getAffiliation(node, entity):
     d = node.getAffiliation(entity)
@@ -180,6 +183,15 @@
             const.OPT_CONSISTENT_PUBLISHER:
                 {"type": "boolean",
                  "label": "Keep publisher on update"},
+            const.OPT_FTS_LANGUAGE:
+                {"type": "list-single",
+                 "label": "Default language to use for full text search",
+                 "options": {
+                     const.VAL_FTS_GENERIC: (
+                         "Generic (use if unsure of if your language is not present)"
+                     )
+                 }
+                },
             }
 
     subscriptionOptions = {
@@ -203,6 +215,16 @@
         self._callbackList = []
         self.config = config
         self.admins = config['admins_jids_list']
+        d = self.storage.getFTSLanguages()
+        d.addCallbacks(self._getFTSLanguagesCb, self._getFTSLanguagesEb)
+
+    def _getFTSLanguagesCb(self, languages):
+        # we skip the first one which is always "generic", as we already have it
+        for l in languages[1:]:
+            self.nodeOptions[const.OPT_FTS_LANGUAGE]["options"][l] = l.title()
+
+    def _getFTSLanguagesEb(self, failure_):
+        log.msg(f"WARNING: can get FTS languages: {failure_}")
 
     def isAdmin(self, entity_jid):
         """Return True if an entity is an administrator"""
@@ -724,7 +746,7 @@
     def setNodeSchema(self, nodeIdentifier, schema, requestor, pep, recipient):
         """set or remove Schema of a node
 
-        @param nodeIdentifier(unicode): identifier of the pubusb node
+        @param nodeIdentifier(unicode): identifier of the pubsub node
         @param schema(domish.Element, None): schema to set
             None to remove schema
         @param requestor(jid.JID): entity doing the request