Mercurial > libervia-backend

annotate doc/libervia-cli/pubsub_cache.rst @ 3884:cea52400623d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
component AP gateway: work around encoding bug in Mastodon: Mastodon in wrongly unquoting URL path in `(request-target)`, and thus Libervia was doing the same to check signature. However that doesn't work with Pleroma which is using the path value used in the request (percent-encoded), and thus Pleroma signature were rejected. To work around that, signature is first checked without unquoting, and if this fails a new check is done with unquoting. Bug has been reported at https://github.com/mastodon/mastodon/issues/18871 rel 371
author Goffi <goffi@goffi.org>
date Wed, 31 Aug 2022 17:07:03 +0200
parents d0b66efc6c0e
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3612
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
1 .. _libervia-cli_pubsub_cache:
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
2
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
3 =====================================
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
4 pubsub/cache: PubSub Cache Management
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
5 =====================================
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
6
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
7 Libervia runs transparently a cache for pubsub. That means that according to internal
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
8 criteria, some pubsub items are stored locally.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
9
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
10 The ``cache`` subcommands let user inspect and manipulate the internal cache.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
11
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
12 get
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
13 ===
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
14
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
15 Retrieve items from internal cache only. Most end-users won't need to use this command, as
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
16 the usual ``pubsub get`` command will use cache transparently. However, it may be useful
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
17 to inspect local cache, notably for debugging.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
18
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
19 The parameters are basically the same as for :ref:`li_pubsub_get`.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
20
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
21 example
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
22 -------
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
23
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
24 Retrieve the last 2 cached items for personal blog::
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
25
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
26 $ li pubsub cache get -n urn:xmpp:microblog:0 -M 2
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
27
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
28 .. _li_pubsub_cache_sync:
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
29
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
30 sync
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
31 ====
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
32
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
33 Synchronise or resynchronise a pubsub node. If the node is already in cache, it will be
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
34 deleted then re-cached. Node will be put in cache even if internal policy doesn't request
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
35 a synchronisation for this kind of nodes. Node will be (re-)subscribed to keep cache
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
36 synchronised.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
37
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
38 All items of the node (up to the internal limit which is high), will be retrieved and put
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
39 in cache, even if a previous version of those items have been deleted by the
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
40 :ref:`li_pubsub_cache_purge` command.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
41
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
42
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
43 example
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
44 -------
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
45
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
46 Resynchronise personal blog::
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
47
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
48 $ li pubusb cache sync -n urn:xmpp:microblog:0
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
49
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
50 .. _li_pubsub_cache_purge:
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
51
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
52 purge
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
53 =====
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
54
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
55 Remove items from cache. This may be desirable to save resource, notably disk space.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
56
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
57 Note that once a pubsub node is cached, the cache is the source of trust. That means that
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
58 if cache is not explicitly bypassed when retrieving items of a pubsub node (notably with
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
59 the ``-C, --no-cache`` option of :ref:`li_pubsub_get`), only items found in cache will be
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
60 returned, thus purged items won't be used or returned anymore even if they still exists on
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
61 the original pubsub service.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
62
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
63 If you have purged items by mistake, it is possible to retrieve them either node by node
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
64 using :ref:`li_pubsub_cache_sync`, or by resetting the whole pubsub cache with
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
65 :ref:`li_pubsub_cache_reset`.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
66
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
67 If you have a node or a profile (e.g. a component) caching a lot of items frequently, you
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
68 may use this command using a scheduler like cron_.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
69
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
70 .. _cron: https://en.wikipedia.org/wiki/Cron
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
71
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
72 examples
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
73 --------
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
74
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
75 Remove all blog and event items from cache if they haven't been updated since 6 months::
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
76
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
77 $ li pubsub cache purge -t blog -t event -b "6 months ago"
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
78
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
79 Remove items from profile ``ap_gateway`` if they have been created more that 2 months
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
80 ago::
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
81
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
82 $ li pubsub cache purge -p ap_gateway --created-before "2 months ago"
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
83
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
84 .. _li_pubsub_cache_reset:
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
85
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
86 reset
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
87 =====
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
88
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
89 Reset the whole pubsub cache. This means that all nodes and all them items will be removed
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
90 from cache. After this command, cache will be re-filled progressively as if it where a new
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
91 one.
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
92
3669
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
93 .. note::
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
94
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
95 Use this command with caution: even if cache will be re-constructed with time, that
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
96 means that items will have to be retrieved again, that may be resource intensive both
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
97 for your machine and for the pubsub services which will be used. That also means that
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
98 searching items will return less results until all desired items are cached again.
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
99
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
100 Also note that all items of cached nodes are retrieved, even if you have previously
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
101 purged items, they will be retrieved again.
23be54db81f1 doc (cli/pubsub_cache): add a warning for `reset`
Goffi <goffi@goffi.org>
parents: 3612
diff changeset
102
3612
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
103 example
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
104 -------
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
105
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
106 Reset the whole pubsub cache::
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
107
21e7d46b988c doc (cli/pubsub): document pubsub cache management commands
Goffi <goffi@goffi.org>
parents:
diff changeset
108 $ li pubsub cache reset
3670
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
109
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
110 search
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
111 ======
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
112
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
113 Search items into pubsub cache. The search is done on the whole cache, it's not restricted
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
114 to a single node/profile (even if it may be if suitable filters are specified). Full-Text
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
115 Search can be done with ``-f FTS, --fts FTS`` argument, as well as filtering on parsed
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
116 data (with ``-F PATH OPERATOR VALUE, --field PATH OPERATOR VALUE``, see below).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
117
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
118 By default, parsed data are returned, with the 3 additional keys ``pubsub_service``,
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
119 ``pubsub_items`` (the search being done on the whole cache, those data are here to get the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
120 full location of each item) and ``node_profile``.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
121
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
122 "Parsed data" are the result of the parsing of the items XML payload by feature aware
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
123 plugins. Those data are usually more readable and easier to work with. Parsed data are
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
124 only stored when a parser is registered for a specific feature, that means that a Pubsub
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
125 item in cache may not have parsed data at all, in which case an empty dict will be used
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
126 instead (and ``-P, --payload`` argument should be used to get content of the item).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
127
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
128 The dates are normally stored as `Unix time`_ in database, but the default output convert
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
129 the ``updated``, ``created`` and ``published`` fields to human readable local time. Use
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
130 ``--output simple`` if you want to keep the float (or int) value.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
131
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
132 XML item payload is not returned by default, but it can be added to the ``item_payload``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
133 field if ``-P, --payload`` argument is set. You can also use the ``--output xml`` (or
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
134 ``xml_raw`` if you don't want prettifying) to output directly the highlighted XML
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
135 — without the parsed data —, to have an output similar to the one of ``li pubsub get``.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
136
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
137 If you are interested only in a specific data (e.g. item id and title), the ``-k KEY,
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
138 --key KEY`` can be used.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
139
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
140 You'll probably want to limit result size by using ``-l LIMIT, --limit LIMIT``, and do
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
141 pagination using ``-i INDEX, --index INDEX``.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
142
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
143 .. _Unix time: https://en.wikipedia.org/wiki/Unix_time
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
144
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
145 Filters
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
146 -------
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
147
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
148 By default search returns all items in cache, you have to use filter to specify what you
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
149 are looking after. We can split filters in 3 categories: nodes/items metadata,
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
150 Full-Text Search query and parsed metadata.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
151
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
152 Nodes/items metadata are the generic information you have on a node: which profile it
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
153 belong too, which pubsub service it's coming from, what's the name or type of the node,
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
154 etc.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
155
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
156 Arguments there should be self-explanatory. Type (set with ``-t TYPE, --type TYPE``) and
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
157 subtype (set with ``-S SUBTYPE, --subtype SUBTYPE``) are values dependent of the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
158 plugin/feature associated with the node, so we can't list them in an exhaustive way here.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
159 The most common type is probably ``blog``, from which a subtype can be ``comment``. An
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
160 empty string can be used to find items with (sub)type not set.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
161
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
162 It's usually a good idea to specify a profile with ``-p PROFILE, --profile PROFILE``,
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
163 otherwise you may get duplicated results.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
164
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
165 Full-Text Search
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
166 ----------------
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
167
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
168 You can specify a Full-Text Search query with the ``-f FTS_QUERY, --fts FTS_QUERY``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
169 argument. The engine is currently SQLite FTS5, and you can check its `query syntax`_.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
170 FTS is done on the whole raw XML payload, that means that all data there can be matched
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
171 (including XML tags and attributes).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
172
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
173 FTS queries are indexed, that means that they are fast and efficient.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
174
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
175 .. note::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
176
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
177 Futures version of Libervia will probably include other FTS engines (support for
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
178 PostgreSQL and MySQL/MariaDB is planned). Thus the syntax may vary depending on the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
179 engine, or a common syntax may be implemented for all engines in the future. Keep that
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
180 in mind if you plan to use FTS capabilities in long-term queries, e.g. in scripts.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
181
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
182 .. _query syntax: https://sqlite.org/fts5.html#full_text_query_syntax
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
183
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
184 Parsed Metadata Filters
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
185 -----------------------
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
186
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
187 It is possible to filter on any field of parsed data. This is done with the ``-F PATH
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
188 OPERATOR VALUE, --field PATH OPERATOR VALUE`` (be careful that the short option is an
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
189 uppercase ``F``, the lower case one being used for Full-Text Search).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
190
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
191 .. note::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
192
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
193 Parsed Metadata Filters are not indexed, that means that using them is less efficient
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
194 than using e.g. Full-Text Search. If you want to filter on a text field, it's often a
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
195 good idea to pre-filter using Full-Text Search to have more efficient queries.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
196
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
197 ``PATH`` and ``VALUE`` can be either specified as string, or using JSON syntax (if the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
198 value can't be decoded as JSON, it is used as plain text).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
199
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
200 ``PATH`` is the name of the field to use. If you must go beyond root level fields, you can
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
201 use a JSON array to specify each element of the path. If a string is used, it's an object
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
202 key, if a number is used it's an array index. Thus you can use ``title`` to access the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
203 root title key, or ``'"title"'`` (JSON string escaped for shell) or ``'["title"]'`` (JSON
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
204 array with the "title" string, escaped for shell).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
205
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
206 .. note::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
207
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
208 The extra fields ``pubsub_service``, ``pubsub_node`` and  ``node_profile`` are added to
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
209 the result after the query, thus they can't be used as fields for filtering (use the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
210 direct arguments for that).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
211
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
212 ``OPERATOR`` indicate how to use the value to make a filter. The currently supported
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
213 operators are:
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
214
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
215 ``==`` or ``eq``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
216 Equality operator, true if field value is the same as given value.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
217
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
218 ``!=`` or ``ne``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
219 Inequality operator, true if the field value is different from given value.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
220
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
221 ``>`` or ``gt``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
222 Greater than, true if the field value is higher than given value. For string, this is
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
223 according to alphabetical order.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
224
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
225 Time Pattern can be used here, see below.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
226
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
227 ``<`` or ``lt``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
228 Lesser than, true if the field value is lower than given value. For string, this is
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
229 according to alphabetical order.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
230
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
231 Time Pattern can be used here, see below.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
232
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
233 ``between``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
234 Given value must be an array with 2 elements. The condition is true if field value is
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
235 between the 2 elements (for string, this is according to alphabetical order).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
236
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
237 Time Pattern can be used here, see below.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
238
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
239 ``in``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
240 Given value must be an array of elements. Field value must be one of them to make the
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
241 condition true.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
242
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
243 ``not_in``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
244 Given value must be an array of elements. Field value must not be any of them the make
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
245 the condition true.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
246
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
247 ``overlap``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
248 This can be used only on array fields.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
249
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
250 If given value is not already an array, it is put in an array. Condition is true if any
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
251 element of field value match any element of given value. Notably useful to filter on
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
252 tags.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
253
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
254 ``ioverlap``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
255 Same as ``overlap`` but done in a case insensitive way.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
256
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
257 ``disjoint``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
258 This can be used only on array fields.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
259
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
260 If given value is not already an array, it is put in an array. Condition is true if no
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
261 element of field value match any element of given value. Notably useful to filter out
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
262 tags.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
263
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
264 ``idisjoint``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
265 Same as ``disjoint`` but done in a case insensitive way.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
266
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
267 ``like``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
268 Does pattern matching on a string. ``%`` can be used to match zero or more characters
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
269 and ``_`` can be used to match any single character.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
270
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
271 If you're not looking after a specific field, it's better to use Full-Text Search when
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
272 possible.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
273
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
274 ``ilike``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
275 Like ``like`` but done in a case insensitive way.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
276
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
277
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
278 ``not_like``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
279 Same as ``like`` except that condition is true when pattern is **not** matching.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
280
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
281 ``not_ilike``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
282 Same as ``not_like`` but done in a case insensitive way.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
283
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
284
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
285 For ``gt``/``>``, ``lt``/``<`` and ``between``, you can use :ref:`time_pattern` by using
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
286 the syntax ``TP(<time pattern>)`` (see examples below).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
287
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
288 Ordering
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
289 --------
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
290
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
291 Result ordering can be done by a well know order, or using a parsed data field. Ordering
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
292 default to ``created`` (see below), but this may be changed with ``-o ORDER [FIELD]
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
293 [DIRECTION], --order-by ORDER [FIELD] [DIRECTION]``.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
294
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
295 ``ORDER`` can be one of the following:
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
296
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
297 ``creation``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
298 Order by item creation date. Note that is this the date of creation of the item in cache
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
299 (which most of time should correspond to order of creation of the item in the source
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
300 pubsub service), and this may differ from the date of publication as specified with some
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
301 feature (like blog). This is important when old items are imported, e.g. when they're
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
302 coming from an other blog engine.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
303
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
304 ``modification``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
305 Order by the date when item has last been modified. Modification date is the same as
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
306 creation date if the item has never been modified since it is in cache. The same warning
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
307 as for ``creation`` applies: this is the date of last modification in cache, not the one
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
308 advertised in parsed data.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
309
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
310 ``item_id``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
311 Order by XMPP id of the item. Notably useful when user-friendly ID are used (like it is
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
312 often the case with blogs).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
313
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
314 ``rank``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
315 Order item by Full-Text Search rank. This one can only be used when Full-Text Search is
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
316 used (via ``-f FTS_QUERY, --fts FTS_QUERY``). Rank is a value indicating how well an
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
317 item match the query. This usually needs to be used with ``desc`` direction, so you get
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
318 most relevant items first.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
319
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
320 ``field``
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
321 This special order indicates that the ordering must be done on an parsed data field. The
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
322 following argument is then the path of the field to used (which can be a plain text name
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
323 of a root field, or a JSON encoded array). An optional direction can be specified as a
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
324 third argument. See examples below.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
325
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
326 examples
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
327 --------
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
328
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
329 Search for blog items cached for the profile ``louise`` which contain the word
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
330 ``Slovakia``::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
331
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
332 $ li pubsub cache search -t blog -p louise -f Slovakia
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
333
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
334 Show title, publication date and id of blog articles (excluding comments) which have been
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
335 published on Louise's blog during the last 6 months, order them by item id. Here we use an
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
336 empty string as a subtype to exclude comments (for which subtype is ``comment``)::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
337
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
338 $ li pubsub cache search -t blog -S "" -p louise -s louise@example.net -n urn:xmpp:microblog:0 -F published gt 'TP(6 months ago)' -k id -k published -k title -o item_id
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
339
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
340 Show all blog items from anywhere which are tagged as XMPP or ActivityPub (case
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
341 insensitive) and which have been published in the last month (according to advertised
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
342 publishing date, not cache creation date).
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
343
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
344 We want to order them by descending publication date (again the advertised publication
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
345 date, not cache creation), and we don't want more than 50 results.
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
346
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
347 We do a FTS query there even if it's not mandatory, because it will do an efficient
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
348 pre-filtering::
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
349
d0b66efc6c0e doc (cli/pubsub_cache): `search` command documentation:
Goffi <goffi@goffi.org>
parents: 3669
diff changeset
350 $ li pubsub cache search -f "xmpp OR activitypub" -F tags ioverlap '["xmpp", "activitypub"]' -F published gt 'TP(1 month ago)' -o field published desc -l 50