Mercurial > libervia-backend

annotate libervia/backend/plugins/plugin_misc_jid_search.py @ 4219:1b5cf2ee1d86

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
plugin XEP-0384, XEP-0391: download missing devices list: when a peer jid was not in our roster, devices list was not retrieved, resulting in failed en/decryption. This patch does check it and download missing devices list in necessary. There is no subscription managed yet, so the list won't be updated in case of new devices, this should be addressed at some point.
author Goffi <goffi@goffi.org>
date Tue, 05 Mar 2024 17:31:36 +0100
parents 238e305f2306
children 0d7bb4df2343
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4108
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
2
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia plugin to handle XMPP entities search
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
5
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
10
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
15
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
18
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
19 from collections import OrderedDict
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
20 from dataclasses import dataclass, asdict
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
21 import difflib
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from typing import List, Optional
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
23
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
24 from twisted.internet import defer
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
25 from twisted.words.protocols.jabber import jid
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
26
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
27 from libervia.backend.core.constants import Const as C
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
28 from libervia.backend.core.core_types import SatXMPPEntity
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
29 from libervia.backend.core.i18n import _
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
30 from libervia.backend.core.log import getLogger
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
31 from libervia.backend.tools.common import data_format
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
32
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
33 log = getLogger(__name__)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
34
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
35
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
36 PLUGIN_INFO = {
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
37 C.PI_NAME: "JID Search",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
38 C.PI_IMPORT_NAME: "JID_SEARCH",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
39 C.PI_TYPE: C.PLUG_TYPE_MISC,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
40 C.PI_MODES: C.PLUG_MODE_BOTH,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
41 C.PI_PROTOCOLS: [],
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
42 C.PI_DEPENDENCIES: [],
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
43 C.PI_RECOMMENDATIONS: [],
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
44 C.PI_MAIN: "JidSearch",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
45 C.PI_HANDLER: "no",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
46 C.PI_DESCRIPTION: _("""Search for XMPP entities"""),
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
47 }
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
48 RATIO_CUTOFF = 0.6
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
49 MAX_CACHE_SIZE = 10
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
50
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
51
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
52 @dataclass
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
53 class JidSearchItem:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
54 entity: jid.JID
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
55 name: str = ""
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
56 in_roster: bool = False
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
57 groups: list[str] | None = None
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
58 exact_match: bool = False
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
59 relevance: float | None = None
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
60
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
61
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
62 JidSearchCache = OrderedDict[str, list[JidSearchItem]]
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
63
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
64
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
65 class JidSearch:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
66 def __init__(self, host) -> None:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
67 log.info(f"plugin {PLUGIN_INFO[C.PI_NAME]!r} initialization")
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
68 self.host = host
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
69 host.bridge.add_method(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
70 "jid_search",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
71 ".plugin",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
72 in_sign="sss",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
73 out_sign="s",
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
74 method=self._search,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
75 async_=True,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
76 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
77
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
78 def profile_connecting(self, client: SatXMPPEntity) -> None:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
79 client._jid_search_cache = JidSearchCache()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
80
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
81 def _search(self, search_term: str, options_s: str, profile: str) -> defer.Deferred:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
82 client = self.host.get_client(profile)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
83 d = defer.ensureDeferred(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
84 self.search(client, search_term, data_format.deserialise(options_s))
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
85 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
86 d.addCallback(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
87 lambda search_items: data_format.serialise([asdict(i) for i in search_items])
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
88 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
89 return d
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
90
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
91 async def search(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
92 self, client: SatXMPPEntity, search_term: str, options: Optional[dict] = None
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
93 ) -> List[JidSearchItem]:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
94 """Searches for entities in various locations.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
95
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
96 @param client: The SatXMPPEntity client where the search is to be performed.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
97 @param search_term: The query to be searched.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
98 @param options: Additional search options.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
99 @return: A list of matches found.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
100 """
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
101 search_term = search_term.strip().lower()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
102 sequence_matcher = difflib.SequenceMatcher()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
103 sequence_matcher.set_seq1(search_term)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
104 # FIXME: cache can give different results due to the filtering mechanism (if a
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
105 # cached search term match the beginning of current search term, its results a
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
106 # re-used and filtered, and sometimes items can be missing in compraison to the
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
107 # results without caching). This may need to be fixed.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
108 cache: JidSearchCache = client._jid_search_cache
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
109
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
110 # Look for a match in the cache
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
111 for cache_key in cache:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
112 if search_term.startswith(cache_key):
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
113 log.debug(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
114 f"Match found in cache for {search_term!r} in [{client.profile}]."
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
115 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
116 # If an exact match is found, return the results as is
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
117 if search_term == cache_key:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
118 log.debug("Exact match found in cache, reusing results.")
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
119 matches = cache[cache_key]
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
120 else:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
121 # If only the beginning matches, filter the cache results
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
122 log.debug("Prefix match found in cache, filtering results.")
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
123 matches = []
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
124 for jid_search_item in cache[cache_key]:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
125 self._process_matching(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
126 search_term, sequence_matcher, matches, jid_search_item
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
127 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
128 cache.move_to_end(cache_key)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
129 break
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
130 else:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
131 # If no match is found in the cache, perform a new search
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
132 matches = await self._perform_search(client, search_term, sequence_matcher)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
133 cache[search_term] = matches
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
134 if len(cache) > MAX_CACHE_SIZE:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
135 cache.popitem(last=False)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
136
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
137 # If no exact match is found, but the search term is a valid JID, we add the JID
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
138 # as a result
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
139 exact_match = any(m.exact_match for m in matches)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
140 if not exact_match and "@" in search_term:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
141 try:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
142 search_jid = jid.JID(search_term)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
143 except jid.InvalidFormat:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
144 pass
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
145 else:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
146 matches.append(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
147 JidSearchItem(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
148 entity=search_jid,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
149 in_roster=False,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
150 exact_match=True,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
151 relevance=1,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
152 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
153 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
154
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
155
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
156 matches.sort(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
157 key=lambda item: (item.exact_match, item.relevance or 0, item.in_roster),
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
158 reverse=True,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
159 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
160
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
161 return matches
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
162
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
163 def _process_matching(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
164 self,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
165 search_term: str,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
166 sequence_matcher: difflib.SequenceMatcher,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
167 matches: List[JidSearchItem],
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
168 item: JidSearchItem,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
169 ) -> None:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
170 """Process matching of items
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
171
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
172 @param sequence_matcher: The sequence matcher to be used for the matching process.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
173 @param matches: A list where the match is to be appended.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
174 @param item: The item that to be matched.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
175 @return: True if it was an exact match
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
176 """
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
177
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
178 item_name_lower = item.name.lower()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
179 item_entity_lower = item.entity.full().lower()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
180
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
181 if search_term in (item_name_lower, item_entity_lower):
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
182 item.exact_match = True
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
183 item.relevance = 1
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
184 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
185 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
186
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
187 item.exact_match = False
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
188
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
189 sequence_matcher.set_seq2(item_name_lower)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
190 name_ratio = sequence_matcher.ratio()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
191 if name_ratio >= RATIO_CUTOFF:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
192 item.relevance = name_ratio
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
193 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
194 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
195
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
196 sequence_matcher.set_seq2(item_entity_lower)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
197 jid_ratio = sequence_matcher.ratio()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
198 if jid_ratio >= RATIO_CUTOFF:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
199 item.relevance = jid_ratio
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
200 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
201 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
202
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
203 localpart = item.entity.user.lower() if item.entity.user else ""
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
204 if localpart:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
205 sequence_matcher.set_seq2(localpart)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
206 domain_ratio = sequence_matcher.ratio()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
207 if domain_ratio >= RATIO_CUTOFF:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
208 item.relevance = domain_ratio
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
209 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
210 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
211
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
212 if item.groups:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
213 group_ratios = []
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
214 for group in item.groups:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
215 sequence_matcher.set_seq2(group.lower())
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
216 group_ratios.append(sequence_matcher.ratio())
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
217 group_ratio = max(group_ratios)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
218 if group_ratio >= RATIO_CUTOFF:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
219 item.relevance = group_ratio
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
220 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
221 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
222
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
223 domain = item.entity.host.lower()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
224 sequence_matcher.set_seq2(domain)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
225 domain_ratio = sequence_matcher.ratio()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
226 if domain_ratio >= RATIO_CUTOFF:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
227 item.relevance = domain_ratio
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
228 matches.append(item)
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
229 return
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
230
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
231 async def _perform_search(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
232 self,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
233 client: SatXMPPEntity,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
234 search_term: str,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
235 sequence_matcher: difflib.SequenceMatcher,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
236 ) -> List[JidSearchItem]:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
237 """Performs a new search when no match is found in the cache.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
238
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
239 @param search_term: The query to be searched.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
240 @param sequence_matcher: The SequenceMatcher object to be used for matching.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
241 @return: A list of matches found.
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
242 """
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
243 matches = []
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
244
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
245 try:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
246 roster = client.roster
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
247 except AttributeError:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
248 # components have no roster
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
249 roster = []
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
250 else:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
251 roster = client.roster.get_items()
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
252
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
253 for roster_item in roster:
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
254 jid_search_item = JidSearchItem(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
255 entity=roster_item.entity,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
256 name=roster_item.name,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
257 in_roster=True,
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
258 groups=list(roster_item.groups),
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
259 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
260
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
261 self._process_matching(
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
262 search_term, sequence_matcher, matches, jid_search_item
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
263 )
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
264
238e305f2306 plugin JID Search: JID search plugin, first draft
Goffi <goffi@goffi.org>
parents:
diff changeset
265 return matches