libervia-backend: sat/plugins/plugin_misc_uri

annotate sat/plugins/plugin_misc_uri_finder.py @ 3914:4cb38c8312a1

plugin XEP-0384, xml_tools: avoid `getItems` timeout + fix empty node crash + parsing: - use `max_items` in `getItems` calls for bundles, as otherwise some pubsub service may return full nodes, which may be huge is `max_items=1` is not set on the node, possibly resulting in timeouts. - the plugin was crashing when TWOMEMO devices list node has no items at all. This is not the case anymore. - a naive parsing method has been implemented in `xml_tools` to replace the serialisation/deserialisation method. This should be more efficient and will avoid annoying `ns0:` prefixes in XML logs.

author	Goffi <goffi@goffi.org>
date	Sat, 24 Sep 2022 16:37:46 +0200
parents	be6d91572633
children	524856bd7b19

rev	line source
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	1 #!/usr/bin/env python3
3137 559a625a236b fixed shebangs Goffi <goffi@goffi.org> parents: 3136 diff changeset	2
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	3
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	4 # SAT plugin to find URIs
3479 be6d91572633 date update Goffi <goffi@goffi.org> parents: 3137 diff changeset	5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	6
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	10 # (at your option) any later version.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	11
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	16
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	19
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	20 from sat.core.i18n import _
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	21 from sat.core.constants import Const as C
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	22 from sat.core.log import getLogger
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	23 from twisted.internet import defer
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	24 import textwrap
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	25 log = getLogger(__name__)
2554 0062d3e79d12 plugin uri finder, jp (merge-request): labels handling: Goffi <goffi@goffi.org> parents: 2548 diff changeset	26 import json
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	27 import os.path
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	28 import os
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	29 import re
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	30
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	31 PLUGIN_INFO = {
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	32 C.PI_NAME: _("URI finder"),
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	33 C.PI_IMPORT_NAME: "uri_finder",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	34 C.PI_TYPE: "EXP",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	35 C.PI_PROTOCOLS: [],
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	36 C.PI_DEPENDENCIES: [],
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	37 C.PI_MAIN: "URIFinder",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	38 C.PI_HANDLER: "no",
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	39 C.PI_DESCRIPTION: textwrap.dedent(_("""\
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	40 Plugin to find URIs in well know location.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	41 This allows to retrieve settings to work with a project (e.g. pubsub node used for merge-requests).
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	42 """))
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	43 }
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	44
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	45
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	46 SEARCH_FILES = ('readme', 'contributing')
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	47
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	48
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	49 class URIFinder(object):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	50
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	51 def __init__(self, host):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	52 log.info(_("URI finder plugin initialization"))
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	53 self.host = host
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	54 host.bridge.addMethod("URIFind", ".plugin",
2554 0062d3e79d12 plugin uri finder, jp (merge-request): labels handling: Goffi <goffi@goffi.org> parents: 2548 diff changeset	55 in_sign='sas', out_sign='a{sa{ss}}',
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	56 method=self.find,
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	57 async_=True)
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	58
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	59 def find(self, path, keys):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	60 """Look for URI in well known locations
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	61
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	62 @param path(unicode): path to start with
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	63 @param keys(list[unicode]): keys lookeds after
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	64 e.g.: "tickets", "merge-requests"
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	65 @return (dict[unicode, unicode]): map from key to found uri
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	66 """
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	67 keys_re = '\|'.join(keys)
2554 0062d3e79d12 plugin uri finder, jp (merge-request): labels handling: Goffi <goffi@goffi.org> parents: 2548 diff changeset	68 label_re = r'"(?P<label>[^"]+)"'
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	69 uri_re = re.compile(r'(?P<key>{keys_re})[ :]? +(?P<uri>xmpp:\S+)(?:.*use {label_re} label)?'.format(
2554 0062d3e79d12 plugin uri finder, jp (merge-request): labels handling: Goffi <goffi@goffi.org> parents: 2548 diff changeset	70 keys_re=keys_re, label_re = label_re))
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	71 path = os.path.normpath(path)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	72 if not os.path.isdir(path) or not os.path.isabs(path):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	73 raise ValueError('path must be an absolute path to a directory')
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	74
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	75 found_uris = {}
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	76 while path != '/':
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	77 for filename in os.listdir(path):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	78 name, __ = os.path.splitext(filename)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	79 if name.lower() in SEARCH_FILES:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	80 file_path = os.path.join(path, filename)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	81 with open(file_path) as f:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	82 for m in uri_re.finditer(f.read()):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	83 key = m.group('key')
ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	84 uri = m.group('uri')
ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	85 label = m.group('label')
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	86 if key in found_uris:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	87 log.warning(_("Ignoring already found uri for key \"{key}\"").format(key=key))
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	88 else:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	89 uri_data = found_uris[key] = {'uri': uri}
2554 0062d3e79d12 plugin uri finder, jp (merge-request): labels handling: Goffi <goffi@goffi.org> parents: 2548 diff changeset	90 if label is not None:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	91 uri_data['labels'] = json.dumps([label])
2548 54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	92 if found_uris:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	93 break
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	94 path = os.path.dirname(path)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	95
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations Goffi <goffi@goffi.org> parents: diff changeset	96 return defer.succeed(found_uris)

Mercurial > libervia-backend

annotate sat/plugins/plugin_misc_uri_finder.py @ 3914:4cb38c8312a1