annotate sat/plugins/plugin_misc_uri_finder.py @ 3028:ab2696e34d29

Python 3 port: /!\ this is a huge commit /!\ starting from this commit, SàT is needs Python 3.6+ /!\ SàT maybe be instable or some feature may not work anymore, this will improve with time This patch port backend, bridge and frontends to Python 3. Roughly this has been done this way: - 2to3 tools has been applied (with python 3.7) - all references to python2 have been replaced with python3 (notably shebangs) - fixed files not handled by 2to3 (notably the shell script) - several manual fixes - fixed issues reported by Python 3 that where not handled in Python 2 - replaced "async" with "async_" when needed (it's a reserved word from Python 3.7) - replaced zope's "implements" with @implementer decorator - temporary hack to handle data pickled in database, as str or bytes may be returned, to be checked later - fixed hash comparison for password - removed some code which is not needed anymore with Python 3 - deactivated some code which needs to be checked (notably certificate validation) - tested with jp, fixed reported issues until some basic commands worked - ported Primitivus (after porting dependencies like urwid satext) - more manual fixes
author Goffi <goffi@goffi.org>
date Tue, 13 Aug 2019 19:08:41 +0200
parents 003b8b4b56a7
children 9d0df638c8b4
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
1 #!/usr/bin/env python3
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
3
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # SAT plugin to find URIs
2771
003b8b4b56a7 date update
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
5 # Copyright (C) 2009-2019 Jérôme Poisson (goffi@goffi.org)
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
6
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
10 # (at your option) any later version.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
11
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
16
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
19
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
20 from sat.core.i18n import _
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
21 from sat.core.constants import Const as C
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from sat.core.log import getLogger
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from twisted.internet import defer
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
24 import textwrap
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
25 log = getLogger(__name__)
2554
0062d3e79d12 plugin uri finder, jp (merge-request): labels handling:
Goffi <goffi@goffi.org>
parents: 2548
diff changeset
26 import json
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
27 import os.path
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
28 import os
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
29 import re
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
30
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
31 PLUGIN_INFO = {
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
32 C.PI_NAME: _("URI finder"),
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
33 C.PI_IMPORT_NAME: "uri_finder",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
34 C.PI_TYPE: "EXP",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
35 C.PI_PROTOCOLS: [],
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
36 C.PI_DEPENDENCIES: [],
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
37 C.PI_MAIN: "URIFinder",
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
38 C.PI_HANDLER: "no",
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
39 C.PI_DESCRIPTION: textwrap.dedent(_("""\
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
40 Plugin to find URIs in well know location.
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
41 This allows to retrieve settings to work with a project (e.g. pubsub node used for merge-requests).
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
42 """))
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
43 }
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
44
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
45
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
46 SEARCH_FILES = ('readme', 'contributing')
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
47
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
48
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
49 class URIFinder(object):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
50
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
51 def __init__(self, host):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
52 log.info(_("URI finder plugin initialization"))
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
53 self.host = host
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
54 host.bridge.addMethod("URIFind", ".plugin",
2554
0062d3e79d12 plugin uri finder, jp (merge-request): labels handling:
Goffi <goffi@goffi.org>
parents: 2548
diff changeset
55 in_sign='sas', out_sign='a{sa{ss}}',
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
56 method=self.find,
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
57 async_=True)
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
58
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
59 def find(self, path, keys):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
60 """Look for URI in well known locations
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
61
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
62 @param path(unicode): path to start with
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
63 @param keys(list[unicode]): keys lookeds after
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
64 e.g.: "tickets", "merge-requests"
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
65 @return (dict[unicode, unicode]): map from key to found uri
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
66 """
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
67 keys_re = '|'.join(keys)
2554
0062d3e79d12 plugin uri finder, jp (merge-request): labels handling:
Goffi <goffi@goffi.org>
parents: 2548
diff changeset
68 label_re = r'"(?P<label>[^"]+)"'
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
69 uri_re = re.compile(r'(?P<key>{keys_re})[ :]? +(?P<uri>xmpp:\S+)(?:.*use {label_re} label)?'.format(
2554
0062d3e79d12 plugin uri finder, jp (merge-request): labels handling:
Goffi <goffi@goffi.org>
parents: 2548
diff changeset
70 keys_re=keys_re, label_re = label_re))
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
71 path = os.path.normpath(path)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
72 if not os.path.isdir(path) or not os.path.isabs(path):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
73 raise ValueError('path must be an absolute path to a directory')
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
74
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
75 found_uris = {}
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
76 while path != '/':
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
77 for filename in os.listdir(path):
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
78 name, __ = os.path.splitext(filename)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
79 if name.lower() in SEARCH_FILES:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
80 file_path = os.path.join(path, filename)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
81 with open(file_path) as f:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
82 for m in uri_re.finditer(f.read()):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
83 key = m.group('key')
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
84 uri = m.group('uri')
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
85 label = m.group('label')
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
86 if key in found_uris:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
87 log.warning(_("Ignoring already found uri for key \"{key}\"").format(key=key))
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
88 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
89 uri_data = found_uris[key] = {'uri': uri}
2554
0062d3e79d12 plugin uri finder, jp (merge-request): labels handling:
Goffi <goffi@goffi.org>
parents: 2548
diff changeset
90 if label is not None:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
91 uri_data['labels'] = json.dumps([label])
2548
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
92 if found_uris:
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
93 break
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
94 path = os.path.dirname(path)
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
95
54b3853b55c0 plugin uri finder: plugin to find URIs in well-known locations
Goffi <goffi@goffi.org>
parents:
diff changeset
96 return defer.succeed(found_uris)