annotate sat/tools/utils.py @ 3028:ab2696e34d29

Python 3 port: /!\ this is a huge commit /!\ starting from this commit, SàT is needs Python 3.6+ /!\ SàT maybe be instable or some feature may not work anymore, this will improve with time This patch port backend, bridge and frontends to Python 3. Roughly this has been done this way: - 2to3 tools has been applied (with python 3.7) - all references to python2 have been replaced with python3 (notably shebangs) - fixed files not handled by 2to3 (notably the shell script) - several manual fixes - fixed issues reported by Python 3 that where not handled in Python 2 - replaced "async" with "async_" when needed (it's a reserved word from Python 3.7) - replaced zope's "implements" with @implementer decorator - temporary hack to handle data pickled in database, as str or bytes may be returned, to be checked later - fixed hash comparison for password - removed some code which is not needed anymore with Python 3 - deactivated some code which needs to be checked (notably certificate validation) - tested with jp, fixed reported issues until some basic commands worked - ported Primitivus (after porting dependencies like urwid satext) - more manual fixes
author Goffi <goffi@goffi.org>
date Tue, 13 Aug 2019 19:08:41 +0200
parents 181735d1b062
children d62fceccff22
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
1 #!/usr/bin/env python3
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
3
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
4 # SAT: a jabber client
2771
003b8b4b56a7 date update
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
5 # Copyright (C) 2009-2019 Jérôme Poisson (goffi@goffi.org)
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
6
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
7 # This program is free software: you can redistribute it and/or modify
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
10 # (at your option) any later version.
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
11
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
12 # This program is distributed in the hope that it will be useful,
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
15 # GNU Affero General Public License for more details.
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
16
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 601
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
19
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
20 """ various useful methods """
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
21
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
22 import unicodedata
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
23 import os.path
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
24 import datetime
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
25 import subprocess
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
26 import time
2089
0931b5a6213c core, quick_frontends: android compatibility hacks:
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
27 import sys
2181
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
28 import random
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
29 import inspect
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
30 import textwrap
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
31 import functools
2793
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
32 from twisted.python import procutils
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
33 from sat.core.constants import Const as C
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
34 from sat.core.log import getLogger
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
35
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
36 log = getLogger(__name__)
916
1a759096ccbd core: use of Const for profile_key + replaced '@DEFAULT@' default profile_key by '@NONE@'
Goffi <goffi@goffi.org>
parents: 811
diff changeset
37
1a759096ccbd core: use of Const for profile_key + replaced '@DEFAULT@' default profile_key by '@NONE@'
Goffi <goffi@goffi.org>
parents: 811
diff changeset
38
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
39 NO_REPOS_DATA = "repository data unknown"
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
40 repos_cache_dict = None
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
41 repos_cache = None
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
42
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
43
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
44 def clean_ustr(ustr):
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
45 """Clean unicode string
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
46
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
47 remove special characters from unicode string
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
48 """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
49
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
50 def valid_chars(unicode_source):
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
51 for char in unicode_source:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
52 if unicodedata.category(char) == "Cc" and char != "\n":
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
53 continue
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
54 yield char
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
55
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
56 return "".join(valid_chars(ustr))
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
57
601
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module
Goffi <goffi@goffi.org>
parents:
diff changeset
58
2793
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
59 def logError(failure_):
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
60 """Genertic errback which log the error as a warning, and re-raise it"""
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
61 log.warning(failure_.value)
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
62 raise failure_
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
63
181735d1b062 plugin mr mercurial, tools(common/utils): moved command protocol to a new module
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
64
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
65 def partial(func, *fixed_args, **fixed_kwargs):
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
66 # FIXME: temporary hack to workaround the fact that inspect.getargspec is not working with functools.partial
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
67 # making partial unusable with current D-bus module (in addMethod).
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
68 # Should not be needed anywore once moved to Python 3
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
69
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
70 ori_args = inspect.getargspec(func).args
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
71 func = functools.partial(func, *fixed_args, **fixed_kwargs)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
72 if ori_args[0] == "self":
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
73 del ori_args[0]
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
74 ori_args = ori_args[len(fixed_args) :]
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
75 for kw in fixed_kwargs:
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
76 ori_args.remove(kw)
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
77
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
78 exec(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
79 textwrap.dedent(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
80 """\
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
81 def method({args}):
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
82 return func({kw_args})
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
83 """
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
84 ).format(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
85 args=", ".join(ori_args), kw_args=", ".join([a + "=" + a for a in ori_args])
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
86 ),
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
87 locals(),
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
88 )
2470
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
89
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
90 return method
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module.
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
91
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
92
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
93 def xmpp_date(timestamp=None, with_time=True):
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
94 """Return date according to XEP-0082 specification
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
95
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
96 to avoid reveling the timezone, we always return UTC dates
1935
1128feb54180 core: removed pyfeed and xe dependencies:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
97 the string returned by this method is valid with RFC 3339
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
98 @param timestamp(None, float): posix timestamp. If None current time will be used
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
99 @param with_time(bool): if True include the time
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
100 @return(unicode): XEP-0082 formatted date and time
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
101 """
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
102 template_date = "%Y-%m-%d"
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
103 template_time = "%H:%M:%SZ"
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
104 template = (
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
105 "{}T{}".format(template_date, template_time) if with_time else template_date
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
106 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
107 return datetime.datetime.utcfromtimestamp(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
108 time.time() if timestamp is None else timestamp
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
109 ).strftime(template)
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
110
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
111
2181
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
112 def generatePassword(vocabulary=None, size=20):
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
113 """Generate a password with random characters.
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
114
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
115 @param vocabulary(iterable): characters to use to create password
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
116 @param size(int): number of characters in the password to generate
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
117 @return (unicode): generated password
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
118 """
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
119 random.seed()
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools:
Goffi <goffi@goffi.org>
parents: 2089
diff changeset
120 if vocabulary is None:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
121 vocabulary = [
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
122 chr(i) for i in list(range(0x30, 0x3A)) + list(range(0x41, 0x5B)) + list(range(0x61, 0x7B))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
123 ]
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
124 return "".join([random.choice(vocabulary) for i in range(15)])
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
125
1502
566908d483f6 core (utils): added a method to generate XEP-0082 style dates
Goffi <goffi@goffi.org>
parents: 1471
diff changeset
126
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
127 def getRepositoryData(module, as_string=True, is_path=False):
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
128 """Retrieve info on current mecurial repository
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
129
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
130 Data is gotten by using the following methods, in order:
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
131 - using "hg" executable
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
132 - looking for a .hg/dirstate in parent directory of module (or in module/.hg if
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
133 is_path is True), and parse dirstate file to get revision
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
134 - checking package version, which should have repository data when we are on a dev version
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
135 @param module(unicode): module to look for (e.g. sat, libervia)
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
136 module can be a path if is_path is True (see below)
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
137 @param as_string(bool): if True return a string, else return a dictionary
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
138 @param is_path(bool): if True "module" is not handled as a module name, but as an
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
139 absolute path to the parent of a ".hg" directory
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
140 @return (unicode, dictionary): retrieved info in a nice string,
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
141 or a dictionary with retrieved data (key is not present if data is not found),
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
142 key can be:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
143 - node: full revision number (40 bits)
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
144 - branch: branch name
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
145 - date: ISO 8601 format date
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
146 - tag: latest tag used in hierarchie
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
147 - distance: number of commits since the last tag
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
148 """
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
149 global repos_cache_dict
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
150 if as_string:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
151 global repos_cache
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
152 if repos_cache is not None:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
153 return repos_cache
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
154 else:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
155 if repos_cache_dict is not None:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
156 return repos_cache_dict
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
157
2089
0931b5a6213c core, quick_frontends: android compatibility hacks:
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
158 if sys.platform == "android":
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
159 #  FIXME: workaround to avoid trouble on android, need to be fixed properly
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
160 repos_cache = "Cagou android build"
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
161 return repos_cache
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
162
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
163 KEYS = ("node", "node_short", "branch", "date", "tag", "distance")
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
164 ori_cwd = os.getcwd()
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
165
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
166 if is_path:
2568
8e204f0d3193 tools(utils): fixed path handling in getRepositoryData
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
167 repos_root = os.path.abspath(module)
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
168 else:
2568
8e204f0d3193 tools(utils): fixed path handling in getRepositoryData
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
169 repos_root = os.path.abspath(os.path.dirname(module.__file__))
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
170
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
171 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
172 hg_path = procutils.which("hg")[0]
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
173 except IndexError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
174 log.warning("Can't find hg executable")
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
175 hg_path = None
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
176 hg_data = {}
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
177
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
178 if hg_path is not None:
1376
28fd9e838f8f core: getRepositoryData now get the module in argument
Goffi <goffi@goffi.org>
parents: 1375
diff changeset
179 os.chdir(repos_root)
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
180 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
181 hg_data_raw = subprocess.check_output(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
182 [
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
183 "hg",
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
184 "log",
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
185 "-r",
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
186 "-1",
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
187 "--template",
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
188 "{node}\n"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
189 "{node|short}\n"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
190 "{branch}\n"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
191 "{date|isodate}\n"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
192 "{latesttag}\n"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
193 "{latesttagdistance}",
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
194 ],
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
195 text=True
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
196 )
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
197 except subprocess.CalledProcessError:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
198 hg_data = {}
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
199 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
200 hg_data = dict(list(zip(KEYS, hg_data_raw.split("\n"))))
1857
2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
201 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
202 hg_data["modified"] = "+" in subprocess.check_output(["hg", "id", "-i"], text=True)
1857
2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
203 except subprocess.CalledProcessError:
2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
204 pass
1471
934e402c90bf core: tools.utils.getRepositoryData now return "hg log -r -1" and short form of node + fixed crash if mercurial is not present:
Goffi <goffi@goffi.org>
parents: 1396
diff changeset
205 else:
934e402c90bf core: tools.utils.getRepositoryData now return "hg log -r -1" and short form of node + fixed crash if mercurial is not present:
Goffi <goffi@goffi.org>
parents: 1396
diff changeset
206 hg_data = {}
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
207
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
208 if not hg_data:
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
209 # .hg/dirstate method
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
210 log.debug("trying dirstate method")
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
211 if is_path:
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
212 os.chdir(repos_root)
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
213 else:
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
214 os.chdir(os.path.abspath(os.path.dirname(repos_root)))
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
215 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
216 with open(".hg/dirstate") as hg_dirstate:
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
217 hg_data["node"] = hg_dirstate.read(20).encode("hex")
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
218 hg_data["node_short"] = hg_data["node"][:12]
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
219 except IOError:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
220 log.debug("Can't access repository data")
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
221
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
222 # we restore original working dir
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
223 os.chdir(ori_cwd)
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
224
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
225 if not hg_data:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
226 log.debug("Mercurial not available or working, trying package version")
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
227 try:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
228 import pkg_resources
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
229
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
230 pkg_version = pkg_resources.get_distribution(C.APP_NAME_FILE).version
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
231 version, local_id = pkg_version.split("+", 1)
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
232 except ImportError:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
233 log.warning("pkg_resources not available, can't get package data")
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
234 except pkg_resources.DistributionNotFound:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
235 log.warning("can't retrieve package data")
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
236 except ValueError:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
237 log.info(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
238 "no local version id in package: {pkg_version}".format(
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
239 pkg_version=pkg_version
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
240 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
241 )
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
242 else:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
243 version = version.replace(".dev0", "D")
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
244 if version != C.APP_VERSION:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
245 log.warning(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
246 "Incompatible version ({version}) and pkg_version ({pkg_version})".format(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
247 version=C.APP_VERSION, pkg_version=pkg_version
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
248 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
249 )
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
250 else:
2576
cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData
Goffi <goffi@goffi.org>
parents: 2573
diff changeset
251 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
252 hg_node, hg_distance = local_id.split(".")
2576
cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData
Goffi <goffi@goffi.org>
parents: 2573
diff changeset
253 except ValueError:
cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData
Goffi <goffi@goffi.org>
parents: 2573
diff changeset
254 log.warning("Version doesn't specify repository data")
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
255 hg_data = {"node_short": hg_node, "distance": hg_distance}
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
256
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
257 repos_cache_dict = hg_data
1858
06e13ae616cf tools (utils): improved repository version detection:
Goffi <goffi@goffi.org>
parents: 1857
diff changeset
258
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
259 if as_string:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
260 if not hg_data:
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
261 repos_cache = NO_REPOS_DATA
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
262 else:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
263 strings = ["rev", hg_data["node_short"]]
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
264 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2612
diff changeset
265 if hg_data["modified"]:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
266 strings.append("[M]")
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
267 except KeyError:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
268 pass
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
269 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
270 strings.extend(["({branch} {date})".format(**hg_data)])
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
271 except KeyError:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
272 pass
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
273 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
274 strings.extend(["+{distance}".format(**hg_data)])
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
275 except KeyError:
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
276 pass
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2793
diff changeset
277 repos_cache = " ".join(strings)
2573
18e2ca5f798e tools (utils): better repository version handling:
Goffi <goffi@goffi.org>
parents: 2568
diff changeset
278 return repos_cache
1375
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
279 else:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version
Goffi <goffi@goffi.org>
parents: 916
diff changeset
280 return hg_data