libervia-backend: sat/tools/utils.py annotate

annotate sat/tools/utils.py @ 2787:298408833ec2

memory (sqlite): optimizations indexes were missing on foreign keys for "message", "subject" and "thread" tables, seriously impacting performances. In addition to those indexes, two indexes have been added to speed ordering by timestamp on "history", and one on "files" table. history.rowid is not used anymore as there is an index on (profile_id, received_timestamp) which will speed up the query. Primary keys order has been changed to use automatic index in most common cases (filtering by profile_id then namespace).

author	Goffi <goffi@goffi.org>
date	Sat, 19 Jan 2019 22:49:32 +0100
parents	003b8b4b56a7
children	181735d1b062

rev	line source
1934 2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang Goffi <goffi@goffi.org> parents: 1858 diff changeset	1 #!/usr/bin/env python2
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	2 # -- coding: utf-8 --
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	3
609 84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	4 # SAT: a jabber client
2771 003b8b4b56a7 date update Goffi <goffi@goffi.org> parents: 2624 diff changeset	5 # Copyright (C) 2009-2019 Jérôme Poisson (goffi@goffi.org)
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	6
609 84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	7 # This program is free software: you can redistribute it and/or modify
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	10 # (at your option) any later version.
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	11
609 84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	12 # This program is distributed in the hope that it will be useful,
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	15 # GNU Affero General Public License for more details.
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	16
609 84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	17 # You should have received a copy of the GNU Affero General Public License
84a6e83157c2 fixed licences in docstrings (they are now in comments) Goffi <goffi@goffi.org> parents: 601 diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	19
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	20 """ various useful methods """
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	21
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	22 import unicodedata
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	23 import os.path
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	24 from sat.core.constants import Const as C
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	25 from sat.core.log import getLogger
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	26
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	27 log = getLogger(__name__)
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	28 import datetime
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	29 from twisted.python import procutils
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	30 import subprocess
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	31 import time
2089 0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1935 diff changeset	32 import sys
2181 968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	33 import random
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	34 import inspect
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	35 import textwrap
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	36 import functools
916 1a759096ccbd core: use of Const for profile_key + replaced '@DEFAULT@' default profile_key by '@NONE@' Goffi <goffi@goffi.org> parents: 811 diff changeset	37
1a759096ccbd core: use of Const for profile_key + replaced '@DEFAULT@' default profile_key by '@NONE@' Goffi <goffi@goffi.org> parents: 811 diff changeset	38
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	39 NO_REPOS_DATA = u"repository data unknown"
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	40 repos_cache_dict = None
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	41 repos_cache = None
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	42
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	43
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	44 def clean_ustr(ustr):
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	45 """Clean unicode string
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	46
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	47 remove special characters from unicode string
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	48 """
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	49
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	50 def valid_chars(unicode_source):
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	51 for char in unicode_source:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	52 if unicodedata.category(char) == "Cc" and char != "\n":
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	53 continue
a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	54 yield char
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	55
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	56 return "".join(valid_chars(ustr))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	57
601 a4f6f78f0620 jp, core: jp's clean_ustr moved to a new general utils module Goffi <goffi@goffi.org> parents: diff changeset	58
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	59 def partial(func, fixed_args, *fixed_kwargs):
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	60 # FIXME: temporary hack to workaround the fact that inspect.getargspec is not working with functools.partial
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	61 # making partial unusable with current D-bus module (in addMethod).
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	62 # Should not be needed anywore once moved to Python 3
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	63
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	64 ori_args = inspect.getargspec(func).args
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	65 func = functools.partial(func, fixed_args, *fixed_kwargs)
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	66 if ori_args[0] == "self":
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	67 del ori_args[0]
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	68 ori_args = ori_args[len(fixed_args) :]
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	69 for kw in fixed_kwargs:
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	70 ori_args.remove(kw)
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	71
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	72 exec(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	73 textwrap.dedent(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	74 """\
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	75 def method({args}):
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	76 return func({kw_args})
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	77 """
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	78 ).format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	79 args=", ".join(ori_args), kw_args=", ".join([a + "=" + a for a in ori_args])
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	80 ),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	81 locals(),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	82 )
2470 8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	83
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	84 return method
8084066ac95b tools (utils): new "partial" function which is a hack to workaround issue with functools.partial while using it with dbus module. Goffi <goffi@goffi.org> parents: 2414 diff changeset	85
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	86
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	87 def xmpp_date(timestamp=None, with_time=True):
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	88 """Return date according to XEP-0082 specification
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	89
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	90 to avoid reveling the timezone, we always return UTC dates
1935 1128feb54180 core: removed pyfeed and xe dependencies: Goffi <goffi@goffi.org> parents: 1934 diff changeset	91 the string returned by this method is valid with RFC 3339
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	92 @param timestamp(None, float): posix timestamp. If None current time will be used
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	93 @param with_time(bool): if True include the time
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	94 @return(unicode): XEP-0082 formatted date and time
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	95 """
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	96 template_date = u"%Y-%m-%d"
566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	97 template_time = u"%H:%M:%SZ"
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	98 template = (
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	99 u"{}T{}".format(template_date, template_time) if with_time else template_date
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	100 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	101 return datetime.datetime.utcfromtimestamp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	102 time.time() if timestamp is None else timestamp
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	103 ).strftime(template)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	104
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	105
2181 968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	106 def generatePassword(vocabulary=None, size=20):
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	107 """Generate a password with random characters.
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	108
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	109 @param vocabulary(iterable): characters to use to create password
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	110 @param size(int): number of characters in the password to generate
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	111 @return (unicode): generated password
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	112 """
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	113 random.seed()
968b0d13bcc7 plugin account, tools: some cleaning account + email and password tools: Goffi <goffi@goffi.org> parents: 2089 diff changeset	114 if vocabulary is None:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	115 vocabulary = [
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	116 chr(i) for i in range(0x30, 0x3A) + range(0x41, 0x5B) + range(0x61, 0x7B)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	117 ]
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	118 return u"".join([random.choice(vocabulary) for i in range(15)])
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	119
1502 566908d483f6 core (utils): added a method to generate XEP-0082 style dates Goffi <goffi@goffi.org> parents: 1471 diff changeset	120
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	121 def getRepositoryData(module, as_string=True, is_path=False):
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	122 """Retrieve info on current mecurial repository
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	123
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	124 Data is gotten by using the following methods, in order:
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	125 - using "hg" executable
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	126 - looking for a .hg/dirstate in parent directory of module (or in module/.hg if
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	127 is_path is True), and parse dirstate file to get revision
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	128 - checking package version, which should have repository data when we are on a dev version
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	129 @param module(unicode): module to look for (e.g. sat, libervia)
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	130 module can be a path if is_path is True (see below)
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	131 @param as_string(bool): if True return a string, else return a dictionary
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	132 @param is_path(bool): if True "module" is not handled as a module name, but as an
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	133 absolute path to the parent of a ".hg" directory
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	134 @return (unicode, dictionary): retrieved info in a nice string,
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	135 or a dictionary with retrieved data (key is not present if data is not found),
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	136 key can be:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	137 - node: full revision number (40 bits)
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	138 - branch: branch name
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	139 - date: ISO 8601 format date
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	140 - tag: latest tag used in hierarchie
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	141 - distance: number of commits since the last tag
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	142 """
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	143 global repos_cache_dict
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	144 if as_string:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	145 global repos_cache
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	146 if repos_cache is not None:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	147 return repos_cache
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	148 else:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	149 if repos_cache_dict is not None:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	150 return repos_cache_dict
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	151
2089 0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1935 diff changeset	152 if sys.platform == "android":
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	153 # FIXME: workaround to avoid trouble on android, need to be fixed properly
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	154 repos_cache = u"Cagou android build"
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	155 return repos_cache
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	156
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	157 KEYS = ("node", "node_short", "branch", "date", "tag", "distance")
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	158 ori_cwd = os.getcwd()
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	159
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	160 if is_path:
2568 8e204f0d3193 tools(utils): fixed path handling in getRepositoryData Goffi <goffi@goffi.org> parents: 2562 diff changeset	161 repos_root = os.path.abspath(module)
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	162 else:
2568 8e204f0d3193 tools(utils): fixed path handling in getRepositoryData Goffi <goffi@goffi.org> parents: 2562 diff changeset	163 repos_root = os.path.abspath(os.path.dirname(module.__file__))
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	164
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	165 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	166 hg_path = procutils.which("hg")[0]
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	167 except IndexError:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	168 log.warning(u"Can't find hg executable")
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	169 hg_path = None
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	170 hg_data = {}
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	171
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	172 if hg_path is not None:
1376 28fd9e838f8f core: getRepositoryData now get the module in argument Goffi <goffi@goffi.org> parents: 1375 diff changeset	173 os.chdir(repos_root)
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	174 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	175 hg_data_raw = subprocess.check_output(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	176 [
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	177 "hg",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	178 "log",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	179 "-r",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	180 "-1",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	181 "--template",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	182 "{node}\n"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	183 "{node\|short}\n"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	184 "{branch}\n"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	185 "{date\|isodate}\n"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	186 "{latesttag}\n"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	187 "{latesttagdistance}",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	188 ]
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	189 )
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	190 except subprocess.CalledProcessError:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	191 hg_data = {}
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	192 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	193 hg_data = dict(zip(KEYS, hg_data_raw.split("\n")))
1857 2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception Goffi <goffi@goffi.org> parents: 1766 diff changeset	194 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	195 hg_data["modified"] = "+" in subprocess.check_output(["hg", "id", "-i"])
1857 2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception Goffi <goffi@goffi.org> parents: 1766 diff changeset	196 except subprocess.CalledProcessError:
2d2617930f97 core (tools): fixed bad call to "hg id" when hg is not working to guest revision, which was the cause of an Exception Goffi <goffi@goffi.org> parents: 1766 diff changeset	197 pass
1471 934e402c90bf core: tools.utils.getRepositoryData now return "hg log -r -1" and short form of node + fixed crash if mercurial is not present: Goffi <goffi@goffi.org> parents: 1396 diff changeset	198 else:
934e402c90bf core: tools.utils.getRepositoryData now return "hg log -r -1" and short form of node + fixed crash if mercurial is not present: Goffi <goffi@goffi.org> parents: 1396 diff changeset	199 hg_data = {}
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	200
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	201 if not hg_data:
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	202 # .hg/dirstate method
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	203 log.debug(u"trying dirstate method")
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	204 if is_path:
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	205 os.chdir(repos_root)
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	206 else:
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	207 os.chdir(os.path.abspath(os.path.dirname(repos_root)))
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	208 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	209 with open(".hg/dirstate") as hg_dirstate:
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	210 hg_data["node"] = hg_dirstate.read(20).encode("hex")
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	211 hg_data["node_short"] = hg_data["node"][:12]
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	212 except IOError:
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	213 log.debug(u"Can't access repository data")
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	214
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	215 # we restore original working dir
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	216 os.chdir(ori_cwd)
06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	217
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	218 if not hg_data:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	219 log.debug(u"Mercurial not available or working, trying package version")
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	220 try:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	221 import pkg_resources
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	222
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	223 pkg_version = pkg_resources.get_distribution(C.APP_NAME_FILE).version
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	224 version, local_id = pkg_version.split("+", 1)
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	225 except ImportError:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	226 log.warning("pkg_resources not available, can't get package data")
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	227 except pkg_resources.DistributionNotFound:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	228 log.warning("can't retrieve package data")
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	229 except ValueError:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	230 log.info(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	231 u"no local version id in package: {pkg_version}".format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	232 pkg_version=pkg_version
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	233 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	234 )
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	235 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	236 version = version.replace(".dev0", "D")
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	237 if version != C.APP_VERSION:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	238 log.warning(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	239 "Incompatible version ({version}) and pkg_version ({pkg_version})".format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	240 version=C.APP_VERSION, pkg_version=pkg_version
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	241 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	242 )
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	243 else:
2576 cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData Goffi <goffi@goffi.org> parents: 2573 diff changeset	244 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	245 hg_node, hg_distance = local_id.split(".")
2576 cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData Goffi <goffi@goffi.org> parents: 2573 diff changeset	246 except ValueError:
cb7bf936d8e8 setup, tools(utils): fixed versioning in setup and getRepositoryData Goffi <goffi@goffi.org> parents: 2573 diff changeset	247 log.warning("Version doesn't specify repository data")
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	248 hg_data = {"node_short": hg_node, "distance": hg_distance}
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	249
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	250 repos_cache_dict = hg_data
1858 06e13ae616cf tools (utils): improved repository version detection: Goffi <goffi@goffi.org> parents: 1857 diff changeset	251
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	252 if as_string:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	253 if not hg_data:
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	254 repos_cache = NO_REPOS_DATA
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	255 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	256 strings = [u"rev", hg_data["node_short"]]
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	257 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	258 if hg_data["modified"]:
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	259 strings.append(u"[M]")
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	260 except KeyError:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	261 pass
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	262 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	263 strings.extend([u"({branch} {date})".format(**hg_data)])
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	264 except KeyError:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	265 pass
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	266 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	267 strings.extend([u"+{distance}".format(**hg_data)])
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	268 except KeyError:
18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	269 pass
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2612 diff changeset	270 repos_cache = u" ".join(strings)
2573 18e2ca5f798e tools (utils): better repository version handling: Goffi <goffi@goffi.org> parents: 2568 diff changeset	271 return repos_cache
1375 3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	272 else:
3a20312d4012 core: if we are in dev version and it's possible, repository data are now checked and added to SàT version Goffi <goffi@goffi.org> parents: 916 diff changeset	273 return hg_data

Mercurial > libervia-backend

annotate sat/tools/utils.py @ 2787:298408833ec2