annotate sat/tools/config.py @ 3028:ab2696e34d29

Python 3 port: /!\ this is a huge commit /!\ starting from this commit, SàT is needs Python 3.6+ /!\ SàT maybe be instable or some feature may not work anymore, this will improve with time This patch port backend, bridge and frontends to Python 3. Roughly this has been done this way: - 2to3 tools has been applied (with python 3.7) - all references to python2 have been replaced with python3 (notably shebangs) - fixed files not handled by 2to3 (notably the shell script) - several manual fixes - fixed issues reported by Python 3 that where not handled in Python 2 - replaced "async" with "async_" when needed (it's a reserved word from Python 3.7) - replaced zope's "implements" with @implementer decorator - temporary hack to handle data pickled in database, as str or bytes may be returned, to be checked later - fixed hash comparison for password - removed some code which is not needed anymore with Python 3 - deactivated some code which needs to be checked (notably certificate validation) - tested with jp, fixed reported issues until some basic commands worked - ported Primitivus (after porting dependencies like urwid satext) - more manual fixes
author Goffi <goffi@goffi.org>
date Tue, 13 Aug 2019 19:08:41 +0200
parents 121c4a2a567c
children 98d1f34ce5b9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
1 #!/usr/bin/env python3
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
3
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT: a jabber client
2771
003b8b4b56a7 date update
Goffi <goffi@goffi.org>
parents: 2671
diff changeset
5 # Copyright (C) 2009-2019 Jérôme Poisson (goffi@goffi.org)
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1445
diff changeset
6 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
7
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # This program is free software: you can redistribute it and/or modify
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # it under the terms of the GNU Affero General Public License as published by
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # the Free Software Foundation, either version 3 of the License, or
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
11 # (at your option) any later version.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
12
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # This program is distributed in the hope that it will be useful,
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
16 # GNU Affero General Public License for more details.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
17
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # You should have received a copy of the GNU Affero General Public License
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
19 # along with this program. If not, see <http://www.gnu.org/licenses/>.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
20
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
21 """ Configuration related useful methods """
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
22
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
23 import os
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
24 import csv
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
25 import json
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
26 from configparser import SafeConfigParser, DEFAULTSECT, NoOptionError, NoSectionError
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
27 from xdg import BaseDirectory
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
28 from sat.core.log import getLogger
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
29 from sat.core.constants import Const as C
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
30 from sat.core.i18n import _
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
31 from sat.core import exceptions
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
32
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
33 log = getLogger(__name__)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
34
2965
121c4a2a567c core (config): if flatpak is detected, config is also looked after in /app
Goffi <goffi@goffi.org>
parents: 2836
diff changeset
35
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
36 def fixConfigOption(section, option, value, silent=True):
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
37 """Force a configuration option value, writing it in the first found user
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
38 config file, eventually creating a new user config file if none is found.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
39
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
40 @param section (str): the config section
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
41 @param option (str): the config option
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
42 @param value (str): the new value
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
43 @param silent (boolean): toggle logging output (must be True when called from sat.sh)
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
44 """
1900
d1615f79dfe8 core (tools/config): fixed fixConfigOption:
Goffi <goffi@goffi.org>
parents: 1859
diff changeset
45 config = SafeConfigParser()
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
46 target_file = None
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
47 for file_ in C.CONFIG_FILES[::-1]:
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
48 # we will eventually update the existing file with the highest priority,
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
49 # if it's a user personal file...
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
50 if not silent:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
51 log.debug(_("Testing file %s") % file_)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
52 if os.path.isfile(file_):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
53 if file_.startswith(os.path.expanduser("~")):
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
54 config.read([file_])
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
55 target_file = file_
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
56 break
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
57 if not target_file:
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
58 # ... otherwise we create a new config file for that user
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
59 target_file = BaseDirectory.save_config_path("sat") + "/sat.conf"
1056
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
60 if section and section.upper() != DEFAULTSECT and not config.has_section(section):
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
61 config.add_section(section)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
62 config.set(section, option, value)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
63 with open(target_file, "wb") as configfile:
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
64 config.write(configfile) # for the next time that user launches sat
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
65 if not silent:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
66 if option in ("passphrase",): # list here the options storing a password
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
67 value = "******"
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
68 log.warning(_("Config auto-update: {option} set to {value} in the file "
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
69 "{config_file}.").format(option=option, value=value,
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
70 config_file=target_file))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
71
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
72
1859
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
73 def parseMainConf():
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
74 """look for main .ini configuration file, and parse it"""
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
75 config = SafeConfigParser(defaults=C.DEFAULT_CONFIG)
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
76 try:
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
77 config.read(C.CONFIG_FILES)
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
78 except Exception as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
79 log.error(_("Can't read main config: {msg}").format(msg=e), exc_info=True)
1859
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
80 return config
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
81
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
82
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
83 def getConfig(config, section, name, default=None):
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
84 """Get a configuration option
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
85
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
86 @param config (SafeConfigParser): the configuration instance
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
87 @param section (str): section of the config file (None or '' for DEFAULT)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
88 @param name (str): name of the option
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
89 @param default: value to use if not found, or Exception to raise an exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
90 @return (unicode, list, dict): parsed value
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
91 @raise: NoOptionError if option is not present and default is Exception
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
92 NoSectionError if section doesn't exists and default is Exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
93 exceptions.ParsingError error while parsing value
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
94 """
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
95 if not section:
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
96 section = DEFAULTSECT
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
97
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
98 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
99 value = config.get(section, name)
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
100 except (NoOptionError, NoSectionError) as e:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
101 if default is Exception:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
102 raise e
1234
9c17bd37e6e5 core: better management of default value in getConfig
Goffi <goffi@goffi.org>
parents: 1064
diff changeset
103 return default
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
104
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
105 if name.endswith("_path") or name.endswith("_dir"):
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
106 value = os.path.expanduser(value)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
107 # thx to Brian (http://stackoverflow.com/questions/186857/splitting-a-semicolon-separated-string-to-a-dictionary-in-python/186873#186873)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
108 elif name.endswith("_list"):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
109 value = next(csv.reader(
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
110 [value], delimiter=",", quotechar='"', skipinitialspace=True
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
111 ))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
112 elif name.endswith("_dict"):
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
113 try:
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
114 value = json.loads(value)
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
115 except ValueError as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
116 raise exceptions.ParsingError("Error while parsing data: {}".format(e))
1833
a123e881f9e5 core (config): _dict values are now handled with json syntax
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
117 if not isinstance(value, dict):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
118 raise exceptions.ParsingError(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
119 "{name} value is not a dict: {value}".format(name=name, value=value)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
120 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
121 elif name.endswith("_json"):
2451
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
122 try:
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
123 value = json.loads(value)
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
124 except ValueError as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
125 raise exceptions.ParsingError("Error while parsing data: {}".format(e))
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
126 return value