Mercurial > libervia-backend

annotate sat/tools/config.py @ 3440:53da72a17139

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
tests (e2e): new fixture to share data between tests: this is usefull in some edge cases when tests are dependents of each other.
author Goffi <goffi@goffi.org>
date Fri, 04 Dec 2020 12:38:38 +0100
parents a3639d6d9643
children be6d91572633
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
3
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT: a jabber client
3136
9d0df638c8b4 dates update
Goffi <goffi@goffi.org>
parents: 3053
diff changeset
5 # Copyright (C) 2009-2020 Jérôme Poisson (goffi@goffi.org)
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1445
diff changeset
6 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
7
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # This program is free software: you can redistribute it and/or modify
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # it under the terms of the GNU Affero General Public License as published by
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # the Free Software Foundation, either version 3 of the License, or
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
11 # (at your option) any later version.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
12
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # This program is distributed in the hope that it will be useful,
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
16 # GNU Affero General Public License for more details.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
17
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # You should have received a copy of the GNU Affero General Public License
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
19 # along with this program. If not, see <http://www.gnu.org/licenses/>.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
20
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
21 """ Configuration related useful methods """
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
22
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
23 import os
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
24 import csv
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
25 import json
3031
98d1f34ce5b9 tools (config), memory: renamed SafeConfigParser following Python 3 port
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
26 from configparser import ConfigParser, DEFAULTSECT, NoOptionError, NoSectionError
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
27 from xdg import BaseDirectory
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
28 from sat.core.log import getLogger
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
29 from sat.core.constants import Const as C
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
30 from sat.core.i18n import _
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
31 from sat.core import exceptions
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
32
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
33 log = getLogger(__name__)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
34
2965
121c4a2a567c core (config): if flatpak is detected, config is also looked after in /app
Goffi <goffi@goffi.org>
parents: 2836
diff changeset
35
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
36 def fixConfigOption(section, option, value, silent=True):
3281
a3639d6d9643 core: replaced `sat` shell script by a python script:
Goffi <goffi@goffi.org>
parents: 3148
diff changeset
37 """Force a configuration option value
a3639d6d9643 core: replaced `sat` shell script by a python script:
Goffi <goffi@goffi.org>
parents: 3148
diff changeset
38
a3639d6d9643 core: replaced `sat` shell script by a python script:
Goffi <goffi@goffi.org>
parents: 3148
diff changeset
39 the option will be written in the first found user config file, a new user
a3639d6d9643 core: replaced `sat` shell script by a python script:
Goffi <goffi@goffi.org>
parents: 3148
diff changeset
40 config will be created if none is found.
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
41
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
42 @param section (str): the config section
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
43 @param option (str): the config option
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
44 @param value (str): the new value
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
45 @param silent (boolean): toggle logging output (must be True when called from sat.sh)
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
46 """
3031
98d1f34ce5b9 tools (config), memory: renamed SafeConfigParser following Python 3 port
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
47 config = ConfigParser()
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
48 target_file = None
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
49 for file_ in C.CONFIG_FILES[::-1]:
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
50 # we will eventually update the existing file with the highest priority,
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
51 # if it's a user personal file...
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
52 if not silent:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
53 log.debug(_("Testing file %s") % file_)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
54 if os.path.isfile(file_):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
55 if file_.startswith(os.path.expanduser("~")):
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
56 config.read([file_])
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
57 target_file = file_
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
58 break
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
59 if not target_file:
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
60 # ... otherwise we create a new config file for that user
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
61 target_file = BaseDirectory.save_config_path("sat") + "/sat.conf"
1056
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
62 if section and section.upper() != DEFAULTSECT and not config.has_section(section):
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
63 config.add_section(section)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
64 config.set(section, option, value)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
65 with open(target_file, "wb") as configfile:
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
66 config.write(configfile) # for the next time that user launches sat
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
67 if not silent:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
68 if option in ("passphrase",): # list here the options storing a password
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
69 value = "******"
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
70 log.warning(_("Config auto-update: {option} set to {value} in the file "
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
71 "{config_file}.").format(option=option, value=value,
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
72 config_file=target_file))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
73
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
74
3148
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
75 def parseMainConf(log_filenames=False):
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
76 """Look for main .ini configuration file, and parse it
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
77
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
78 @param log_filenames(bool): if True, log filenames of read config files
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
79 """
3031
98d1f34ce5b9 tools (config), memory: renamed SafeConfigParser following Python 3 port
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
80 config = ConfigParser(defaults=C.DEFAULT_CONFIG)
1859
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
81 try:
3148
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
82 filenames = config.read(C.CONFIG_FILES)
2835
6aa22011bc6d tools (config): log error message if config can't be read
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
83 except Exception as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
84 log.error(_("Can't read main config: {msg}").format(msg=e), exc_info=True)
3148
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
85 else:
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
86 if log_filenames:
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
87 log.info(
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
88 _("Configuration was read from: {filenames}").format(
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
89 filenames=', '.join(filenames)))
60a9e47ef988 core: log filenames of read config files
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
90
1859
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
91 return config
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
92
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
93
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
94 def getConfig(config, section, name, default=None):
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
95 """Get a configuration option
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
96
3031
98d1f34ce5b9 tools (config), memory: renamed SafeConfigParser following Python 3 port
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
97 @param config (ConfigParser): the configuration instance
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
98 @param section (str): section of the config file (None or '' for DEFAULT)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
99 @param name (str): name of the option
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
100 @param default: value to use if not found, or Exception to raise an exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
101 @return (unicode, list, dict): parsed value
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
102 @raise: NoOptionError if option is not present and default is Exception
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
103 NoSectionError if section doesn't exists and default is Exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
104 exceptions.ParsingError error while parsing value
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
105 """
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
106 if not section:
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
107 section = DEFAULTSECT
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
108
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
109 try:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
110 value = config.get(section, name)
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
111 except (NoOptionError, NoSectionError) as e:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
112 if default is Exception:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
113 raise e
1234
9c17bd37e6e5 core: better management of default value in getConfig
Goffi <goffi@goffi.org>
parents: 1064
diff changeset
114 return default
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
115
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
116 if name.endswith("_path") or name.endswith("_dir"):
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
117 value = os.path.expanduser(value)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
118 # thx to Brian (http://stackoverflow.com/questions/186857/splitting-a-semicolon-separated-string-to-a-dictionary-in-python/186873#186873)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
119 elif name.endswith("_list"):
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
120 value = next(csv.reader(
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
121 [value], delimiter=",", quotechar='"', skipinitialspace=True
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
122 ))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
123 elif name.endswith("_dict"):
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
124 try:
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
125 value = json.loads(value)
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
126 except ValueError as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
127 raise exceptions.ParsingError("Error while parsing data: {}".format(e))
1833
a123e881f9e5 core (config): _dict values are now handled with json syntax
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
128 if not isinstance(value, dict):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
129 raise exceptions.ParsingError(
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
130 "{name} value is not a dict: {value}".format(name=name, value=value)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
131 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
132 elif name.endswith("_json"):
2451
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
133 try:
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
134 value = json.loads(value)
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
135 except ValueError as e:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2965
diff changeset
136 raise exceptions.ParsingError("Error while parsing data: {}".format(e))
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
137 return value