Mercurial > libervia-backend

annotate sat/tools/config.py @ 2766:93a421de0e3d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
tools (common/data_objects): metadata parsing in BlogItems: metadata are parsed to deserialise some well known values like rsm index and count, and some properties have been added to have easier access. A "complete" property (or item in metadata) is set to True if we are on the last page, False if we are not, or None if we don't have enough data to know if we are on the last page or not.
author Goffi <goffi@goffi.org>
date Fri, 11 Jan 2019 16:35:13 +0100
parents 0fa217fafabf
children 003b8b4b56a7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1900
diff changeset
1 #!/usr/bin/env python2
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
3
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT: a jabber client
2483
0046283a285d dates update
Goffi <goffi@goffi.org>
parents: 2451
diff changeset
5 # Copyright (C) 2009-2018 Jérôme Poisson (goffi@goffi.org)
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1445
diff changeset
6 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
7
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # This program is free software: you can redistribute it and/or modify
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # it under the terms of the GNU Affero General Public License as published by
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # the Free Software Foundation, either version 3 of the License, or
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
11 # (at your option) any later version.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
12
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # This program is distributed in the hope that it will be useful,
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
16 # GNU Affero General Public License for more details.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
17
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # You should have received a copy of the GNU Affero General Public License
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
19 # along with this program. If not, see <http://www.gnu.org/licenses/>.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
20
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
21 """ Configuration related useful methods """
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
22
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
23 from sat.core.log import getLogger
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
24
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
25 log = getLogger(__name__)
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
26
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
27 from sat.core.constants import Const as C
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
28 from sat.core.i18n import _
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
29 from sat.core import exceptions
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
30
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
31 from ConfigParser import SafeConfigParser, DEFAULTSECT, NoOptionError, NoSectionError
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
32 from xdg import BaseDirectory
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
33 import os
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
34 import csv
1833
a123e881f9e5 core (config): _dict values are now handled with json syntax
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
35 import json
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
36
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
37
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
38 def fixConfigOption(section, option, value, silent=True):
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
39 """Force a configuration option value, writing it in the first found user
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
40 config file, eventually creating a new user config file if none is found.
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
41
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
42 @param section (str): the config section
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
43 @param option (str): the config option
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
44 @param value (str): the new value
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
45 @param silent (boolean): toggle logging output (must be True when called from sat.sh)
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
46 """
1900
d1615f79dfe8 core (tools/config): fixed fixConfigOption:
Goffi <goffi@goffi.org>
parents: 1859
diff changeset
47 config = SafeConfigParser()
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
48 target_file = None
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
49 for file_ in C.CONFIG_FILES[::-1]:
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
50 # we will eventually update the existing file with the highest priority,
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
51 # if it's a user personal file...
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
52 if not silent:
1409
3265a2639182 massive (preventive) addition of 'u' (unicode) before the strings passed to logging functions
souliane <souliane@mailoo.org>
parents: 1396
diff changeset
53 log.debug(_(u"Testing file %s") % file_)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
54 if os.path.isfile(file_):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
55 if file_.startswith(os.path.expanduser("~")):
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
56 config.read([file_])
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
57 target_file = file_
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
58 break
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
59 if not target_file:
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
60 # ... otherwise we create a new config file for that user
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
61 target_file = BaseDirectory.save_config_path("sat") + "/sat.conf"
1056
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
62 if section and section.upper() != DEFAULTSECT and not config.has_section(section):
a5cfa9bb4541 tools (config): fixConfigOption creates the section if it doesn't exist
souliane <souliane@mailoo.org>
parents: 1046
diff changeset
63 config.add_section(section)
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
64 config.set(section, option, value)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
65 with open(target_file, "wb") as configfile:
1046
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
66 config.write(configfile) # for the next time that user launches sat
a874a79ad0f5 tools: add missing file src/tools/config.py
souliane <souliane@mailoo.org>
parents:
diff changeset
67 if not silent:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
68 if option in ("passphrase",): # list here the options storing a password
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
69 value = "******"
2671
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
70 log.warning(_(u"Config auto-update: {option} set to {value} in the file "
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
71 u"{config_file}.").format(option=option, value=value,
0fa217fafabf tools (common/template), jp: refactoring to handle multiple sites:
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
72 config_file=target_file))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
73
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
74
1859
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
75 def parseMainConf():
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
76 """look for main .ini configuration file, and parse it"""
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
77 config = SafeConfigParser(defaults=C.DEFAULT_CONFIG)
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
78 try:
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
79 config.read(C.CONFIG_FILES)
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
80 except:
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
81 log.error(_("Can't read main config !"))
ac2ac7fe8a9b core (memory, config): moved parseMainConf to tools/config so it can be used by frontends too
Goffi <goffi@goffi.org>
parents: 1835
diff changeset
82 return config
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
83
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
84
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
85 def getConfig(config, section, name, default=None):
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
86 """Get a configuration option
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
87
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
88 @param config (SafeConfigParser): the configuration instance
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
89 @param section (str): section of the config file (None or '' for DEFAULT)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
90 @param name (str): name of the option
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
91 @param default: value to use if not found, or Exception to raise an exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
92 @return (unicode, list, dict): parsed value
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
93 @raise: NoOptionError if option is not present and default is Exception
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
94 NoSectionError if section doesn't exists and default is Exception
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
95 exceptions.ParsingError error while parsing value
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
96 """
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
97 if not section:
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
98 section = DEFAULTSECT
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
99
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
100 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
101 value = config.get(section, name).decode("utf-8")
1235
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
102 except (NoOptionError, NoSectionError) as e:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
103 if default is Exception:
80d2ed788b40 core (config): added the Exception default value which raise an exception instead of returning the default in getConfig
Goffi <goffi@goffi.org>
parents: 1234
diff changeset
104 raise e
1234
9c17bd37e6e5 core: better management of default value in getConfig
Goffi <goffi@goffi.org>
parents: 1064
diff changeset
105 return default
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
106
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
107 if name.endswith("_path") or name.endswith("_dir"):
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
108 value = os.path.expanduser(value)
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
109 # thx to Brian (http://stackoverflow.com/questions/186857/splitting-a-semicolon-separated-string-to-a-dictionary-in-python/186873#186873)
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
110 elif name.endswith("_list"):
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
111 value = csv.reader(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
112 [value], delimiter=",", quotechar='"', skipinitialspace=True
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
113 ).next()
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
114 elif name.endswith("_dict"):
1835
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
115 try:
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
116 value = json.loads(value)
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
117 except ValueError as e:
5b8a859d5bb4 core (config): getConfig now returns unicode and raise exceptions.ParsingError in case of parsing problem
Goffi <goffi@goffi.org>
parents: 1833
diff changeset
118 raise exceptions.ParsingError(u"Error while parsing data: {}".format(e))
1833
a123e881f9e5 core (config): _dict values are now handled with json syntax
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
119 if not isinstance(value, dict):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
120 raise exceptions.ParsingError(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
121 u"{name} value is not a dict: {value}".format(name=name, value=value)
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
122 )
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
123 elif name.endswith("_json"):
2451
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
124 try:
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
125 value = json.loads(value)
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
126 except ValueError as e:
d1153ce68ca0 tools (config): complexe data can now be set using json and the "_json" suffix
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
127 raise exceptions.ParsingError(u"Error while parsing data: {}".format(e))
1064
7ee9d9db67b9 memory, tools (config): move special config retrieval from memory to tools
souliane <souliane@mailoo.org>
parents: 1056
diff changeset
128 return value