libervia-backend: sat/core/log

annotate sat/core/log_config.py @ 3370:2157880ba3b4

core: plugins can now be loaded from a directory: A plugin can now be a directory with a name starting by `plugin_`. It will then be loaded as a module. The `__init__.py` file must contain the `PLUGIN_INFO` dict with metadata.

author	Goffi <goffi@goffi.org>
date	Sun, 20 Sep 2020 14:05:40 +0200
parents	96b9b65b4368
children	be6d91572633

rev	line source
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2836 diff changeset	1 #!/usr/bin/env python3
3137 559a625a236b fixed shebangs Goffi <goffi@goffi.org> parents: 3136 diff changeset	2
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	3
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	4 # SàT: a XMPP client
3136 9d0df638c8b4 dates update Goffi <goffi@goffi.org> parents: 3056 diff changeset	5 # Copyright (C) 2009-2020 Jérôme Poisson (goffi@goffi.org)
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	6
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	10 # (at your option) any later version.
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	11
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	16
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	19
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	20 """High level logging functions"""
1005 b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	21 # XXX: this module use standard logging module when possible, but as SàT can work in different cases where logging is not the best choice (twisted, pyjamas, etc), it is necessary to have a dedicated module. Additional feature like environment variables and colors are also managed.
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	22
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	23 from sat.core.constants import Const as C
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	24 from sat.core import log
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	25
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	26
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	27 class TwistedLogger(log.Logger):
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	28 colors = True
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	29 force_colors = False
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	30
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	31 def __init__(self, args, *kwargs):
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	32 super().__init__(args, *kwargs)
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	33 from twisted.logger import Logger
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	34 self.twisted_log = Logger()
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	35
2836 ad00f61fd9f5 core (log): add traceback when "exc_info" is set Goffi <goffi@goffi.org> parents: 2771 diff changeset	36 def out(self, message, level=None, **kwargs):
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	37 """Actually log the message
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	38
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	39 @param message: formatted message
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	40 """
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	41 if kwargs.pop('exc_info', False):
2836 ad00f61fd9f5 core (log): add traceback when "exc_info" is set Goffi <goffi@goffi.org> parents: 2771 diff changeset	42 message = self.addTraceback(message)
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	43 self.twisted_log.emit(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	44 level=self.level_map[level],
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	45 format=message,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	46 sat_logged=True,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	47 **kwargs,
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	48 )
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	49
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	50
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	51 class ConfigureBasic(log.ConfigureBase):
1942 7f053e1f0b67 core (logs): taints: Goffi <goffi@goffi.org> parents: 1941 diff changeset	52 def configureColors(self, colors, force_colors, levels_taints_dict):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	53 super(ConfigureBasic, self).configureColors(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	54 colors, force_colors, levels_taints_dict
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	55 )
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	56 if colors:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	57 import sys
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	58
2089 0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1942 diff changeset	59 try:
0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1942 diff changeset	60 isatty = sys.stdout.isatty()
0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1942 diff changeset	61 except AttributeError:
0931b5a6213c core, quick_frontends: android compatibility hacks: Goffi <goffi@goffi.org> parents: 1942 diff changeset	62 isatty = False
2732 e55f871fa9db core (log): fixed double call to ansiColors Goffi <goffi@goffi.org> parents: 2624 diff changeset	63 # FIXME: isatty should be tested on each handler, not globaly
e55f871fa9db core (log): fixed double call to ansiColors Goffi <goffi@goffi.org> parents: 2624 diff changeset	64 if (force_colors or isatty):
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	65 # we need colors
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	66 log.Logger.post_treat = lambda logger, level, message: self.ansiColors(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	67 level, message
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	68 )
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	69 elif force_colors:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	70 raise ValueError("force_colors can't be used if colors is False")
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	71
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	72 @staticmethod
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	73 def getProfile():
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	74 """Try to find profile value using introspection"""
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	75 import inspect
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	76
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	77 stack = inspect.stack()
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	78 current_path = stack[0][1]
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	79 for frame_data in stack[:-1]:
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	80 if frame_data[1] != current_path:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	81 if (
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	82 log.backend == C.LOG_BACKEND_STANDARD
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	83 and "/logging/__init__.py" in frame_data[1]
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	84 ):
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	85 continue
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	86 break
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	87
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	88 frame = frame_data[0]
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	89 args = inspect.getargvalues(frame)
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	90 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	91 profile = args.locals.get("profile") or args.locals["profile_key"]
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	92 except (TypeError, KeyError):
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	93 try:
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	94 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	95 profile = args.locals["self"].profile
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	96 except AttributeError:
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	97 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	98 profile = args.locals["self"].parent.profile
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	99 except AttributeError:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	100 profile = args.locals[
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	101 "self"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	102 ].host.profile # used in quick_frontend for single profile configuration
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	103 except Exception:
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	104 # we can't find profile, we return an empty value
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	105 profile = ""
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	106 return profile
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	107
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	108
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	109 class ConfigureTwisted(ConfigureBasic):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	110 LOGGER_CLASS = TwistedLogger
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	111
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	112 def preTreatment(self):
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	113 from twisted import logger
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	114 global logger
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	115 self.level_map = {
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	116 C.LOG_LVL_DEBUG: logger.LogLevel.debug,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	117 C.LOG_LVL_INFO: logger.LogLevel.info,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	118 C.LOG_LVL_WARNING: logger.LogLevel.warn,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	119 C.LOG_LVL_ERROR: logger.LogLevel.error,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	120 C.LOG_LVL_CRITICAL: logger.LogLevel.critical,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	121 }
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	122 self.LOGGER_CLASS.level_map = self.level_map
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	123
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	124 def configureLevel(self, level):
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	125 self.level = self.level_map[level]
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	126
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	127 def configureOutput(self, output):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	128 import sys
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	129 from twisted.python import logfile
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	130 self.log_publisher = logger.LogPublisher()
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	131
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	132 if output is None:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	133 output = C.LOG_OPT_OUTPUT_SEP + C.LOG_OPT_OUTPUT_DEFAULT
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	134 self.manageOutputs(output)
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	135
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	136 if C.LOG_OPT_OUTPUT_DEFAULT in log.handlers:
1129 08f50fdac21b core (logging): new backend_data parameter can be used to transmit specific data to a backend + Twisted backend use this option to know if we are in debug or nodaemon mode Goffi <goffi@goffi.org> parents: 1121 diff changeset	137 if self.backend_data is None:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	138 raise ValueError(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	139 "You must pass options as backend_data with Twisted backend"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	140 )
1129 08f50fdac21b core (logging): new backend_data parameter can be used to transmit specific data to a backend + Twisted backend use this option to know if we are in debug or nodaemon mode Goffi <goffi@goffi.org> parents: 1121 diff changeset	141 options = self.backend_data
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	142 log_file = logfile.LogFile.fromFullPath(options['logfile'])
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	143 self.log_publisher.addObserver(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	144 logger.FileLogObserver(log_file, self.textFormatter))
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	145 # we also want output to stdout if we are in debug or nodaemon mode
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	146 if options.get("nodaemon", False) or options.get("debug", False):
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	147 self.log_publisher.addObserver(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	148 logger.FileLogObserver(sys.stdout, self.textFormatter))
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	149
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	150 if C.LOG_OPT_OUTPUT_FILE in log.handlers:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	151
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	152 for path in log.handlers[C.LOG_OPT_OUTPUT_FILE]:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	153 log_file = (
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	154 sys.stdout if path == "-" else logfile.LogFile.fromFullPath(path)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	155 )
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	156 self.log_publisher.addObserver(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	157 logger.FileLogObserver(log_file, self.textFormatter))
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	158
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	159 if C.LOG_OPT_OUTPUT_MEMORY in log.handlers:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	160 raise NotImplementedError(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	161 "Memory observer is not implemented in Twisted backend"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	162 )
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	163
1942 7f053e1f0b67 core (logs): taints: Goffi <goffi@goffi.org> parents: 1941 diff changeset	164 def configureColors(self, colors, force_colors, levels_taints_dict):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	165 super(ConfigureTwisted, self).configureColors(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	166 colors, force_colors, levels_taints_dict
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	167 )
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	168 self.LOGGER_CLASS.colors = colors
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	169 self.LOGGER_CLASS.force_colors = force_colors
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	170 if force_colors and not colors:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	171 raise ValueError("colors must be True if force_colors is True")
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	172
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	173 def postTreatment(self):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	174 """Install twistedObserver which manage non SàT logs"""
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	175 # from twisted import logger
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	176 import sys
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	177 filtering_obs = logger.FilteringLogObserver(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	178 observer=self.log_publisher,
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	179 predicates=[
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	180 logger.LogLevelFilterPredicate(self.level),
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	181 ]
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	182 )
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	183 logger.globalLogBeginner.beginLoggingTo([filtering_obs])
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	184
3280 96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	185 def textFormatter(self, event):
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	186 if event.get('sat_logged', False):
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	187 timestamp = ''.join([logger.formatTime(event.get("log_time", None)), " "])
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	188 return f"{timestamp}{event.get('log_format', '')}\n"
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	189 else:
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	190 eventText = logger.eventAsText(
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	191 event, includeSystem=True)
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	192 if not eventText:
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	193 return None
96b9b65b4368 core (log): logging with Twisted now uses the new twisted.logger Goffi <goffi@goffi.org> parents: 3137 diff changeset	194 return eventText.replace("\n", "\n\t") + "\n"
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	195
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	196
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	197 class ConfigureStandard(ConfigureBasic):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	198 def __init__(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	199 self,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	200 level=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	201 fmt=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	202 output=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	203 logger=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	204 colors=False,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	205 levels_taints_dict=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	206 force_colors=False,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	207 backend_data=None,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	208 ):
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	209 if fmt is None:
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	210 fmt = C.LOG_OPT_FORMAT[1]
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	211 if output is None:
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	212 output = C.LOG_OPT_OUTPUT[1]
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	213 super(ConfigureStandard, self).__init__(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	214 level,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	215 fmt,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	216 output,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	217 logger,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	218 colors,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	219 levels_taints_dict,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	220 force_colors,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	221 backend_data,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	222 )
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	223
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	224 def preTreatment(self):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	225 """We use logging methods directly, instead of using Logger"""
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	226 import logging
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	227
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	228 log.getLogger = logging.getLogger
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	229 log.debug = logging.debug
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	230 log.info = logging.info
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	231 log.warning = logging.warning
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	232 log.error = logging.error
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	233 log.critical = logging.critical
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	234
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	235 def configureLevel(self, level):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	236 if level is None:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	237 level = C.LOG_LVL_DEBUG
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	238 self.level = level
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	239
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	240 def configureFormat(self, fmt):
1940 3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	241 super(ConfigureStandard, self).configureFormat(fmt)
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	242 import logging
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	243
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	244 class SatFormatter(logging.Formatter):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2836 diff changeset	245 """Formatter which manage SàT specificities"""
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	246 _format = fmt
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	247 _with_profile = "%(profile)s" in fmt
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	248
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	249 def __init__(self, can_colors=False):
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	250 super(SatFormatter, self).__init__(self._format)
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	251 self.can_colors = can_colors
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	252
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	253 def format(self, record):
1013 11409a6c16c7 core (log/standard backend): added "%(profile)s" format management Goffi <goffi@goffi.org> parents: 1012 diff changeset	254 if self._with_profile:
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	255 record.profile = ConfigureStandard.getProfile()
1940 3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	256 do_color = self.with_colors and (self.can_colors or self.force_colors)
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	257 if ConfigureStandard._color_location:
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	258 # we copy raw formatting strings for color_*
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	259 # as formatting is handled in ansiColors in this case
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	260 if do_color:
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	261 record.color_start = log.COLOR_START
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	262 record.color_end = log.COLOR_END
3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	263 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	264 record.color_start = record.color_end = ""
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	265 s = super(SatFormatter, self).format(record)
1940 3fdacba9da68 core (logs): log color location can now be specified with %(color_start)s and %(color_end)s Goffi <goffi@goffi.org> parents: 1934 diff changeset	266 if do_color:
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	267 s = ConfigureStandard.ansiColors(record.levelname, s)
3056 49c7bc3afb2b core (log): removed Android hack: Goffi <goffi@goffi.org> parents: 3028 diff changeset	268 return s
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	269
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	270 self.formatterClass = SatFormatter
1006 325fd230c15d core (log): added advanced feature to basic backend (colors/formatting/level and logger filtering) Goffi <goffi@goffi.org> parents: 1005 diff changeset	271
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	272 def configureOutput(self, output):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	273 self.manageOutputs(output)
1006 325fd230c15d core (log): added advanced feature to basic backend (colors/formatting/level and logger filtering) Goffi <goffi@goffi.org> parents: 1005 diff changeset	274
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	275 def configureLogger(self, logger):
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	276 self.name_filter = log.FilterName(logger) if logger else None
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	277
1942 7f053e1f0b67 core (logs): taints: Goffi <goffi@goffi.org> parents: 1941 diff changeset	278 def configureColors(self, colors, force_colors, levels_taints_dict):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	279 super(ConfigureStandard, self).configureColors(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	280 colors, force_colors, levels_taints_dict
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	281 )
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	282 self.formatterClass.with_colors = colors
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	283 self.formatterClass.force_colors = force_colors
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	284 if not colors and force_colors:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	285 raise ValueError("force_colors can't be used if colors is False")
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	286
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	287 def _addHandler(self, root_logger, hdlr, can_colors=False):
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	288 hdlr.setFormatter(self.formatterClass(can_colors))
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	289 root_logger.addHandler(hdlr)
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	290 root_logger.setLevel(self.level)
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	291 if self.name_filter is not None:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	292 hdlr.addFilter(self.name_filter)
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	293
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	294 def postTreatment(self):
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	295 import logging
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	296
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	297 root_logger = logging.getLogger()
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	298 if len(root_logger.handlers) == 0:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2836 diff changeset	299 for handler, options in list(log.handlers.items()):
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	300 if handler == C.LOG_OPT_OUTPUT_DEFAULT:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	301 hdlr = logging.StreamHandler()
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	302 try:
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	303 can_colors = hdlr.stream.isatty()
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	304 except AttributeError:
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	305 can_colors = False
c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	306 self._addHandler(root_logger, hdlr, can_colors=can_colors)
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	307 elif handler == C.LOG_OPT_OUTPUT_MEMORY:
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	308 from logging.handlers import BufferingHandler
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	309
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	310 class SatMemoryHandler(BufferingHandler):
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	311 def emit(self, record):
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	312 super(SatMemoryHandler, self).emit(self.format(record))
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	313
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	314 hdlr = SatMemoryHandler(options)
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	315 log.handlers[
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	316 handler
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	317 ] = (
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	318 hdlr
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	319 ) # we keep a reference to the handler to read the buffer later
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	320 self._addHandler(root_logger, hdlr, can_colors=False)
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	321 elif handler == C.LOG_OPT_OUTPUT_FILE:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	322 import os.path
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	323
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	324 for path in options:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	325 hdlr = logging.FileHandler(os.path.expanduser(path))
1012 c8771279497e core(log): standard backend: colors are now checked on a per handler basis Goffi <goffi@goffi.org> parents: 1010 diff changeset	326 self._addHandler(root_logger, hdlr, can_colors=False)
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	327 else:
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	328 raise ValueError("Unknown handler type")
a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	329 else:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2836 diff changeset	330 root_logger.warning("Handlers already set on root logger")
1007 a7d33c7a8277 core (log): refactoring + twisted backend: Goffi <goffi@goffi.org> parents: 1006 diff changeset	331
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	332 @staticmethod
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	333 def memoryGet(size=None):
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	334 """Return buffered logs
1006 325fd230c15d core (log): added advanced feature to basic backend (colors/formatting/level and logger filtering) Goffi <goffi@goffi.org> parents: 1005 diff changeset	335
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	336 @param size: number of logs to return
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	337 """
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	338 mem_handler = log.handlers[C.LOG_OPT_OUTPUT_MEMORY]
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	339 return (
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	340 log_msg for log_msg in mem_handler.buffer[size if size is None else -size :]
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	341 )
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	342
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	343
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	344 log.configure_cls[C.LOG_BACKEND_BASIC] = ConfigureBasic
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	345 log.configure_cls[C.LOG_BACKEND_TWISTED] = ConfigureTwisted
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	346 log.configure_cls[C.LOG_BACKEND_STANDARD] = ConfigureStandard
a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	347
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	348
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	349 def configure(backend, **options):
1006 325fd230c15d core (log): added advanced feature to basic backend (colors/formatting/level and logger filtering) Goffi <goffi@goffi.org> parents: 1005 diff changeset	350 """Configure logging behaviour
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	351 @param backend: can be:
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	352 C.LOG_BACKEND_STANDARD: use standard logging module
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	353 C.LOG_BACKEND_TWISTED: use twisted logging module (with standard logging observer)
05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	354 C.LOG_BACKEND_BASIC: use a basic print based logging
1017 0ea97f483464 core (log): added "custom" backend Goffi <goffi@goffi.org> parents: 1016 diff changeset	355 C.LOG_BACKEND_CUSTOM: use a given Logger subclass
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	356 """
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	357 return log.configure(backend, **options)
991 05e02f8b7eb4 core: logging refactoring, first step: Goffi <goffi@goffi.org> parents: diff changeset	358
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	359
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	360 def _parseOptions(options):
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	361 """Parse string options as given in conf or environment variable, and return expected python value
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	362
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	363 @param options (dict): options with (key: name, value: string value)
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	364 """
1005 b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	365 COLORS = C.LOG_OPT_COLORS[0]
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	366 LEVEL = C.LOG_OPT_LEVEL[0]
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	367
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	368 if COLORS in options:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	369 if options[COLORS].lower() in ("1", "true"):
1005 b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	370 options[COLORS] = True
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	371 elif options[COLORS] == "force":
1005 b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	372 options[COLORS] = True
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	373 options["force_colors"] = True
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	374 else:
1005 b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	375 options[COLORS] = False
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	376 if LEVEL in options:
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	377 level = options[LEVEL].upper()
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	378 if level not in C.LOG_LEVELS:
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	379 level = C.LOG_LVL_INFO
b4af31a8a4f2 core (logs): added formatting, name filter and outputs management: Goffi <goffi@goffi.org> parents: 994 diff changeset	380 options[LEVEL] = level
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	381
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	382
1129 08f50fdac21b core (logging): new backend_data parameter can be used to transmit specific data to a backend + Twisted backend use this option to know if we are in debug or nodaemon mode Goffi <goffi@goffi.org> parents: 1121 diff changeset	383 def satConfigure(backend=C.LOG_BACKEND_STANDARD, const=None, backend_data=None):
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	384 """Configure logging system for SàT, can be used by frontends
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	385
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	386 logs conf is read in SàT conf, then in environment variables. It must be done before Memory init
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	387 @param backend: backend to use, it can be:
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	388 - C.LOG_BACKEND_BASIC: print based backend
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	389 - C.LOG_BACKEND_TWISTED: Twisted logging backend
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	390 - C.LOG_BACKEND_STANDARD: standard logging backend
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	391 @param const: Const class to use instead of sat.core.constants.Const (mainly used to change default values)
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	392 """
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	393 if const is not None:
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	394 global C
73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	395 C = const
1021 a836b6da2c5c core (log): moved configuration to core.log_config; this avoid import issues with pyjamas. Goffi <goffi@goffi.org> parents: 1017 diff changeset	396 log.C = const
1941 befdcfc55569 core (logs): use tools.config to handle sat.conf Goffi <goffi@goffi.org> parents: 1940 diff changeset	397 from sat.tools import config
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	398 import os
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	399
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	400 log_conf = {}
1941 befdcfc55569 core (logs): use tools.config to handle sat.conf Goffi <goffi@goffi.org> parents: 1940 diff changeset	401 sat_conf = config.parseMainConf()
1010 73a0b7f94674 primitivus: use of new logging system: Goffi <goffi@goffi.org> parents: 1008 diff changeset	402 for opt_name, opt_default in C.LOG_OPTIONS():
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	403 try:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	404 log_conf[opt_name] = os.environ[
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	405 "".join((C.ENV_PREFIX, C.LOG_OPT_PREFIX.upper(), opt_name.upper()))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	406 ]
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	407 except KeyError:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	408 log_conf[opt_name] = config.getConfig(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	409 sat_conf, C.LOG_OPT_SECTION, C.LOG_OPT_PREFIX + opt_name, opt_default
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	410 )
994 652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	411
652c01ca69b1 core (log): configuration and environment variables are now checked for log level and colors: Goffi <goffi@goffi.org> parents: 992 diff changeset	412 _parseOptions(log_conf)
1129 08f50fdac21b core (logging): new backend_data parameter can be used to transmit specific data to a backend + Twisted backend use this option to know if we are in debug or nodaemon mode Goffi <goffi@goffi.org> parents: 1121 diff changeset	413 configure(backend, backend_data=backend_data, **log_conf)

Mercurial > libervia-backend

annotate sat/core/log_config.py @ 3370:2157880ba3b4