libervia-web: libervia/web/server/tasks/manager.py annotate

annotate libervia/web/server/tasks/manager.py @ 1603:e105d7719479

doc (user/calls): Add a section to explain remote control: fix 436

author	Goffi <goffi@goffi.org>
date	Sat, 11 May 2024 14:02:54 +0200
parents	eb00d593801d
children

rev	line source
1239 f511f8fbbf8a fixed shebangs Goffi <goffi@goffi.org> parents: 1237 diff changeset	1 #!/usr/bin/env python3
f511f8fbbf8a fixed shebangs Goffi <goffi@goffi.org> parents: 1237 diff changeset	2
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia: a Salut à Toi frontend
1396 822bd0139769 date update Goffi <goffi@goffi.org> parents: 1364 diff changeset	4 # Copyright (C) 2011-2021 Jérôme Poisson <goffi@goffi.org>
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	18 import os
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 import os.path
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	20 from pathlib import Path
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	21 from typing import Dict
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	22 import importlib.util
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	23 from twisted.internet import defer
1518 eb00d593801d refactoring: rename `libervia` to `libervia.web` + update imports following backend changes Goffi <goffi@goffi.org> parents: 1509 diff changeset	24 from libervia.backend.core.log import getLogger
eb00d593801d refactoring: rename `libervia` to `libervia.web` + update imports following backend changes Goffi <goffi@goffi.org> parents: 1509 diff changeset	25 from libervia.backend.core import exceptions
eb00d593801d refactoring: rename `libervia` to `libervia.web` + update imports following backend changes Goffi <goffi@goffi.org> parents: 1509 diff changeset	26 from libervia.backend.core.i18n import _
eb00d593801d refactoring: rename `libervia` to `libervia.web` + update imports following backend changes Goffi <goffi@goffi.org> parents: 1509 diff changeset	27 from libervia.backend.tools import utils
eb00d593801d refactoring: rename `libervia` to `libervia.web` + update imports following backend changes Goffi <goffi@goffi.org> parents: 1509 diff changeset	28 from libervia.web.server.constants import Const as C
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	29 from . import implicit
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	30 from .task import Task
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	31
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	32 log = getLogger(__name__)
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	33
1252 80a92eb82b7f server (tasks manager): added a label for default site Goffi <goffi@goffi.org> parents: 1247 diff changeset	34 DEFAULT_SITE_LABEL = _("default site")
80a92eb82b7f server (tasks manager): added a label for default site Goffi <goffi@goffi.org> parents: 1247 diff changeset	35
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	36
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	37 class TasksManager:
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	38 """Handle tasks of a Libervia site"""
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	39
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	40 def __init__(self, host, site_resource):
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	41 """
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 @param site_resource(LiberviaRootResource): root resource of the site to manage
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	43 """
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	44 self.host = host
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	45 self.resource = site_resource
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	46 self.tasks_dir = self.site_path / C.TASKS_DIR
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	47 self.tasks = {}
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 self._build_path = None
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	49 self._current_task = None
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	50
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 @property
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	52 def site_path(self):
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	53 return Path(self.resource.site_path)
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	54
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	55 @property
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	56 def build_path(self):
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	57 """path where generated files will be build for this site"""
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	58 if self._build_path is None:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	59 self._build_path = self.host.get_build_path(self.site_name)
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	60 return self._build_path
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	61
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	62 @property
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	63 def site_name(self):
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	64 return self.resource.site_name
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	65
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	66 def validate_data(self, task):
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	67 """Check workflow attributes in task"""
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	68
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	69 for var, allowed in (("ON_ERROR", ("continue", "stop")),
079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	70 ("LOG_OUTPUT", bool),
079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	71 ("WATCH_DIRS", list)):
079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	72 value = getattr(task, var)
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	73
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	74 if isinstance(allowed, type):
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	75 if allowed is list and value is None:
079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	76 continue
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	77 if not isinstance(value, allowed):
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	78 raise ValueError(
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	79 _("Unexpected value for {var}, {allowed} is expected.")
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	80 .format(var=var, allowed=allowed))
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	81 else:
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	82 if not value in allowed:
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	83 raise ValueError(_("Unexpected value for {var}: {value!r}").format(
079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	84 var=var, value=value))
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	85
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	86 async def import_task(
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	87 self,
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	88 task_name: str,
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	89 task_path: Path,
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	90 to_import: Dict[str, Path]
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	91 ) -> None:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	92 if task_name in self.tasks:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	93 log.debug(f"skipping task {task_name} which is already imported")
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	94 return
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	95 module_name = f"{self.site_name or C.SITE_NAME_DEFAULT}.task.{task_name}"
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	96
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	97 spec = importlib.util.spec_from_file_location(module_name, task_path)
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	98 task_module = importlib.util.module_from_spec(spec)
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	99 spec.loader.exec_module(task_module)
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	100 task = task_module.Task(self, task_name)
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	101 if task.AFTER is not None:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	102 for pre_task_name in task.AFTER:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	103 log.debug(
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	104 f"task {task_name!r} must be run after {pre_task_name!r}")
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	105 try:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	106 pre_task_path = to_import[pre_task_name]
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	107 except KeyError:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	108 raise ValueError(
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	109 f"task {task_name!r} must be run after {pre_task_name!r}, "
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	110 f"however there is no task with such name")
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	111 await self.import_task(pre_task_name, pre_task_path, to_import)
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	112
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	113 # we launch prepare, which is a method used to prepare
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	114 # data at runtime (e.g. set WATCH_DIRS using config)
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	115 try:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	116 prepare = task.prepare
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	117 except AttributeError:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	118 pass
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	119 else:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	120 log.info(_('== preparing task "{task_name}" for {site_name} =='.format(
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	121 task_name=task_name, site_name=self.site_name or DEFAULT_SITE_LABEL)))
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	122 try:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	123 await utils.as_deferred(prepare)
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	124 except exceptions.CancelError as e:
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	125 log.debug(f"Skipping {task_name} which cancelled itself: {e}")
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	126 return
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	127
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	128 self.tasks[task_name] = task
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	129 self.validate_data(task)
1364 df40708c4c76 server: renamed `--dev_mode` to `--dev-mode` and set it as a flag: Goffi <goffi@goffi.org> parents: 1261 diff changeset	130 if self.host.options['dev-mode']:
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	131 dirs = task.WATCH_DIRS or []
a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	132 for dir_ in dirs:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	133 self.host.files_watcher.watch_dir(
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	134 dir_, auto_add=True, recursive=True,
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	135 callback=self._autorun_task, task_name=task_name)
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	136
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	137 async def parse_tasks_dir(self, dir_path: Path) -> None:
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	138 log.debug(f"parsing tasks in {dir_path}")
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	139 tasks_paths = sorted(dir_path.glob('task_*.py'))
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	140 to_import = {}
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	141 for task_path in tasks_paths:
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	142 if not task_path.is_file():
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	143 continue
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	144 task_name = task_path.stem[5:].lower().strip()
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	145 if not task_name:
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	146 continue
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	147 if task_name in self.tasks:
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	148 raise exceptions.ConflictError(
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	149 "A task with the name [{name}] already exists".format(
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	150 name=task_name))
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	151 log.debug(f"task {task_name} found")
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	152 to_import[task_name] = task_path
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	153
1261 a46d0e0f383b tasks: `AFTER` attribute to handle tasks order: Goffi <goffi@goffi.org> parents: 1260 diff changeset	154 for task_name, task_path in to_import.items():
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	155 await self.import_task(task_name, task_path, to_import)
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	156
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	157 async def parse_tasks(self):
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	158 # implicit tasks are always run
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	159 implicit_path = Path(implicit.__file__).parent
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	160 await self.parse_tasks_dir(implicit_path)
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	161 # now we check if there are tasks specific to this site
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	162 if not self.tasks_dir.is_dir():
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	163 log.debug(_("{name} has no task to launch.").format(
1252 80a92eb82b7f server (tasks manager): added a label for default site Goffi <goffi@goffi.org> parents: 1247 diff changeset	164 name = self.resource.site_name or DEFAULT_SITE_LABEL))
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	165 return
a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	166 else:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	167 await self.parse_tasks_dir(self.tasks_dir)
1247 a6c7f07f1e4d tasks: implicit tasks + Brython task: Goffi <goffi@goffi.org> parents: 1245 diff changeset	168
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	169 def _autorun_task(self, host, filepath, flags, task_name):
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	170 """Called when an event is received from a watched directory"""
813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	171 if flags == ['create']:
813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	172 return
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	173 try:
6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	174 task = self.tasks[task_name]
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	175 on_dir_event_cb = task.on_dir_event
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	176 except AttributeError:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	177 return defer.ensureDeferred(self.run_task(task_name))
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	178 else:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	179 return utils.as_deferred(
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	180 on_dir_event_cb, host, Path(filepath.path.decode()), flags)
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	181
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	182 async def run_task_instance(self, task: Task) -> None:
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	183 self._current_task = task.name
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	184 log.info(_('== running task "{task_name}" for {site_name} =='.format(
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	185 task_name=task.name, site_name=self.site_name or DEFAULT_SITE_LABEL)))
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	186 os.chdir(self.site_path)
813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	187 try:
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	188 await utils.as_deferred(task.start)
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	189 except Exception as e:
1245 079e8eb6e327 server (tasks): refactoring: Goffi <goffi@goffi.org> parents: 1239 diff changeset	190 on_error = task.ON_ERROR
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	191 if on_error == 'stop':
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	192 raise e
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	193 elif on_error == 'continue':
b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	194 log.warning(_('Task "{task_name}" failed for {site_name}: {reason}')
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	195 .format(task_name=task.name, site_name=self.site_name, reason=e))
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	196 else:
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	197 raise exceptions.InternalError("we should never reach this point")
1155 813d54af8c0c server (tasks): tasks can now be automatically ran when something happen in a watched dir: Goffi <goffi@goffi.org> parents: 1154 diff changeset	198 self._current_task = None
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	199
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	200 async def run_task(self, task_name: str) -> None:
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	201 """Run a single task
6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	202
6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	203 @param task_name(unicode): name of the task to run
6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	204 """
6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	205 task = self.tasks[task_name]
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	206 await self.run_task_instance(task)
1260 6161076193e0 tasks: dir event filter: Goffi <goffi@goffi.org> parents: 1252 diff changeset	207
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	208 async def run_tasks(self):
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 """Run all the tasks found"""
76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	210 old_path = os.getcwd()
1216 b2d067339de3 python 3 port: Goffi <goffi@goffi.org> parents: 1189 diff changeset	211 for task_name, task_value in self.tasks.items():
1509 106bae41f5c8 massive refactoring from camelCase -> snake_case. See backend commit log for more details Goffi <goffi@goffi.org> parents: 1396 diff changeset	212 await self.run_task(task_name)
1146 76d75423ef53 server: tasks manager first draft: Goffi <goffi@goffi.org> parents: diff changeset	213 os.chdir(old_path)

Mercurial > libervia-web

annotate libervia/web/server/tasks/manager.py @ 1603:e105d7719479