libervia-backend: tests/e2e/run

annotate tests/e2e/run_e2e.py @ 4208:fd9e78b7a0cd

doc (cli/call): add outputs documentation: outputs are now documentated, and notably the new `gui` one. fix 427

author	Goffi <goffi@goffi.org>
date	Sun, 11 Feb 2024 23:22:18 +0100
parents	4b842c1fb686
children	dfccc90cacc6

rev	line source
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
3480 7550ae9cfbac Renamed the project from "Salut à Toi" to "Libervia": Goffi <goffi@goffi.org> parents: 3479 diff changeset	3 # Libervia: an XMPP client
3479 be6d91572633 date update Goffi <goffi@goffi.org> parents: 3429 diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 """Run end-to-end tests in appropriate Docker environment"""
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	20
3662 0661b2aac167 tests (e2e): print revision used in `run_e2e` Goffi <goffi@goffi.org> parents: 3656 diff changeset	21 import sys, os
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	22 from pathlib import Path
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	23 import tempfile
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	24 from textwrap import dedent
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	25 from datetime import datetime
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	26 import sh
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	27 import io
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	28 import sat_templates
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	29 import libervia
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 3702 diff changeset	30 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 3702 diff changeset	31 from libervia.backend.tools.common import regex
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	32 import yaml
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	33 try:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	34 from yaml import CLoader as Loader, CDumper as Dumper
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	35 except ImportError:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	36 from yaml import Loader, Dumper
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	37
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	38
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	39 OPT_KEEP_CONTAINERS = "--keep-containers"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	40 OPT_KEEP_PROFILES = "--keep-profiles"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	41 OPT_KEEP_VNC = "--keep-vnc"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	42 OPT_KEEP_BROWSER = "--keep-browser"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	43 OPT_VISUAL = "--visual"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	44 OPT_DEV_MODE = "--dev-mode"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	45
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	46 dev_mode_inst = dedent("""\
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	47 Here is a short script to start working with a logged account:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	48
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	49 from helium import *
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	50 start_firefox()
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	51 go_to("https://libervia-web.test:8443/login")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	52 write("account1", "login")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	53 write("test", "password")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	54 click("log in")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	55 """)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	56 report_buffer = io.StringIO()
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	57 live_out_buf = []
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	58 live_err_buf = []
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	59
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	60
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	61 def live_out(data):
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	62 if live_out_buf:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	63 # we may get bytes when buffer is reached and we are in the middle of an unicode
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	64 # sequence. In this case we buffer it, and print it when it's complete
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	65 if isinstance(data, str):
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	66 data = b''.join(live_out_buf).decode() + data
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	67 live_out_buf.clear()
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	68 else:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	69 live_out_buf.append(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	70 return
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	71 try:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	72 sys.stdout.write(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	73 except TypeError:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	74 live_out_buf.append(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	75 return
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	76 sys.stdout.flush()
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	77 report_buffer.write(data)
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	78
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	79
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	80 def live_err(data):
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	81 if live_err_buf:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	82 if isinstance(data, str):
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	83 data = b''.join(live_err_buf).decode() + data
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	84 live_err_buf.clear()
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	85 else:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	86 live_err_buf.append(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	87 return
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	88 try:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	89 sys.stderr.write(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	90 except TypeError:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	91 live_err_buf.append(data)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	92 return
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	93 sys.stderr.flush()
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	94 report_buffer.write(data)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	95
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	96
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	97 def get_opt(opt_name):
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	98 """Check is an option flag is set, and remove it for sys.argv
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	99
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	100 This allow to have simple flags without interfering with pytest options
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	101 """
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	102 if opt_name in sys.argv:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	103 sys.argv.remove(opt_name)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	104 return True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	105 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	106 return False
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	107
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	108
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	109 def set_env(override, name, value="1"):
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	110 """Set environement variable"""
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	111 environment = override["services"]["backend"].setdefault("environment", {})
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	112 environment[name] = value
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	113
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	114 def write_report_log(path, log_raw, with_ansi=False):
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	115 log_raw = str(log_raw)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	116 if with_ansi:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	117 # we save 2 versions: one with ANSI escape codes
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	118 report_ansi = path.with_suffix(".ansi")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	119 with report_ansi.open('w') as f:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	120 f.write(log_raw)
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	121 # and one cleaner, without them
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	122 report_log = path.with_suffix(".log")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	123 with report_log.open('w') as f:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	124 f.write(regex.RE_ANSI_REMOVE.sub('', log_raw))
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	125 else:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	126 report_log = path.with_suffix(".log")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	127 with report_log.open('w') as f:
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	128 f.write(log_raw)
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	129
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 def use_e2e_env():
3662 0661b2aac167 tests (e2e): print revision used in `run_e2e` Goffi <goffi@goffi.org> parents: 3656 diff changeset	131 rev = os.getenv("DOCKER_LIBERVIA_REV", "dev")
0661b2aac167 tests (e2e): print revision used in `run_e2e` Goffi <goffi@goffi.org> parents: 3656 diff changeset	132 print(f"Running tests for Libervia {rev}")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	133 visual = get_opt(OPT_VISUAL)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	134 keep_containers = get_opt(OPT_KEEP_CONTAINERS)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	135 keep_profiles = get_opt(OPT_KEEP_PROFILES)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	136 keep_vnc = get_opt(OPT_KEEP_VNC)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	137 keep_browser = get_opt(OPT_KEEP_BROWSER)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	138 if keep_browser:
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	139 keep_containers = True
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	140 keep_vnc = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	141 if keep_vnc:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	142 visual = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	143 dev_mode = get_opt(OPT_DEV_MODE)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	144 if dev_mode:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	145 keep_containers = keep_profiles = keep_vnc = visual = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	146
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	147 for p in Path.cwd().parents:
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	148 package_path = p / "sat"
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	149 docker_path = p / "docker"
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	150 if package_path.is_dir() and docker_path.is_dir():
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	151 backend_root_path = p
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	152 break
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	153 else:
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	154 raise exceptions.NotFound(
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	155 "Can't find root of SàT code, are you sure that you are running the test "
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	156 "from the backend repository?"
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	157 )
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	158
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	159 libervia_web_path = Path(libervia.__file__).parent.resolve()
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	160 libervia_web_root_path = libervia_web_path.parent
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	161 if (libervia_web_root_path / ".hg").is_dir():
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	162 libervia_web_source = libervia_web_root_path
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	163 libervia_web_target = "/src/libervia-web"
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	164 else:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	165 libervia_web_source = libervia_web_path
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	166 libervia_web_target = "/src/libervia-web/libervia"
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	167
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	168 sat_templates_path = Path(sat_templates.__file__).parent.resolve()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	169 sat_templates_root_path = sat_templates_path.parent
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	170 if (sat_templates_root_path / ".hg").is_dir():
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	171 sat_templates_source = sat_templates_root_path
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	172 sat_templates_target = "/src/libervia-templates"
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	173 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	174 sat_templates_source = sat_templates_path
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	175 sat_templates_target = "/src/libervia-templates/sat_templates"
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	176
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	177 compose_e2e_path = docker_path / "docker-compose-e2e.yml"
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	178 if not compose_e2e_path.is_file():
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	179 raise exceptions.NotFound('"docker-compose-e2e.yml" file can\'t be found')
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	180
3630 8ee0140cc8b6 tests (run_e2e): update tmp dir prefix following name change Goffi <goffi@goffi.org> parents: 3498 diff changeset	181 with tempfile.TemporaryDirectory(prefix="libervia_test_e2e_") as temp_dir:
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	182 override_path = Path(temp_dir) / "test_override.yml"
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	183 override = yaml.load(
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	184 dedent(f"""\
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	185 version: "3.6"
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	186 services:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	187 backend:
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	188 volumes:
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	189 - type: bind
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	190 source: {backend_root_path}
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	191 target: /src/libervia-backend
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	192 read_only: true
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	193 web:
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	194 volumes:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	195 - type: bind
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	196 source: {backend_root_path}
3656 015e13f88960 tests (e2e): fix names in `run_e2e` script, following global name change. Goffi <goffi@goffi.org> parents: 3630 diff changeset	197 target: /src/libervia-backend
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	198 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	199 - type: bind
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	200 source: {libervia_web_source}
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	201 target: {libervia_web_target}
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	202 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	203 - type: bind
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	204 source: {sat_templates_source}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	205 target: {sat_templates_target}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	206 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	207 """
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	208 ),
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	209 Loader=Loader
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	210 )
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	211
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	212 if keep_profiles:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	213 set_env(override, "LIBERVIA_TEST_E2E_KEEP_PROFILES")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	214
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	215 if visual:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	216 set_env(override, "LIBERVIA_TEST_E2E_WEB_NO_HEADLESS")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	217
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	218 if keep_browser:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	219 set_env(override, "LIBERVIA_TEST_E2E_WEB_KEEP_BROWSER")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	220
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	221 with override_path.open("w") as f:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	222 yaml.dump(override, f, Dumper=Dumper)
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	223
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 docker_compose = sh.docker_compose.bake(
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	225 "-f", compose_e2e_path, "-f", override_path)
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 docker_compose.up("-d")
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	227
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	228 p = docker_compose.exec(
3702 6f9f4674f997 tests (e2e/run_e2e): fix inappropriate use of `-T` option Goffi <goffi@goffi.org> parents: 3662 diff changeset	229 "--workdir", "/src/libervia-backend/tests", "backend",
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	230 "pytest", "-o", "cache_dir=/tmp", *sys.argv[1:], color="yes",
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	231 _in=sys.stdin, _out=live_out, _out_bufsize=0, _err=live_err, _err_bufsize=0,
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	232 _bg=True
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	233 )
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	234 if visual:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	235 vnc_port = docker_compose.port("backend", "5900").split(':', 1)[1].strip()
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	236 p_vnc = sh.vncviewer(
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	237 f"localhost:{vnc_port}",
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	238 _bg=True,
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	239 # vncviewer exits with 1 when we send an SIGTERM to it, and it's printed
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	240 # before we can catch it (it's happening in a thread). Thus we set exit
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	241 # code 1 as OK to avoid the backtrace.
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	242 _ok_code=[0, 1]
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	243 )
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	244 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	245 p_vnc = None
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	246
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	247 try:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	248 p.wait()
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	249 except sh.ErrorReturnCode as e:
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	250 libervia_cont_id = docker_compose.ps("-q", "backend").strip()
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	251 report_dest = Path(f"report_{datetime.now().isoformat()}/")
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	252 # we need to make `report_dest` explicitely local with "./", otherwise
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	253 # docker parse takes it as a container path due to the presence of ":"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	254 # with `isoformat()`.
3498 d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	255 sh.docker.cp(f"{libervia_cont_id}:/reports", f"./{report_dest}")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	256 write_report_log(
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	257 report_dest/"report",
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	258 report_buffer.getvalue(),
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	259 with_ansi=True
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	260 )
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	261 write_report_log(
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	262 report_dest/"backend",
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	263 docker_compose.logs("--no-log-prefix", "backend")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	264 )
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	265 write_report_log(
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	266 report_dest/"web",
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	267 docker_compose.logs("--no-log-prefix", "web")
d78b5eae912a tests: update following names change Goffi <goffi@goffi.org> parents: 3480 diff changeset	268 )
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	269 print(f"report saved to {report_dest}")
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	270 sys.exit(e.exit_code)
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	271 finally:
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	272 if p_vnc is not None and p_vnc.is_alive() and not keep_vnc:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	273 p_vnc.terminate()
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	274 if not keep_containers:
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	275 docker_compose.down(volumes=True)
3429 d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	276 if dev_mode:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	277 print(dev_mode_inst)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia: Goffi <goffi@goffi.org> parents: 3415 diff changeset	278
3415 814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	279
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	280 if __name__ == "__main__":
814e118d9ef3 tests: end-2-end tests first draft: Goffi <goffi@goffi.org> parents: diff changeset	281 use_e2e_env()

Mercurial > libervia-backend

annotate tests/e2e/run_e2e.py @ 4208:fd9e78b7a0cd