annotate tests/e2e/run_e2e.py @ 3429:d4558f3cbf13

tests, docker(e2e): added e2e tests for Libervia: - moved jp tests to `e2e/jp` - new fixtures - adapted docker-compose - improved `run_e2e` with several flags + report on failure - doc to come
author Goffi <goffi@goffi.org>
date Fri, 27 Nov 2020 16:39:40 +0100
parents 814e118d9ef3
children be6d91572633
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # SàT: an XMPP client
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2020 Jérôme Poisson (goffi@goffi.org)
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 """Run end-to-end tests in appropriate Docker environment"""
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
20
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 import sys
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from pathlib import Path
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 import tempfile
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
24 from textwrap import dedent
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
25 from datetime import datetime
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
26 import sh
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
27 import io
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
28 import re
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
29 import sat_templates
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
30 import libervia
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 from sat.core import exceptions
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
32 import yaml
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
33 try:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
34 from yaml import CLoader as Loader, CDumper as Dumper
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
35 except ImportError:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
36 from yaml import Loader, Dumper
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
37
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
38
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
39 OPT_KEEP_CONTAINERS = "--keep-containers"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
40 OPT_KEEP_PROFILES = "--keep-profiles"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
41 OPT_KEEP_VNC = "--keep-vnc"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
42 OPT_KEEP_BROWSER = "--keep-browser"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
43 OPT_VISUAL = "--visual"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
44 OPT_DEV_MODE = "--dev-mode"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
45
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
46 dev_mode_inst = dedent("""\
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
47 Here is a short script to start working with a logged account:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
48
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
49 from helium import *
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
50 start_firefox()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
51 go_to("https://libervia.test:8443/login")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
52 write("account1", "login")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
53 write("test", "password")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
54 click("log in")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
55 """)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
56 report_buffer = io.StringIO()
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
57
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
58
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
59 def live_out(data):
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 sys.stdout.write(data)
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 sys.stdout.flush()
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
62 report_buffer.write(data)
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
63
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
64
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
65 def live_err(data):
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 sys.stderr.write(data)
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 sys.stderr.flush()
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
68 report_buffer.write(data)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
69
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
70
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
71 def get_opt(opt_name):
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
72 """Check is an option flag is set, and remove it for sys.argv
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
73
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
74 This allow to have simple flags without interfering with pytest options
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
75 """
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
76 if opt_name in sys.argv:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
77 sys.argv.remove(opt_name)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
78 return True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
79 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
80 return False
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
81
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
82
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
83 def set_env(override, name, value="1"):
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
84 """Set environement variable"""
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
85 environment = override["services"]["sat"].setdefault("environment", {})
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
86 environment[name] = value
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
87
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
88
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
89 def use_e2e_env():
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
90 visual = get_opt(OPT_VISUAL)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
91 keep_containers = get_opt(OPT_KEEP_CONTAINERS)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
92 keep_profiles = get_opt(OPT_KEEP_PROFILES)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
93 keep_vnc = get_opt(OPT_KEEP_VNC)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
94 keep_browser = get_opt(OPT_KEEP_BROWSER)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
95 if keep_browser:
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
96 keep_containers = True
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
97 keep_vnc = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
98 if keep_vnc:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
99 visual = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
100 dev_mode = get_opt(OPT_DEV_MODE)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
101 if dev_mode:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
102 keep_containers = keep_profiles = keep_vnc = visual = True
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
103
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
104 for p in Path.cwd().parents:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
105 package_path = p / "sat"
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
106 docker_path = p / "docker"
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
107 if package_path.is_dir() and docker_path.is_dir():
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
108 sat_root_path = p
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
109 break
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
110 else:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
111 raise exceptions.NotFound(
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
112 "Can't find root of SàT code, are you sure that you are running the test "
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
113 "from the backend repository?"
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
114 )
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
115
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
116 libervia_path = Path(libervia.__file__).parent.resolve()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
117 libervia_root_path = libervia_path.parent
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
118 if (libervia_root_path / ".hg").is_dir():
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
119 libervia_source = libervia_root_path
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
120 libervia_target = "/src/libervia"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
121 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
122 libervia_source = libervia_path
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
123 libervia_target = "/src/libervia/libervia"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
124
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
125 sat_templates_path = Path(sat_templates.__file__).parent.resolve()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
126 sat_templates_root_path = sat_templates_path.parent
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
127 if (sat_templates_root_path / ".hg").is_dir():
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
128 sat_templates_source = sat_templates_root_path
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
129 sat_templates_target = "/src/sat_templates"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
130 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
131 sat_templates_source = sat_templates_path
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
132 sat_templates_target = "/src/sat_templates/sat_templates"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
133
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
134 compose_e2e_path = docker_path / "docker-compose_e2e.yml"
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
135 if not compose_e2e_path.is_file():
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
136 raise exceptions.NotFound('"docker-compose_e2e.yml" file can\'t be found')
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
137
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
138 with tempfile.TemporaryDirectory(prefix="sat_test_e2e_") as temp_dir:
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
139 override_path = Path(temp_dir) / "test_override.yml"
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
140 override = yaml.load(
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
141 dedent(f"""\
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
142 version: "3.6"
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
143 services:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
144 sat:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
145 volumes:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
146 - type: bind
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
147 source: {sat_root_path}
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
148 target: /src/sat
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
149 read_only: true
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
150 libervia:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
151 volumes:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
152 - type: bind
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
153 source: {sat_root_path}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
154 target: /src/sat
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
155 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
156 - type: bind
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
157 source: {libervia_source}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
158 target: {libervia_target}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
159 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
160 - type: bind
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
161 source: {sat_templates_source}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
162 target: {sat_templates_target}
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
163 read_only: true
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
164 """
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
165 ),
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
166 Loader=Loader
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
167 )
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
168
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
169 if keep_profiles:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
170 set_env(override, "SAT_TEST_E2E_KEEP_PROFILES")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
171
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
172 if visual:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
173 set_env(override, "SAT_TEST_E2E_LIBERVIA_NO_HEADLESS")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
174
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
175 if keep_browser:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
176 set_env(override, "SAT_TEST_E2E_LIBERVIA_KEEP_BROWSER")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
177
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
178 with override_path.open("w") as f:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
179 yaml.dump(override, f, Dumper=Dumper)
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
180
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
181 docker_compose = sh.docker_compose.bake(
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
182 "-f", compose_e2e_path, "-f", override_path)
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
183 docker_compose.up("-d")
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
184
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
185 p = docker_compose.exec(
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
186 "-T", "--workdir", "/src/sat/tests", "sat",
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
187 "pytest", "-o", "cache_dir=/tmp", *sys.argv[1:], color="yes",
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
188 _in=sys.stdin, _out=live_out, _out_bufsize=0, _err=live_err, _err_bufsize=0,
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
189 _bg=True
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
190 )
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
191 if visual:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
192 vnc_port = docker_compose.port("sat", "5900").split(':', 1)[1].strip()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
193 p_vnc = sh.vncviewer(
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
194 f"localhost:{vnc_port}",
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
195 _bg=True,
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
196 # vncviewer exits with 1 when we send an SIGTERM to it, and it's printed
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
197 # before we can catch it (it's happening in a thread). Thus we set exit
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
198 # code 1 as OK to avoid the backtrace.
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
199 _ok_code=[0, 1]
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
200 )
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
201 else:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
202 p_vnc = None
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
203
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
204 try:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
205 p.wait()
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
206 except sh.ErrorReturnCode as e:
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
207 sat_cont_id = docker_compose.ps("-q", "sat").strip()
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
208 report_dest = Path(f"reports_{datetime.now().isoformat()}/")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
209 # we need to make `report_dest` explicitely local with "./", otherwise
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
210 # docker parse takes it as a container path due to the presence of ":"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
211 # with `isoformat()`.
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
212 sh.docker.cp(f"{sat_cont_id}:/reports", f"./{report_dest}")
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
213 # we save 2 versions: one with ANSI escape codes
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
214 report_ansi = report_dest / "report.ansi"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
215 with report_ansi.open('w') as f:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
216 f.write(report_buffer.getvalue())
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
217 # and one without (cf. https://stackoverflow.com/a/14693789)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
218 ansi_escape = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
219 report_log = report_dest / "report.log"
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
220 with report_log.open('w') as f:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
221 f.write(ansi_escape.sub('', report_buffer.getvalue()))
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
222
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
223 print(f"report saved to {report_dest}")
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
224 sys.exit(e.exit_code)
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
225 finally:
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
226 if p_vnc is not None and p_vnc.is_alive() and not keep_vnc:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
227 p_vnc.terminate()
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
228 if not keep_containers:
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
229 docker_compose.down(volumes=True)
3429
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
230 if dev_mode:
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
231 print(dev_mode_inst)
d4558f3cbf13 tests, docker(e2e): added e2e tests for Libervia:
Goffi <goffi@goffi.org>
parents: 3415
diff changeset
232
3415
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
233
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
234 if __name__ == "__main__":
814e118d9ef3 tests: end-2-end tests first draft:
Goffi <goffi@goffi.org>
parents:
diff changeset
235 use_e2e_env()