Mercurial > libervia-backend
annotate doc/contribuing/testing.rst @ 3452:bb0225aaf4e6
plugin XEP-0346: "Form Discovery and Publishing" implementation:
this implementation replaces the former non standard node schema, and works in a similar
way (the schema is put in a separated node instead of a special field, thus it will now
work with most/all PubSub services, and not only SàT PubSub).
The implementation has been done in a way that nothing should be changed in frontends
(bridge methods names and arguments stay the same). The nodes are modified, but if values
are taken from backend, it's automatically adapted.
author | Goffi <goffi@goffi.org> |
---|---|
date | Fri, 11 Dec 2020 17:57:00 +0100 |
parents | b166a59b1724 |
children | d78b5eae912a |
rev | line source |
---|---|
3430
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
1 ======= |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
2 testing |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
3 ======= |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
4 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
5 You'll find here the documentation to run tests on Salut à Toi. If you plan to contribute |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
6 to the ecosystem, you should use them to check that your modification is not breaking |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
7 anything, and if possible you should extend them to cover any new feature. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
8 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
9 .. _contributing-overview: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
10 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
11 overview |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
12 ======== |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
13 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
14 Tests are run using `pytest`_ and are located in the ``tests`` directory. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
15 You'll also find legacy tests in ``sat/test`` but those one are old, not maintained and |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
16 only kept there temporarily until they are ported to the new system. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
17 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
18 For now, emphasis is put on end-2-end tests, as they are covering the whole ecosystem, and |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
19 are quite easy to write. The downside is that they are quite long to run. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
20 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
21 Several `fixtures`_ are available in the various ``conftest.py`` files, it's a good idea |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
22 to have an overview of them if you're willing to write your own tests. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
23 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
24 .. _pytest: https://www.pytest.org |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
25 .. _fixtures: https://docs.pytest.org/en/latest/fixture.html |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
26 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
27 end-to-end tests |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
28 ================ |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
29 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
30 End-to-end tests are located in ``tests/e2e``. They are launched in a well defined |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
31 environment managed through Docker. The ``docker/docker-compose_e2e.yml`` is used to |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
32 create the suitable containers. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
33 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
34 A script is available at ``tests/e2e/run_e2e.py`` to launch the tests. It will create the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
35 containers, bind the current code to them, and set environment variables according to |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
36 arguments. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
37 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
38 The arguments set to this script are the ``pytest`` options, thus you can have a verbose |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
39 mode with ``-v`` and select specific test with ``-k EXPRESSION`` (see ``pytest --help`` for |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
40 details). |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
41 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
42 In addition to pytest option, some flags can be set with the following arguments: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
43 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
44 ``--visual`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
45 Launch a VNC viewer to see in real time browser based tests. You must have ``vncviewer`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
46 executable available in your path (this is part of `TigerVNC`_) |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
47 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
48 ``--keep-containers`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
49 Do no remove Docker container after the end of tests. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
50 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
51 ``--keep-profiles`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
52 Do not delete test profiles after the end of tests |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
53 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
54 ``--keep-vnc`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
55 Do not stop VNC viewer after the end of tests. This argument implies ``--visual``. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
56 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
57 ``--keep-browser`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
58 Do not kill the browser inside the container after tests are done. This argument implies |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
59 ``--keep-container`` and ``--keep-vnc``. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
60 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
61 ``--dev-mode`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
62 Shortcut for ``--keep-containers``, ``--keep-profiles`` and ``--keep-vnc``. This is |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
63 useful, as you guess with its names, for development of tests. User can then log-in into |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
64 the ``sat`` container, launch a Python console, and work with the automated browser in |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
65 real-time. Basic commands to launch a browser and log-in with test account are printed |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
66 at the end of the tests. Note that if you want to have profiles created, or extra tools |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
67 like the fake SMTP server, you'll have to launch at least one test which require them. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
68 To log-in into the ``sat`` container, you can use the following command, from |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
69 ``/docker`` directory:: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
70 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
71 $ docker-compose -f docker-compose_e2e.yml exec sat /bin/bash |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
72 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
73 Then run a python console with given instructions |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
74 |
3439
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
75 It's also good to know that in the e2e test environment, the following pytest plugins are |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
76 installed and used: |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
77 |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
78 `pytest-timeout`_ |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
79 To avoid having test stuck, it's good to terminate them after a while. A timeout of 60s |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
80 is set by default for each test (lower value can give false negatives, as some e2e tests |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
81 can be long, notably with Selenium). |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
82 |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
83 `pytest-dependency`_ |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
84 Even if good testing practice normally means that tests can be run independently, in the |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
85 case of e2e tests we are using a real environment, and some tests do create files, |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
86 PubSub nodes, accounts, etc. It would be resource consuming to delete then recreate them |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
87 only to have standalone tests, thus to keep tests short and simple, some of them must be |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
88 run in order. The *dependecy* plugin is used to manage that, and will skip tests if one |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
89 of their dependencies is failing. The markup help also to document the tests order. |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
90 |
3430
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
91 .. _TigerVNC: https://tigervnc.org |
3439
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
92 .. _pytest-timeout: https://github.com/pytest-dev/pytest-timeout |
b166a59b1724
tests, doc(testing): added `pytest-dependency` plugin to `backend_e2e` image + doc
Goffi <goffi@goffi.org>
parents:
3431
diff
changeset
|
93 .. _pytest-dependency: https://github.com/RKrahl/pytest-dependency |
3430
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
94 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
95 common fixtures |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
96 --------------- |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
97 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
98 Here are the fixture common to all e2e tests which are good to know: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
99 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
100 ``test_profiles`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
101 Creates a bunch of test accounts which are available during the whole test session. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
102 Those account are destroyed once all the tests are finished (successful or not), except |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
103 if you set the ``SAT_TEST_E2E_KEEP_PROFILES`` environment variable (or use the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
104 ``--keep-profiles`` flag in ``run_e2e.py``. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
105 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
106 The profiles created are in the form ``accountX`` for account on the ``server1.test``, |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
107 or ``accountX_sY`` for account on other servers (see the docstring for details). |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
108 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
109 This fixture should be used on top of each e2e test module. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
110 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
111 ``pubsub_nodes`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
112 Create 2 pubsub nodes with ``open`` access model and named ``test`` (one on ``account1`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
113 PEP service, and the other one on ``pubsub.server1.test``, created with the same |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
114 account). |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
115 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
116 Those node are created for the scope of the class. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
117 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
118 ``fake_file`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
119 Create files filled with random bytes, and check them. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
120 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
121 A file is created by calling ``fake_file.size(size)``, and by default files of the same |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
122 size are re-used (set ``use_cache=False`` to create new files). This method returns a |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
123 ``pathlib.Path``. SHA-256 hash of the created file can be retrieved using |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
124 ``fake_file.get_source_hash(source_file_path)`` with the file path as argument. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
125 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
126 ``fake_file.new_dest_file()`` will return a Path to a randomly named destination file, |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
127 and ``fake_file.get_dest_hash(dest_file_path)`` will generate its hash once written. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
128 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
129 ``sent_emails`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
130 When used, a fake SMTP server (already configured in container's ``sat.conf``) will be |
3431 | 131 launched if it's not already, and all messages sent to it since the beginning of the test |
3430
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
132 will be available in the given list. Message are subclasses of |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
133 ``email.message.EmailMessage`` with the extra properties ``from_``, ``to``, ``subject`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
134 and ``body`` available for easy access to their content. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
135 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
136 The SMTP server is terminated at the end of the test session. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
137 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
138 jp e2e tests |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
139 ------------ |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
140 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
141 End-to-end tests for ``jp`` are a good way to tests backend features without having to |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
142 deal with frontends UI. Those tests use extensively the ``sh`` module, which helps |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
143 writing ``jp`` commands like if they where methods. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
144 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
145 Among the helping fixture (check the various ``conftest.py`` files for details), the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
146 following are specially good to know: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
147 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
148 ``jp_json`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
149 Set the ``json_raw`` output are parse it. When you use this instead of the normal ``jp``, |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
150 you'll get a Python object that you can manipulate easily. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
151 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
152 ``jp_elt`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
153 Set the ``xml_raw`` output and parse it as a Twisted ``domish.Element``. When you use a |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
154 command which can return XML, it is useful to get this object which is easy to |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
155 manipulate in Python. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
156 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
157 ``editor`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
158 Create a fake editor (replacing the one normally set in ``EDITOR`` environment |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
159 variable), to automatically modify and/or check the text sent by a command. You can |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
160 specify Python code to execute to modify the received text with the ``set_filter`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
161 method (this code is in a string which will be executed by Python interpreter, where the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
162 ``content`` variable is the received text). By default, the text is kept unmodified. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
163 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
164 After ``editor`` has been used by the ``jp`` command, you can check its |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
165 ``original_content`` property to see the text that it received, and ``new_content`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
166 property to see the text that has been written after updating the original content with |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
167 the code set in ``set_filter``. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
168 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
169 Libervia e2e tests |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
170 ------------------ |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
171 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
172 E2e tests for Libervia are executed, as it is common in web world, with `Selenium`_: user |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
173 actions are simulated in automated browser, and results are checked. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
174 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
175 To make the tests as easy to write as possible, and as natural to read as possible, the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
176 higher level `Helium`_ Python module is used. Thanks to it, the tests can read pretty much |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
177 like instructions we would give to a human user. Helium makes also easy to do some tasks |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
178 more complicated with Selenium alone, like dropping a file to an element. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
179 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
180 If a test is failing, a screenshot of the browser is taken. If you run the tests though |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
181 the ``run_e2e.py`` command (which you should), you'll find the screenshots in the |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
182 ``report_*`` directory which is created in working dir in case of failure. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
183 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
184 Here are the helping fixtures which are notably good to know, you should always use either |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
185 ``log_in_account1`` or ``nobody_logged_in``: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
186 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
187 ``log_in_account1`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
188 Start the test with the main test account logged. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
189 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
190 ``nobody_logged_in`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
191 Start the test without anybody logged (this is done by clearing all cookies). |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
192 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
193 .. _Selenium: https://www.selenium.dev |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
194 .. _Helium: https://github.com/mherrmann/selenium-python-helium |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
195 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
196 examples |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
197 -------- |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
198 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
199 Following examples have to be run from ``tests/e2e`` directory. |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
200 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
201 Run all tests for ``jp``:: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
202 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
203 $ ./run_e2e.py -k jp |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
204 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
205 Run all tests for ``Libervia`` with real-time visual feedback (note that you need to have |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
206 ``vncviewer`` installed and available in path, see above):: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
207 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
208 $ ./run_e2e.py -k libervia --visual |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
209 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
210 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
211 Run all tests with verbose mode (useful to know which test is currently running):: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
212 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
213 $ ./run_e2e.py -v |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
214 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
215 Run pubsub tests in verbose mode:: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
216 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
217 $ ./run_e2e.py -k pubsub -v |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
218 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
219 Run in dev mode, to work on new tests, note that we run the ``user_can_create_account`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
220 test to be sure to have test profiles created and fake SMTP server run…:: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
221 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
222 $ ./run_e2e.py -k user_can_create_account --dev-mode |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
223 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
224 …then to go into the ``sat`` container and work with the browser (to be run in ``docker`` |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
225 directory)…:: |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
226 |
3431 | 227 $ docker-compose -f docker-compose_e2e.yml exec sat /bin/bash |
3430
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
228 |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
229 …and, inside the container, you can now run ``python3`` and enter instruction prints at |
4ba31dd1f0f5
doc: documentation for testing in new `contributing` section
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
230 the end of the test session. |