Mercurial > libervia-backend
changeset 3661:cbb988a6f507
merge bookmark `@`
| author | Goffi <goffi@goffi.org> |
|---|---|
| date | Wed, 08 Sep 2021 11:20:37 +0200 |
| parents | 05db744f194f (current diff) d4b8f1e8f0b4 (diff) |
| children | 162866ca4be7 |
| files | doc/index.rst sat/core/sat_main.py sat/memory/persistent.py sat/plugins/plugin_xep_0277.py sat_frontends/jp/constants.py |
| diffstat | 31 files changed, 514 insertions(+), 387 deletions(-) [+] |
line wrap: on
line diff
--- a/doc/contribuing/index.rst Wed Sep 01 15:25:02 2021 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,15 +0,0 @@ -.. _contribuing: - -=========== -contribuing -=========== - -This part of the documentation is for people willing to contribute to the development of -Libervia and its ecosystem, it is not needed to read this if you only plan to use the -software. - -.. toctree:: - :glob: - :maxdepth: 2 - - testing
--- a/doc/contribuing/testing.rst Wed Sep 01 15:25:02 2021 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,230 +0,0 @@ -======= -testing -======= - -You'll find here the documentation to run tests on Libervia. If you plan to contribute -to the ecosystem, you should use them to check that your modification is not breaking -anything, and if possible you should extend them to cover any new feature. - -.. _contributing-overview: - -overview -======== - -Tests are run using `pytest`_ and are located in the ``tests`` directory. -You'll also find legacy tests in ``sat/test`` but those one are old, not maintained and -only kept there temporarily until they are ported to the new system. - -For now, emphasis is put on end-2-end tests, as they are covering the whole ecosystem, and -are quite easy to write. The downside is that they are quite long to run. - -Several `fixtures`_ are available in the various ``conftest.py`` files, it's a good idea -to have an overview of them if you're willing to write your own tests. - -.. _pytest: https://www.pytest.org -.. _fixtures: https://docs.pytest.org/en/latest/fixture.html - -end-to-end tests -================ - -End-to-end tests are located in ``tests/e2e``. They are launched in a well defined -environment managed through Docker. The ``docker/docker-compose_e2e.yml`` is used to -create the suitable containers. - -A script is available at ``tests/e2e/run_e2e.py`` to launch the tests. It will create the -containers, bind the current code to them, and set environment variables according to -arguments. - -The arguments set to this script are the ``pytest`` options, thus you can have a verbose -mode with ``-v`` and select specific test with ``-k EXPRESSION`` (see ``pytest --help`` for -details). - -In addition to pytest option, some flags can be set with the following arguments: - -``--visual`` - Launch a VNC viewer to see in real time browser based tests. You must have ``vncviewer`` - executable available in your path (this is part of `TigerVNC`_) - -``--keep-containers`` - Do no remove Docker container after the end of tests. - -``--keep-profiles`` - Do not delete test profiles after the end of tests - -``--keep-vnc`` - Do not stop VNC viewer after the end of tests. This argument implies ``--visual``. - -``--keep-browser`` - Do not kill the browser inside the container after tests are done. This argument implies - ``--keep-container`` and ``--keep-vnc``. - -``--dev-mode`` - Shortcut for ``--keep-containers``, ``--keep-profiles`` and ``--keep-vnc``. This is - useful, as you guess with its names, for development of tests. User can then log-in into - the ``backend`` container, launch a Python console, and work with the automated browser in - real-time. Basic commands to launch a browser and log-in with test account are printed - at the end of the tests. Note that if you want to have profiles created, or extra tools - like the fake SMTP server, you'll have to launch at least one test which require them. - To log-in into the ``backend`` container, you can use the following command, from - ``/docker`` directory:: - - $ docker-compose -f docker-compose_e2e.yml exec backend /bin/bash - - Then run a python console with given instructions - -It's also good to know that in the e2e test environment, the following pytest plugins are -installed and used: - -`pytest-timeout`_ - To avoid having test stuck, it's good to terminate them after a while. A timeout of 60s - is set by default for each test (lower value can give false negatives, as some e2e tests - can be long, notably with Selenium). - -`pytest-dependency`_ - Even if good testing practice normally means that tests can be run independently, in the - case of e2e tests we are using a real environment, and some tests do create files, - PubSub nodes, accounts, etc. It would be resource consuming to delete then recreate them - only to have standalone tests, thus to keep tests short and simple, some of them must be - run in order. The *dependecy* plugin is used to manage that, and will skip tests if one - of their dependencies is failing. The markup help also to document the tests order. - -.. _TigerVNC: https://tigervnc.org -.. _pytest-timeout: https://github.com/pytest-dev/pytest-timeout -.. _pytest-dependency: https://github.com/RKrahl/pytest-dependency - -common fixtures ---------------- - -Here are the fixture common to all e2e tests which are good to know: - -``test_profiles`` - Creates a bunch of test accounts which are available during the whole test session. - Those account are destroyed once all the tests are finished (successful or not), except - if you set the ``LIBERVIA_TEST_E2E_KEEP_PROFILES`` environment variable (or use the - ``--keep-profiles`` flag in ``run_e2e.py``. - - The profiles created are in the form ``accountX`` for account on the ``server1.test``, - or ``accountX_sY`` for account on other servers (see the docstring for details). - - This fixture should be used on top of each e2e test module. - -``pubsub_nodes`` - Create 2 pubsub nodes with ``open`` access model and named ``test`` (one on ``account1`` - PEP service, and the other one on ``pubsub.server1.test``, created with the same - account). - - Those node are created for the scope of the class. - -``fake_file`` - Create files filled with random bytes, and check them. - - A file is created by calling ``fake_file.size(size)``, and by default files of the same - size are re-used (set ``use_cache=False`` to create new files). This method returns a - ``pathlib.Path``. SHA-256 hash of the created file can be retrieved using - ``fake_file.get_source_hash(source_file_path)`` with the file path as argument. - - ``fake_file.new_dest_file()`` will return a Path to a randomly named destination file, - and ``fake_file.get_dest_hash(dest_file_path)`` will generate its hash once written. - -``sent_emails`` - When used, a fake SMTP server (already configured in container's ``libervia.conf``) will be - launched if it's not already, and all messages sent to it since the beginning of the test - will be available in the given list. Message are subclasses of - ``email.message.EmailMessage`` with the extra properties ``from_``, ``to``, ``subject`` - and ``body`` available for easy access to their content. - - The SMTP server is terminated at the end of the test session. - -libervia-cli e2e tests ----------------------- - -End-to-end tests for ``libervia-cli`` are a good way to tests backend features without having to -deal with frontends UI. Those tests use extensively the ``sh`` module, which helps -writing ``libervia-cli`` commands like if they where methods. - -Among the helping fixture (check the various ``conftest.py`` files for details), the -following are specially good to know: - -``li_json`` - Set the ``json_raw`` output are parse it. When you use this instead of the normal ``libervia-cli``, - you'll get a Python object that you can manipulate easily. - -``li_elt`` - Set the ``xml_raw`` output and parse it as a Twisted ``domish.Element``. When you use a - command which can return XML, it is useful to get this object which is easy to - manipulate in Python. - -``editor`` - Create a fake editor (replacing the one normally set in ``EDITOR`` environment - variable), to automatically modify and/or check the text sent by a command. You can - specify Python code to execute to modify the received text with the ``set_filter`` - method (this code is in a string which will be executed by Python interpreter, where the - ``content`` variable is the received text). By default, the text is kept unmodified. - - After ``editor`` has been used by the ``libervia-cli`` command, you can check its - ``original_content`` property to see the text that it received, and ``new_content`` - property to see the text that has been written after updating the original content with - the code set in ``set_filter``. - -Libervia e2e tests ------------------- - -E2e tests for Libervia are executed, as it is common in web world, with `Selenium`_: user -actions are simulated in automated browser, and results are checked. - -To make the tests as easy to write as possible, and as natural to read as possible, the -higher level `Helium`_ Python module is used. Thanks to it, the tests can read pretty much -like instructions we would give to a human user. Helium makes also easy to do some tasks -more complicated with Selenium alone, like dropping a file to an element. - -If a test is failing, a screenshot of the browser is taken. If you run the tests though -the ``run_e2e.py`` command (which you should), you'll find the screenshots in the -``report_*`` directory which is created in working dir in case of failure. - -Here are the helping fixtures which are notably good to know, you should always use either -``log_in_account1`` or ``nobody_logged_in``: - -``log_in_account1`` - Start the test with the main test account logged. - -``nobody_logged_in`` - Start the test without anybody logged (this is done by clearing all cookies). - -.. _Selenium: https://www.selenium.dev -.. _Helium: https://github.com/mherrmann/selenium-python-helium - -examples --------- - -Following examples have to be run from ``tests/e2e`` directory. - -Run all tests for ``Libervia CLI``:: - - $ ./run_e2e.py -k libervia-cli - -Run all tests for ``Libervia Web`` with real-time visual feedback (note that you need to have -``vncviewer`` installed and available in path, see above):: - - $ ./run_e2e.py -k libervia-web --visual - - -Run all tests with verbose mode (useful to know which test is currently running):: - - $ ./run_e2e.py -v - -Run pubsub tests in verbose mode:: - - $ ./run_e2e.py -k pubsub -v - -Run in dev mode, to work on new tests, note that we run the ``user_can_create_account`` -test to be sure to have test profiles created and fake SMTP server run…:: - - $ ./run_e2e.py -k user_can_create_account --dev-mode - -…then to go into the ``backend`` container and work with the browser (to be run in ``docker`` -directory)…:: - - $ docker-compose -f docker-compose_e2e.yml exec backend /bin/bash - -…and, inside the container, you can now run ``python3`` and enter instruction prints at -the end of the test session.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/contributing/index.rst Wed Sep 08 11:20:37 2021 +0200 @@ -0,0 +1,15 @@ +.. _contributing: + +============ +contributing +============ + +This part of the documentation is for people willing to contribute to the development of +Libervia and its ecosystem, it is not needed to read this if you only plan to use the +software. + +.. toctree:: + :glob: + :maxdepth: 2 + + testing
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/contributing/testing.rst Wed Sep 08 11:20:37 2021 +0200 @@ -0,0 +1,230 @@ +======= +testing +======= + +You'll find here the documentation to run tests on Libervia. If you plan to contribute +to the ecosystem, you should use them to check that your modification is not breaking +anything, and if possible you should extend them to cover any new feature. + +.. _contributing-overview: + +overview +======== + +Tests are run using `pytest`_ and are located in the ``tests`` directory. +You'll also find legacy tests in ``sat/test`` but those one are old, not maintained and +only kept there temporarily until they are ported to the new system. + +For now, emphasis is put on end-2-end tests, as they are covering the whole ecosystem, and +are quite easy to write. The downside is that they are quite long to run. + +Several `fixtures`_ are available in the various ``conftest.py`` files, it's a good idea +to have an overview of them if you're willing to write your own tests. + +.. _pytest: https://www.pytest.org +.. _fixtures: https://docs.pytest.org/en/latest/fixture.html + +end-to-end tests +================ + +End-to-end tests are located in ``tests/e2e``. They are launched in a well defined +environment managed through Docker. The ``docker/docker-compose-e2e.yml`` is used to +create the suitable containers. + +A script is available at ``tests/e2e/run_e2e.py`` to launch the tests. It will create the +containers, bind the current code to them, and set environment variables according to +arguments. + +The arguments set to this script are the ``pytest`` options, thus you can have a verbose +mode with ``-v`` and select specific test with ``-k EXPRESSION`` (see ``pytest --help`` for +details). + +In addition to pytest option, some flags can be set with the following arguments: + +``--visual`` + Launch a VNC viewer to see in real time browser based tests. You must have ``vncviewer`` + executable available in your path (this is part of `TigerVNC`_) + +``--keep-containers`` + Do no remove Docker container after the end of tests. + +``--keep-profiles`` + Do not delete test profiles after the end of tests + +``--keep-vnc`` + Do not stop VNC viewer after the end of tests. This argument implies ``--visual``. + +``--keep-browser`` + Do not kill the browser inside the container after tests are done. This argument implies + ``--keep-container`` and ``--keep-vnc``. + +``--dev-mode`` + Shortcut for ``--keep-containers``, ``--keep-profiles`` and ``--keep-vnc``. This is + useful, as you guess with its names, for development of tests. User can then log-in into + the ``backend`` container, launch a Python console, and work with the automated browser in + real-time. Basic commands to launch a browser and log-in with test account are printed + at the end of the tests. Note that if you want to have profiles created, or extra tools + like the fake SMTP server, you'll have to launch at least one test which require them. + To log-in into the ``backend`` container, you can use the following command, from + ``/docker`` directory:: + + $ docker-compose -f docker-compose-e2e.yml exec backend /bin/bash + + Then run a python console with given instructions + +It's also good to know that in the e2e test environment, the following pytest plugins are +installed and used: + +`pytest-timeout`_ + To avoid having test stuck, it's good to terminate them after a while. A timeout of 60s + is set by default for each test (lower value can give false negatives, as some e2e tests + can be long, notably with Selenium). + +`pytest-dependency`_ + Even if good testing practice normally means that tests can be run independently, in the + case of e2e tests we are using a real environment, and some tests do create files, + PubSub nodes, accounts, etc. It would be resource consuming to delete then recreate them + only to have standalone tests, thus to keep tests short and simple, some of them must be + run in order. The *dependecy* plugin is used to manage that, and will skip tests if one + of their dependencies is failing. The markup help also to document the tests order. + +.. _TigerVNC: https://tigervnc.org +.. _pytest-timeout: https://github.com/pytest-dev/pytest-timeout +.. _pytest-dependency: https://github.com/RKrahl/pytest-dependency + +common fixtures +--------------- + +Here are the fixture common to all e2e tests which are good to know: + +``test_profiles`` + Creates a bunch of test accounts which are available during the whole test session. + Those account are destroyed once all the tests are finished (successful or not), except + if you set the ``LIBERVIA_TEST_E2E_KEEP_PROFILES`` environment variable (or use the + ``--keep-profiles`` flag in ``run_e2e.py``. + + The profiles created are in the form ``accountX`` for account on the ``server1.test``, + or ``accountX_sY`` for account on other servers (see the docstring for details). + + This fixture should be used on top of each e2e test module. + +``pubsub_nodes`` + Create 2 pubsub nodes with ``open`` access model and named ``test`` (one on ``account1`` + PEP service, and the other one on ``pubsub.server1.test``, created with the same + account). + + Those node are created for the scope of the class. + +``fake_file`` + Create files filled with random bytes, and check them. + + A file is created by calling ``fake_file.size(size)``, and by default files of the same + size are re-used (set ``use_cache=False`` to create new files). This method returns a + ``pathlib.Path``. SHA-256 hash of the created file can be retrieved using + ``fake_file.get_source_hash(source_file_path)`` with the file path as argument. + + ``fake_file.new_dest_file()`` will return a Path to a randomly named destination file, + and ``fake_file.get_dest_hash(dest_file_path)`` will generate its hash once written. + +``sent_emails`` + When used, a fake SMTP server (already configured in container's ``libervia.conf``) will be + launched if it's not already, and all messages sent to it since the beginning of the test + will be available in the given list. Message are subclasses of + ``email.message.EmailMessage`` with the extra properties ``from_``, ``to``, ``subject`` + and ``body`` available for easy access to their content. + + The SMTP server is terminated at the end of the test session. + +libervia-cli e2e tests +---------------------- + +End-to-end tests for ``libervia-cli`` are a good way to tests backend features without having to +deal with frontends UI. Those tests use extensively the ``sh`` module, which helps +writing ``libervia-cli`` commands like if they where methods. + +Among the helping fixture (check the various ``conftest.py`` files for details), the +following are specially good to know: + +``li_json`` + Set the ``json_raw`` output are parse it. When you use this instead of the normal ``libervia-cli``, + you'll get a Python object that you can manipulate easily. + +``li_elt`` + Set the ``xml_raw`` output and parse it as a Twisted ``domish.Element``. When you use a + command which can return XML, it is useful to get this object which is easy to + manipulate in Python. + +``editor`` + Create a fake editor (replacing the one normally set in ``EDITOR`` environment + variable), to automatically modify and/or check the text sent by a command. You can + specify Python code to execute to modify the received text with the ``set_filter`` + method (this code is in a string which will be executed by Python interpreter, where the + ``content`` variable is the received text). By default, the text is kept unmodified. + + After ``editor`` has been used by the ``libervia-cli`` command, you can check its + ``original_content`` property to see the text that it received, and ``new_content`` + property to see the text that has been written after updating the original content with + the code set in ``set_filter``. + +Libervia e2e tests +------------------ + +E2e tests for Libervia are executed, as it is common in web world, with `Selenium`_: user +actions are simulated in automated browser, and results are checked. + +To make the tests as easy to write as possible, and as natural to read as possible, the +higher level `Helium`_ Python module is used. Thanks to it, the tests can read pretty much +like instructions we would give to a human user. Helium makes also easy to do some tasks +more complicated with Selenium alone, like dropping a file to an element. + +If a test is failing, a screenshot of the browser is taken. If you run the tests though +the ``run_e2e.py`` command (which you should), you'll find the screenshots in the +``report_*`` directory which is created in working dir in case of failure. + +Here are the helping fixtures which are notably good to know, you should always use either +``log_in_account1`` or ``nobody_logged_in``: + +``log_in_account1`` + Start the test with the main test account logged. + +``nobody_logged_in`` + Start the test without anybody logged (this is done by clearing all cookies). + +.. _Selenium: https://www.selenium.dev +.. _Helium: https://github.com/mherrmann/selenium-python-helium + +examples +-------- + +Following examples have to be run from ``tests/e2e`` directory. + +Run all tests for ``Libervia CLI``:: + + $ ./run_e2e.py -k libervia-cli + +Run all tests for ``Libervia Web`` with real-time visual feedback (note that you need to have +``vncviewer`` installed and available in path, see above):: + + $ ./run_e2e.py -k libervia-web --visual + + +Run all tests with verbose mode (useful to know which test is currently running):: + + $ ./run_e2e.py -v + +Run pubsub tests in verbose mode:: + + $ ./run_e2e.py -k pubsub -v + +Run in dev mode, to work on new tests, note that we run the ``user_can_create_account`` +test to be sure to have test profiles created and fake SMTP server run…:: + + $ ./run_e2e.py -k user_can_create_account --dev-mode + +…then to go into the ``backend`` container and work with the browser (to be run in ``docker`` +directory)…:: + + $ docker-compose -f docker-compose-e2e.yml exec backend /bin/bash + +…and, inside the container, you can now run ``python3`` and enter instruction prints at +the end of the test session.
--- a/doc/index.rst Wed Sep 01 15:25:02 2021 +0200 +++ b/doc/index.rst Wed Sep 08 11:20:37 2021 +0200 @@ -29,7 +29,7 @@ developer.rst /libervia-cli/index.rst /libervia-tui/index.rst - /contribuing/index.rst + /contributing/index.rst Indices and tables
--- a/docker/backend-dev-e2e/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/backend-dev-e2e/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,4 +1,5 @@ -FROM libervia/backend:dev +ARG REVISION=dev +FROM libervia/backend:${REVISION} LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>" @@ -46,7 +47,7 @@ RUN \ # pytest and its plugins pip install pytest pytest-timeout pytest-dependency \ - # needed to test jp + # needed to test libervia-cli sh \ # needed to test libervia helium \ @@ -57,7 +58,7 @@ RUN ./entrypoint.sh \ # we create the file sharing component which will autoconnect when backend is started - jp profile create file_sharing -j files.server1.test -p "" --xmpp-password test_e2e -C file_sharing -A && \ + li profile create file_sharing -j files.server1.test -p "" --xmpp-password test_e2e -C file_sharing -A && \ libervia-backend stop ENV LIBERVIA_TEST_REPORT_DIR=/reports
--- a/docker/backend-dev/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/backend-dev/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,13 +1,15 @@ -FROM debian:buster-slim +FROM debian:bullseye-slim LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>" +ARG REVISION ARG DEBIAN_FRONTEND=noninteractive -RUN apt-get update && apt-get upgrade -y && \ +RUN echo "Building image for Libervia ${REVISION:-dev}" && \ + apt-get update && apt-get upgrade -y && \ apt-get install -y --no-install-recommends locales python3-dev python3-venv python3-wheel mercurial libxml2-dev libxslt-dev libcairo2-dev libjpeg-dev libgirepository1.0-dev libdbus-1-dev libdbus-glib-1-dev dbus-x11 cmake && \ \ - # will be used to put many SàT specific data + # will be used to put many Libervia specific data mkdir -p /usr/share/libervia/certificates && \ addgroup tls-cert --gid 9999 && \ chown :tls-cert /usr/share/libervia/certificates && \ @@ -18,12 +20,12 @@ mkdir /src && chown libervia:libervia /src && \ \ # we need UTF-8 locale - sed -i "s/# en_US.UTF-8/en_US.UTF-8/" /etc/locale.gen && locale-gen && \ - \ - # we need a TCP socket for D-Bus - sed -i "s&<listen>unix:tmpdir=/tmp</listen>&\0\n <listen>tcp:host=localhost,bind=*,port=55555,family=ipv4</listen>\n <auth>ANONYMOUS</auth>\n <allow_anonymous/>&" /usr/share/dbus-1/session.conf + sed -i "s/# en_US.UTF-8/en_US.UTF-8/" /etc/locale.gen && locale-gen ENV LC_ALL en_US.UTF-8 +ENV DOCKER_LIBERVIA_REV=${REVISION:-dev} + +COPY --chown=root:root session.conf /usr/share/dbus-1/session.conf WORKDIR /home/libervia USER libervia @@ -34,15 +36,16 @@ # local repos without it them cause troubles hg clone https://repos.goffi.org/urwid-satext && ~/libervia_env/bin/pip install -e urwid-satext && \ mv urwid-satext/urwid_satext.egg-info ~/libervia_env/lib/python3.*/site-packages && \ - hg clone https://repos.goffi.org/sat_tmp && ~/libervia_env/bin/pip install -e sat_tmp && \ + hg clone https://repos.goffi.org/sat_tmp -u "${REVISION:-tip}" && ~/libervia_env/bin/pip install -e sat_tmp && \ mv sat_tmp/sat_tmp.egg-info ~/libervia_env/lib/python3.*/site-packages && \ - hg clone https://repos.goffi.org/sat_templates && ~/libervia_env/bin/pip install -e sat_templates && \ - mv sat_templates/libervia_templates.egg-info ~/libervia_env/lib/python3.*/site-packages && \ - hg clone https://repos.goffi.org/sat_media && \ - hg clone https://repos.goffi.org/sat && ~/libervia_env/bin/pip install -e 'sat[SVG]' && \ - mv sat/libervia_backend.egg-info ~/libervia_env/lib/python3.*/site-packages + hg clone https://repos.goffi.org/libervia-templates && ~/libervia_env/bin/pip install -e libervia-templates && \ + mv libervia-templates/libervia_templates.egg-info ~/libervia_env/lib/python3.*/site-packages && \ + hg clone https://repos.goffi.org/libervia-media && \ + hg clone https://repos.goffi.org/libervia-backend -u "${REVISION:-tip}" && ~/libervia_env/bin/pip install -e 'libervia-backend[SVG]' && \ + mv libervia-backend/libervia_backend.egg-info ~/libervia_env/lib/python3.*/site-packages && \ + mkdir -p /home/libervia/.local/share/libervia -ENV DBUS_SESSION_BUS_ADDRESS=tcp:host=backend,port=55555,family=ipv4 +ENV DBUS_SESSION_BUS_ADDRESS="tcp:host=backend,port=55555,family=ipv4;unix:path=/home/libervia/.local/share/libervia/dbus_socket" ENV PATH=/home/libervia/libervia_env/bin:$PATH COPY --chown=root:root libervia.conf /etc/_libervia.conf
--- a/docker/backend-dev/libervia.conf Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/backend-dev/libervia.conf Wed Sep 08 11:20:37 2021 +0200 @@ -1,5 +1,5 @@ [DEFAULT] -media_dir = /src/sat_media +media_dir = /src/libervia-media [cli] # FIXME: default "auto" value cause trouble
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docker/backend-dev/session.conf Wed Sep 08 11:20:37 2021 +0200 @@ -0,0 +1,32 @@ +<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN" +"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"> +<busconfig> + <type>session</type> + <keep_umask/> + <listen>unix:path=/home/libervia/.local/share/libervia/dbus_socket</listen> + <listen>tcp:host=localhost,bind=*,port=55555,family=ipv4</listen> + <auth>ANONYMOUS</auth> + <allow_anonymous/> + <standard_session_servicedirs /> + <policy context="default"> + <allow send_destination="*" eavesdrop="true"/> + <allow eavesdrop="true"/> + <allow own="*"/> + </policy> + <include if_selinux_enabled="yes" selinux_root_relative="yes">contexts/dbus_contexts</include> + <limit name="max_incoming_bytes">1000000000</limit> + <limit name="max_incoming_unix_fds">250000000</limit> + <limit name="max_outgoing_bytes">1000000000</limit> + <limit name="max_outgoing_unix_fds">250000000</limit> + <limit name="max_message_size">1000000000</limit> + <limit name="service_start_timeout">120000</limit> + <limit name="auth_timeout">240000</limit> + <limit name="pending_fd_timeout">150000</limit> + <limit name="max_completed_connections">100000</limit> + <limit name="max_incomplete_connections">10000</limit> + <limit name="max_connections_per_user">100000</limit> + <limit name="max_pending_service_starts">10000</limit> + <limit name="max_names_per_connection">50000</limit> + <limit name="max_match_rules_per_connection">50000</limit> + <limit name="max_replies_per_connection">50000</limit> +</busconfig>
--- a/docker/docker-compose-e2e.yml Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/docker-compose-e2e.yml Wed Sep 08 11:20:37 2021 +0200 @@ -38,8 +38,11 @@ LIBERVIA_PUBSUB_XMPP_PWD: test_e2e backend: - image: libervia/backend:dev-e2e - build: backend-dev-e2e + image: libervia/backend:${DOCKER_LIBERVIA_REV:-dev}-e2e + build: + context: backend-dev-e2e + args: + REVISION: ${DOCKER_LIBERVIA_REV:-} environment: LIBERVIA_TEST_ENV_E2E: "1" LIBERVIA_TEST_ENV_E2E_WEB: "1" @@ -54,8 +57,11 @@ - libervia-backend.test web: - image: libervia/web:dev-e2e - build: libervia-web-dev-e2e + image: libervia/web:${DOCKER_LIBERVIA_REV:-dev}-e2e + build: + context: libervia-web-dev-e2e + args: + REVISION: ${DOCKER_LIBERVIA_REV:-} depends_on: - backend environment:
--- a/docker/docker-compose.yml Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/docker-compose.yml Wed Sep 08 11:20:37 2021 +0200 @@ -2,11 +2,17 @@ services: backend: - image: libervia/backend:dev - build: backend-dev + image: "libervia/backend:${DOCKER_LIBERVIA_REV:-dev}" + build: + context: backend-dev + args: + REVISION: "${DOCKER_LIBERVIA_REV:-}" web: - image: libervia/web:dev - build: libervia-web-dev + image: "libervia/web:${DOCKER_LIBERVIA_REV:-dev}" + build: + context: libervia-web-dev + args: + REVISION: "${DOCKER_LIBERVIA_REV:-}" depends_on: - backend
--- a/docker/libervia-web-dev-demo/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/libervia-web-dev-demo/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,4 +1,5 @@ -FROM libervia/web:dev +ARG REVISION=dev +FROM libervia/web:${REVISION} LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>"
--- a/docker/libervia-web-dev-e2e/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/libervia-web-dev-e2e/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,4 +1,5 @@ -FROM libervia/web:dev +ARG REVISION=dev +FROM libervia/web:${REVISION} LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>"
--- a/docker/libervia-web-dev/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/libervia-web-dev/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,7 +1,9 @@ -FROM libervia/backend:dev +ARG REVISION +FROM libervia/backend:${REVISION:-dev} LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>" +ARG REVISION ARG DEBIAN_FRONTEND=noninteractive USER root @@ -9,9 +11,9 @@ RUN apt-get install -y --no-install-recommends yarnpkg WORKDIR /home/libervia USER libervia -RUN cd /src && hg clone https://repos.goffi.org/libervia && \ - ~/libervia_env/bin/pip install -e libervia && \ - mv libervia/libervia_web.egg-info ~/libervia_env/lib/python3.*/site-packages +RUN cd /src && hg clone https://repos.goffi.org/libervia-web -u "${REVISION:-tip}" && \ + ~/libervia_env/bin/pip install -e libervia-web && \ + mv libervia-web/libervia_web.egg-info ~/libervia_env/lib/python3.*/site-packages RUN ./entrypoint.sh \ # we build here to avoid re-downloading node modules or other browser
--- a/docker/pubsub/Dockerfile Wed Sep 01 15:25:02 2021 +0200 +++ b/docker/pubsub/Dockerfile Wed Sep 08 11:20:37 2021 +0200 @@ -1,4 +1,4 @@ -FROM debian:buster-slim +FROM debian:bullseye-slim LABEL maintainer="Goffi <tmp_dockerfiles@goffi.org>" @@ -29,6 +29,6 @@ hg clone https://repos.goffi.org/sat_tmp && ~/libervia_env/bin/pip install -e sat_tmp && \ mv sat_tmp/sat_tmp.egg-info ~/libervia_env/lib/python3.*/site-packages && \ hg clone https://repos.goffi.org/sat_pubsub && ~/libervia_env/bin/pip install -e sat_pubsub && \ - mv sat_pubsub/libervia_pubsub.egg-info ~/libervia_env/lib/python3.*/site-packages + mv sat_pubsub/sat_pubsub.egg-info ~/libervia_env/lib/python3.*/site-packages ENTRYPOINT ["/home/libervia/entrypoint.sh"]
--- a/sat/bridge/bridge_constructor/constructors/dbus/dbus_frontend_template.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/bridge/bridge_constructor/constructors/dbus/dbus_frontend_template.py Wed Sep 08 11:20:37 2021 +0200 @@ -140,7 +140,21 @@ kwargs['reply_handler'] = _callback kwargs['error_handler'] = lambda err: _errback(dbus_to_bridge_exception(err)) - return method(*args, **kwargs) + try: + return method(*args, **kwargs) + except ValueError as e: + if e.args[0].startswith("Unable to guess signature"): + # XXX: if frontend is started too soon after backend, the + # inspection misses methods (notably plugin dynamically added + # methods). The following hack works around that by redoing the + # cache of introspected methods signatures. + log.debug("using hack to work around inspection issue") + proxy = self.db_plugin_iface.proxy_object + IN_PROGRESS = proxy.INTROSPECT_STATE_INTROSPECT_IN_PROGRESS + proxy._introspect_state = IN_PROGRESS + proxy._Introspect() + return self.db_plugin_iface.get_dbus_method(name)(*args, **kwargs) + raise e return getPluginMethod @@ -167,13 +181,32 @@ fut.set_result, ret) error_handler = lambda err: loop.call_soon_threadsafe( fut.set_exception, dbus_to_bridge_exception(err)) - method( - *args, - **kwargs, - timeout=const_TIMEOUT, - reply_handler=reply_handler, - error_handler=error_handler - ) + try: + method( + *args, + **kwargs, + timeout=const_TIMEOUT, + reply_handler=reply_handler, + error_handler=error_handler + ) + except ValueError as e: + if e.args[0].startswith("Unable to guess signature"): + # same hack as for Bridge.__getattribute__ + log.warning("using hack to work around inspection issue") + proxy = self.db_plugin_iface.proxy_object + IN_PROGRESS = proxy.INTROSPECT_STATE_INTROSPECT_IN_PROGRESS + proxy._introspect_state = IN_PROGRESS + proxy._Introspect() + self.db_plugin_iface.get_dbus_method(name)( + *args, + **kwargs, + timeout=const_TIMEOUT, + reply_handler=reply_handler, + error_handler=error_handler + ) + + else: + raise e return fut return getPluginMethod
--- a/sat/core/sat_main.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/core/sat_main.py Wed Sep 08 11:20:37 2021 +0200 @@ -228,11 +228,7 @@ ui_contact_list.ContactList(self) ui_profile_manager.ProfileManager(self) except Exception as e: - log.error( - _("Could not initialize backend: {reason}").format( - reason=str(e).decode("utf-8", "ignore") - ) - ) + log.error(f"Could not initialize backend: {e}") sys.exit(1) self._addBaseMenus() @@ -344,6 +340,12 @@ plugin_modes = plugin_info["modes"] = set( plugin_info.setdefault("modes", C.PLUG_MODE_DEFAULT) ) + if not plugin_modes.intersection(C.PLUG_MODE_BOTH): + log.error( + f"Can't import plugin at {plugin_path}, invalid {C.PI_MODES!r} " + f"value: {plugin_modes!r}" + ) + continue # if the plugin is an entry point, it must work in component mode if plugin_info["type"] == C.PLUG_TYPE_ENTRY_POINT:
--- a/sat/memory/persistent.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/memory/persistent.py Wed Sep 08 11:20:37 2021 +0200 @@ -204,7 +204,7 @@ )) def __repr__(self): - raise NotImplementedError + return self.__str__() def __str__(self): return "LazyPersistentBinaryDict (namespace: {})".format(self.namespace)
--- a/sat/plugins/plugin_misc_android.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/plugins/plugin_misc_android.py Wed Sep 08 11:20:37 2021 +0200 @@ -309,9 +309,11 @@ # "XXX has no attribute 'invoke'" error (looks like the same issue as # https://github.com/kivy/pyjnius/issues/59) self.br = BroadcastReceiver( - callback=lambda *args, **kwargs: reactor.callLater(0, - self.onConnectivityChange), - actions=["android.net.conn.CONNECTIVITY_CHANGE"]) + callback=lambda *args, **kwargs: reactor.callFromThread( + self.onConnectivityChange + ), + actions=["android.net.conn.CONNECTIVITY_CHANGE"] + ) self.br.start() @property
--- a/sat/plugins/plugin_misc_file.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/plugins/plugin_misc_file.py Wed Sep 08 11:20:37 2021 +0200 @@ -237,8 +237,8 @@ data_cb=file_data.get("data_cb"), ) - def _gotConfirmation( - self, data, client, peer_jid, transfer_data, file_data, stream_object + async def _gotConfirmation( + self, client, data, peer_jid, transfer_data, file_data, stream_object ): """Called when the permission and dest path have been received @@ -258,19 +258,7 @@ # we manage case where file already exists if os.path.exists(file_path): - - def check_overwrite(overwrite): - if overwrite: - self.openFileWrite( - client, file_path, transfer_data, file_data, stream_object - ) - return True - else: - return defer.ensureDeferred( - self.getDestDir(client, peer_jid, transfer_data, file_data) - ) - - exists_d = xml_tools.deferConfirm( + overwrite = await xml_tools.deferConfirm( self.host, _(CONFIRM_OVERWRITE).format(file_path), _(CONFIRM_OVERWRITE_TITLE), @@ -282,8 +270,9 @@ security_limit=SECURITY_LIMIT, profile=client.profile, ) - exists_d.addCallback(check_overwrite) - return exists_d + + if not overwrite: + return await self.getDestDir(client, peer_jid, transfer_data, file_data) self.openFileWrite(client, file_path, transfer_data, file_data, stream_object) return True @@ -315,7 +304,7 @@ "size_human" will also be added with human readable file size @param stream_object(bool): if True, a stream_object will be used instead of file_obj a stream.FileStreamObject will be used - return (defer.Deferred): True if transfer is accepted + return: True if transfer is accepted """ cont, ret_value = await self.host.trigger.asyncReturnPoint( "FILE_getDestDir", client, peer_jid, transfer_data, file_data, stream_object @@ -327,7 +316,7 @@ assert PROGRESS_ID_KEY in file_data # human readable size file_data["size_human"] = common_utils.getHumanSize(file_data["size"]) - d = xml_tools.deferDialog( + resp_data = await xml_tools.deferDialog( self.host, _(CONFIRM).format(peer=peer_jid.full(), **file_data), _(CONFIRM_TITLE), @@ -341,12 +330,13 @@ security_limit=SECURITY_LIMIT, profile=client.profile, ) - d.addCallback( - self._gotConfirmation, + + accepted = await self._gotConfirmation( client, + resp_data, peer_jid, transfer_data, file_data, stream_object, ) - return d + return accepted
--- a/sat/plugins/plugin_xep_0277.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/plugins/plugin_xep_0277.py Wed Sep 08 11:20:37 2021 +0200 @@ -19,7 +19,6 @@ import time import dateutil import calendar -import urllib.parse from secrets import token_urlsafe from typing import Optional from functools import partial @@ -1032,13 +1031,12 @@ will return(JID(u'sat-pubsub.example.net'), 'urn:xmpp:comments:_af43b363-3259-4b2a-ba4c-1bc33aa87634__urn:xmpp:groupblog:somebody@example.net') @return (tuple[jid.JID, unicode]): service and node """ - parsed_url = urllib.parse.urlparse(node_url, "xmpp") - service = jid.JID(parsed_url.path) - parsed_queries = urllib.parse.parse_qs(parsed_url.query) - node = parsed_queries.get("node", [""])[0] - - if not node: - raise exceptions.DataError("Invalid comments link") + try: + parsed_url = xmpp_uri.parseXMPPUri(node_url) + service = jid.JID(parsed_url["path"]) + node = parsed_url["node"] + except Exception as e: + raise exceptions.DataError(f"Invalid comments link: {e}") return (service, node)
--- a/sat/plugins/plugin_xep_0353.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/plugins/plugin_xep_0353.py Wed Sep 08 11:20:37 2021 +0200 @@ -19,7 +19,7 @@ from zope.interface import implementer from twisted.internet import defer from twisted.internet import reactor -from twisted.words.protocols.jabber import xmlstream, jid +from twisted.words.protocols.jabber import xmlstream, jid, error from twisted.words.xish import domish from wokkel import disco, iwokkel from sat.core.i18n import _, D_ @@ -38,7 +38,7 @@ C.PI_NAME: "Jingle Message Initiation", C.PI_IMPORT_NAME: "XEP-0353", C.PI_TYPE: "XEP", - C.PI_MODES: C.PLUG_MODE_CLIENT, + C.PI_MODES: [C.PLUG_MODE_CLIENT], C.PI_PROTOCOLS: ["XEP-0353"], C.PI_DEPENDENCIES: ["XEP-0166"], C.PI_MAIN: "XEP_0353", @@ -85,8 +85,15 @@ if peer_jid.resource: return True - infos = await self.host.memory.disco.getInfos(client, peer_jid) - categories = {c for c, __ in infos.identities} + try: + infos = await self.host.memory.disco.getInfos(client, peer_jid) + except error.StanzaError as e: + if e.condition == "service-unavailable": + categories = {} + else: + raise e + else: + categories = {c for c, __ in infos.identities} if "component" in categories: # we don't use message initiation with components return True
--- a/sat/tools/common/uri.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat/tools/common/uri.py Wed Sep 08 11:20:37 2021 +0200 @@ -19,6 +19,7 @@ """ XMPP uri parsing tools """ +import sys import urllib.parse import urllib.request, urllib.parse, urllib.error @@ -53,7 +54,13 @@ if query_end == -1 or "=" in query_type: raise ValueError("no query type, invalid XMPP URI") - pairs = urllib.parse.parse_qs(uri_split.geturl()) + if sys.version_info >= (3, 9): + # parse_qs behaviour has been modified in Python 3.9, ";" is not understood as a + # parameter separator anymore but the "separator" argument has been added to + # change it. + pairs = urllib.parse.parse_qs(uri_split.geturl(), separator=";") + else: + pairs = urllib.parse.parse_qs(uri_split.geturl()) for k, v in list(pairs.items()): if len(v) != 1: raise NotImplementedError("multiple values not managed")
--- a/sat_frontends/bridge/dbus_bridge.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat_frontends/bridge/dbus_bridge.py Wed Sep 08 11:20:37 2021 +0200 @@ -140,7 +140,21 @@ kwargs['reply_handler'] = _callback kwargs['error_handler'] = lambda err: _errback(dbus_to_bridge_exception(err)) - return method(*args, **kwargs) + try: + return method(*args, **kwargs) + except ValueError as e: + if e.args[0].startswith("Unable to guess signature"): + # XXX: if frontend is started too soon after backend, the + # inspection misses methods (notably plugin dynamically added + # methods). The following hack works around that by redoing the + # cache of introspected methods signatures. + log.debug("using hack to work around inspection issue") + proxy = self.db_plugin_iface.proxy_object + IN_PROGRESS = proxy.INTROSPECT_STATE_INTROSPECT_IN_PROGRESS + proxy._introspect_state = IN_PROGRESS + proxy._Introspect() + return self.db_plugin_iface.get_dbus_method(name)(*args, **kwargs) + raise e return getPluginMethod @@ -930,13 +944,32 @@ fut.set_result, ret) error_handler = lambda err: loop.call_soon_threadsafe( fut.set_exception, dbus_to_bridge_exception(err)) - method( - *args, - **kwargs, - timeout=const_TIMEOUT, - reply_handler=reply_handler, - error_handler=error_handler - ) + try: + method( + *args, + **kwargs, + timeout=const_TIMEOUT, + reply_handler=reply_handler, + error_handler=error_handler + ) + except ValueError as e: + if e.args[0].startswith("Unable to guess signature"): + # same hack as for Bridge.__getattribute__ + log.warning("using hack to work around inspection issue") + proxy = self.db_plugin_iface.proxy_object + IN_PROGRESS = proxy.INTROSPECT_STATE_INTROSPECT_IN_PROGRESS + proxy._introspect_state = IN_PROGRESS + proxy._Introspect() + self.db_plugin_iface.get_dbus_method(name)( + *args, + **kwargs, + timeout=const_TIMEOUT, + reply_handler=reply_handler, + error_handler=error_handler + ) + + else: + raise e return fut return getPluginMethod
--- a/sat_frontends/jp/constants.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat_frontends/jp/constants.py Wed Sep 08 11:20:37 2021 +0200 @@ -54,6 +54,11 @@ OUTPUT_XMLUI, OUTPUT_LIST_XMLUI, ) + OUTPUT_NAME_SIMPLE = "simple" + OUTPUT_NAME_XML = "xml" + OUTPUT_NAME_XML_RAW = "xml-raw" + OUTPUT_NAME_JSON = "json" + OUTPUT_NAME_JSON_RAW = "json-raw" # Pubsub options flags SERVICE = "service" # service required
--- a/sat_frontends/jp/output_std.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat_frontends/jp/output_std.py Wed Sep 08 11:20:37 2021 +0200 @@ -26,9 +26,6 @@ import json __outputs__ = ["Simple", "Json"] -SIMPLE = "simple" -JSON = "json" -JSON_RAW = "json_raw" class Simple(object): @@ -36,13 +33,13 @@ def __init__(self, host): self.host = host - host.register_output(C.OUTPUT_TEXT, SIMPLE, self.simple_print) - host.register_output(C.OUTPUT_LIST, SIMPLE, self.list) - host.register_output(C.OUTPUT_DICT, SIMPLE, self.dict) - host.register_output(C.OUTPUT_LIST_DICT, SIMPLE, self.list_dict) - host.register_output(C.OUTPUT_DICT_DICT, SIMPLE, self.dict_dict) - host.register_output(C.OUTPUT_MESS, SIMPLE, self.messages) - host.register_output(C.OUTPUT_COMPLEX, SIMPLE, self.simple_print) + host.register_output(C.OUTPUT_TEXT, C.OUTPUT_NAME_SIMPLE, self.simple_print) + host.register_output(C.OUTPUT_LIST, C.OUTPUT_NAME_SIMPLE, self.list) + host.register_output(C.OUTPUT_DICT, C.OUTPUT_NAME_SIMPLE, self.dict) + host.register_output(C.OUTPUT_LIST_DICT, C.OUTPUT_NAME_SIMPLE, self.list_dict) + host.register_output(C.OUTPUT_DICT_DICT, C.OUTPUT_NAME_SIMPLE, self.dict_dict) + host.register_output(C.OUTPUT_MESS, C.OUTPUT_NAME_SIMPLE, self.messages) + host.register_output(C.OUTPUT_COMPLEX, C.OUTPUT_NAME_SIMPLE, self.simple_print) def simple_print(self, data): self.host.disp(str(data)) @@ -109,19 +106,19 @@ def __init__(self, host): self.host = host - host.register_output(C.OUTPUT_TEXT, JSON, self.dump) - host.register_output(C.OUTPUT_LIST, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_LIST, JSON_RAW, self.dump) - host.register_output(C.OUTPUT_DICT, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_DICT, JSON_RAW, self.dump) - host.register_output(C.OUTPUT_LIST_DICT, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_LIST_DICT, JSON_RAW, self.dump) - host.register_output(C.OUTPUT_DICT_DICT, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_DICT_DICT, JSON_RAW, self.dump) - host.register_output(C.OUTPUT_MESS, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_MESS, JSON_RAW, self.dump) - host.register_output(C.OUTPUT_COMPLEX, JSON, self.dump_pretty) - host.register_output(C.OUTPUT_COMPLEX, JSON_RAW, self.dump) + host.register_output(C.OUTPUT_TEXT, C.OUTPUT_NAME_JSON, self.dump) + host.register_output(C.OUTPUT_LIST, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_LIST, C.OUTPUT_NAME_JSON_RAW, self.dump) + host.register_output(C.OUTPUT_DICT, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_DICT, C.OUTPUT_NAME_JSON_RAW, self.dump) + host.register_output(C.OUTPUT_LIST_DICT, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_LIST_DICT, C.OUTPUT_NAME_JSON_RAW, self.dump) + host.register_output(C.OUTPUT_DICT_DICT, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_DICT_DICT, C.OUTPUT_NAME_JSON_RAW, self.dump) + host.register_output(C.OUTPUT_MESS, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_MESS, C.OUTPUT_NAME_JSON_RAW, self.dump) + host.register_output(C.OUTPUT_COMPLEX, C.OUTPUT_NAME_JSON, self.dump_pretty) + host.register_output(C.OUTPUT_COMPLEX, C.OUTPUT_NAME_JSON_RAW, self.dump) def dump(self, data): self.host.disp(json.dumps(data, default=str))
--- a/sat_frontends/jp/output_xml.py Wed Sep 01 15:25:02 2021 +0200 +++ b/sat_frontends/jp/output_xml.py Wed Sep 08 11:20:37 2021 +0200 @@ -1,7 +1,6 @@ #! /usr/bin/env python3 - -# jp: a SàT command line tool +# Libervia CLI frontend # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org) # This program is free software: you can redistribute it and/or modify @@ -36,8 +35,6 @@ __outputs__ = ["XML"] -RAW = "xml_raw" -PRETTY = "xml_pretty" class XML(object): @@ -45,16 +42,20 @@ def __init__(self, host): self.host = host - host.register_output(C.OUTPUT_XML, PRETTY, self.pretty, default=True) - host.register_output(C.OUTPUT_LIST_XML, PRETTY, self.pretty_list, default=True) - host.register_output(C.OUTPUT_XML, RAW, self.raw) - host.register_output(C.OUTPUT_LIST_XML, RAW, self.list_raw) + host.register_output(C.OUTPUT_XML, C.OUTPUT_NAME_XML, self.pretty, default=True) + host.register_output( + C.OUTPUT_LIST_XML, C.OUTPUT_NAME_XML, self.pretty_list, default=True + ) + host.register_output(C.OUTPUT_XML, C.OUTPUT_NAME_XML_RAW, self.raw) + host.register_output(C.OUTPUT_LIST_XML, C.OUTPUT_NAME_XML_RAW, self.list_raw) def colorize(self, xml): if pygments is None: self.host.disp( _( - "Pygments is not available, syntax highlighting is not possible. Please install if from http://pygments.org or with pip install pygments" + "Pygments is not available, syntax highlighting is not possible. " + "Please install if from http://pygments.org or with pip install " + "pygments" ), error=True, )
--- a/tests/e2e/libervia-cli/conftest.py Wed Sep 01 15:25:02 2021 +0200 +++ b/tests/e2e/libervia-cli/conftest.py Wed Sep 08 11:20:37 2021 +0200 @@ -35,7 +35,7 @@ def __call__(self, *args, **kwargs): args = self.subcommands + list(args) self.subcommands.clear() - kwargs['output'] = 'json_raw' + kwargs['output'] = 'json-raw' kwargs['_tty_out'] = False cmd = li(*args, **kwargs) return json.loads(cmd.stdout) @@ -60,7 +60,7 @@ def __call__(self, *args, **kwargs): args = self.subcommands + list(args) self.subcommands.clear() - kwargs['output'] = 'xml_raw' + kwargs['output'] = 'xml-raw' kwargs['_tty_out'] = False cmd = li(*args, **kwargs) return self.parser(cmd.stdout.decode().strip()) @@ -153,13 +153,13 @@ @pytest.fixture(scope="session") def li_json(): - """Run li with "json_raw" output, and returns the parsed value""" + """Run li with "json-raw" output, and returns the parsed value""" return LiberviaCliJson() @pytest.fixture(scope="session") def li_elt(): - """Run li with "xml_raw" output, and returns the parsed value""" + """Run li with "xml-raw" output, and returns the parsed value""" return LiberviaCliElt()
--- a/tests/e2e/libervia-cli/test_libervia-cli.py Wed Sep 01 15:25:02 2021 +0200 +++ b/tests/e2e/libervia-cli/test_libervia-cli.py Wed Sep 08 11:20:37 2021 +0200 @@ -176,7 +176,7 @@ li.blog.edit(_env=editor.env) assert len(editor.original_content) == 0 assert editor.new_content == payload_md - items_data = li_json.blog.get(max=1) + items_data = li_json.blog.get(max_items=1) last_item = items_data[0][0] last_item_id = last_item['id'] assert last_item['content'] == "content in markdown" @@ -187,7 +187,7 @@ li.blog.edit("--last-item", _env=editor.env) assert editor.original_content == payload_md assert editor.new_content == f"{payload_md} extended" - items_data = li_json.blog.get(max=1) + items_data = li_json.blog.get(max_items=1) last_item = items_data[0][0] # we check that the id hasn't been modified assert last_item['id'] == last_item_id
--- a/tests/e2e/libervia-web/test_libervia-web.py Wed Sep 01 15:25:02 2021 +0200 +++ b/tests/e2e/libervia-web/test_libervia-web.py Wed Sep 08 11:20:37 2021 +0200 @@ -94,9 +94,9 @@ click(album_link) wait_until(lambda: S("#file_drop").exists()) wait_until(lambda: not S("#loading_screen").exists()) - drag_file("/src/sat/tests/_files/test_1.jpg", "drop photos here") + drag_file("/src/libervia-backend/tests/_files/test_1.jpg", "drop photos here") wait_until(lambda: len(find_all(S("div.progress_finished")))==1) - drag_file("/src/sat/tests/_files/test_2.jpg", "drop photos here") + drag_file("/src/libervia-backend/tests/_files/test_2.jpg", "drop photos here") wait_until(lambda: len(find_all(S("div.progress_finished")))==2) assert S('img[alt="test_1.jpg"]').exists() assert S('img[alt="test_2.jpg"]').exists()
--- a/tests/e2e/run_e2e.py Wed Sep 01 15:25:02 2021 +0200 +++ b/tests/e2e/run_e2e.py Wed Sep 08 11:20:37 2021 +0200 @@ -158,19 +158,19 @@ libervia_web_root_path = libervia_web_path.parent if (libervia_web_root_path / ".hg").is_dir(): libervia_web_source = libervia_web_root_path - libervia_web_target = "/src/libervia" + libervia_web_target = "/src/libervia-web" else: libervia_web_source = libervia_web_path - libervia_web_target = "/src/libervia/libervia" + libervia_web_target = "/src/libervia-web/libervia" sat_templates_path = Path(sat_templates.__file__).parent.resolve() sat_templates_root_path = sat_templates_path.parent if (sat_templates_root_path / ".hg").is_dir(): sat_templates_source = sat_templates_root_path - sat_templates_target = "/src/sat_templates" + sat_templates_target = "/src/libervia-templates" else: sat_templates_source = sat_templates_path - sat_templates_target = "/src/sat_templates/sat_templates" + sat_templates_target = "/src/libervia-templates/sat_templates" compose_e2e_path = docker_path / "docker-compose-e2e.yml" if not compose_e2e_path.is_file(): @@ -186,13 +186,13 @@ volumes: - type: bind source: {backend_root_path} - target: /src/sat + target: /src/libervia-backend read_only: true web: volumes: - type: bind source: {backend_root_path} - target: /src/sat + target: /src/libervia-backend read_only: true - type: bind source: {libervia_web_source} @@ -224,7 +224,7 @@ docker_compose.up("-d") p = docker_compose.exec( - "-T", "--workdir", "/src/sat/tests", "backend", + "-T", "--workdir", "/src/libervia-backend/tests", "backend", "pytest", "-o", "cache_dir=/tmp", *sys.argv[1:], color="yes", _in=sys.stdin, _out=live_out, _out_bufsize=0, _err=live_err, _err_bufsize=0, _bg=True