Mercurial > libervia-backend

annotate doc/jp/input.rst @ 3398:467d6c709f1d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
test: fixed use of `client.host` in legacy test: those tests are not maintained anymore, but as the use of `client.host` as been fixed elsewhere, it was the occasion to do it there too. Those legacy tests should be ported or to the new workflow, or deleted if they are not relevant anymore.
author Goffi <goffi@goffi.org>
date Thu, 12 Nov 2020 14:53:15 +0100
parents 72583524cfd3
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3041
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 ================================================
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
2 input: automatise commands using external inputs
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 ================================================
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
4
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
5 ``input`` is a way to use external data (like file in a specific format) as input
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 arguments. It may be seen as a something similar to ``--output`` but for inputs.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
7
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
8
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 csv
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
10 ===
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
11
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 CSV (for Comma-Separated Values) is a common format for tabular data. It is widely used in
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 spreadsheet software (at least at en export format). With ``csv`` command, you can use
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 columns a CSV file as arguments to jp commands.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 To set the command, you'll write in sequence what to do with each column of your data.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 For each column you can:
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 - specify a short option name using ``-s ARGUMENTS, --short ARGUMENTS`` (short options are
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
20 the ones with a single ``-``)
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 - specify a long option name using ``-l ARGUMENTS, --long ARGUMENTS`` (long options are
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 the ones with two ``-``)
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 - specify a positional argument using ``-p ARGUMENTS, --positional ARGUMENTS``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
24 - indicate to use the column data with ``stdin`` using ``-i, --stdin``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
25 - ignore the column if it's not used in the jp command, using ``-x, --ignore``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
26
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
27 After each column specification, you may use a filter to manage the value. So far the
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 following filters are available:
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
29
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 ``-S, --split``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 This will split the value (on any whitespace character, discarding empty values) and
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 repeat the option which each item. This is useful when you are using an option which can
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 be repeated (like ``-t TAG, --tag TAG`` with ``jp blog set``).
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
34
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 ``-E ARGUMENTS, --empty ARGUMENTS``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 Indicate what to do if the column value is empty (by default en empty string is used).
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 You can use either ``skip`` to skip the whole row, or ``ignore`` so the option will not
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 be set at all (which is different from the default which will still set the option but
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 with an empty string).
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
40
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 CSV file is read from stdin, and by default unicode is expected. You may force an encoding
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
42 by using ``--encoding ENCODING``.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
43
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 By default all the rows are read, but you may want to ignore first rows (if they are used
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
45 for columns title, or if you have already handled part of the list). To do that, use the
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
46 ``-r ROW, --row ROW`` option.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
47
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
48 When you test your command, it is better to do a dry run to see what will happen. The
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 ``-D, --debug`` option is here for that: if you set it, the commands won't be actually
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
50 run, but the exact command which would be executed will be printed on screen. You should
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 always use this option first until you're sure that what you want will be executed.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
53 You may add verbosity level to help debugging. With a verbosity level of 2 (i.e. ``-vv``)
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
54 the value read from CSV will be printed.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
55
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
56 By default stdout and stderr of each launched command is ignored, but you can log them to
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
57 files using respectively ``--log LOG`` and ``--log-err LOG_ERR`` where ``LOG`` and
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
58 ``LOG_ERR`` are paths to a log file to create.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
59
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 Once all the sequence and options are set, you write the jp command that you want to use,
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 with all the needed static option (i.e. options which must be used each time).
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
62
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
63
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
64 example
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
65 -------
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
66
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 Louise as a spreadsheet with a table like this:
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
68
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 ============================ ============ ============= ===============
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 title body internal data tags
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 ============================ ============ ============= ===============
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 Some title a body ABC jp demo
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 Something else another body XYZ jp demo
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 Third one third body VWZ special_tag jp
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
75 This one doesn't have a body 123 jp demo numbers
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
76 last one last body 456 jp demo numbers
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
77 ============================ ============ ============= ===============
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
78
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
79 She wants to use it as input data to create blog posts.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
80
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
81 She first saves the file using CSV format, let's say to ``~/blog_input.csv``.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
82
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
83 Then she checks ``jp blog set --help`` to get name of options to use. She'll need to use
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
84 ``--title`` for title, ``stdin`` for the body and ``-t`` for tags. Louise wants to
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
85 activate comments, so she also wants to use ``-C`` for all posts, and a tag to says it's
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
86 coming from the spreadsheet (using ``-t spreadsheet``) .
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
87
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
88 The first row of the table is used for columns headers, so she'll start at row 1 with ``-r
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
89 1``.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
90
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
91 There is one row without body, Louise want to skip any row without body so she'll use the
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 ``-E skip`` filter, just after specifying the body row.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
93
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
94 Reading column by column, the sequence is like this:
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
95
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
96 ``-l title``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
97 a title which goes to the ``--title`` long option of ``jp blog``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
98 ``-i -E skip``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
99 a body which goes to the stdin of ``jp blog``. If the body is empty, the ``-E skip``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
100 filter tells to skip the whole row.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
101 ``-x``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
102 the ``internal data`` column is not used, so it is ignored
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
103 ``-s t -S``
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
104 the last column are the tags, so the ``-t`` short option is used. There are several of
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
105 them separated by spaces, so the ``-S`` filter is used to split the values.
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
106
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
107 First she'll use the ``-D, --debug`` to check that the commands which will be executed are
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
108 the expected one::
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
109
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
110 $ jp input csv -D -r 1 -l title -i -E skip -x -s t -S blog set -C -t spreadsheet < ~/blog_input.csv
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
111
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
112 Everything seems fine, so she'll actually launch the command by running the same command
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
113 line but without the ``-D`` option::
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
114
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 $ jp input csv -r 1 -l title -i -E skip -x -s t -S blog set -C -t spreadsheet < ~/blog_input.csv
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
116
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 She could also have used ``--log`` and ``--log-err`` to check the logs of each command::
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
118
72583524cfd3 doc (jp): jp commands are now fully documented:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 $ jp input csv -r 1 -l title -i -E skip -x -s t -S --log /tmp/jp_blog_stdout.log --log-err /tmp/jp_blog_stderr.log blog set -C -t spreadsheet < ~/blog_input.csv