Mercurial > libervia-backend

comparison doc/jp/input.rst @ 3041:72583524cfd3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc (jp): jp commands are now fully documented: rel 232
author Goffi <goffi@goffi.org>
date Tue, 01 Oct 2019 22:49:06 +0200
parents
children
comparison
equal deleted inserted replaced
3040:fee60f17ebac 3041:72583524cfd3
1 ================================================
2 input: automatise commands using external inputs
3 ================================================
4
5 ``input`` is a way to use external data (like file in a specific format) as input
6 arguments. It may be seen as a something similar to ``--output`` but for inputs.
7
8
9 csv
10 ===
11
12 CSV (for Comma-Separated Values) is a common format for tabular data. It is widely used in
13 spreadsheet software (at least at en export format). With ``csv`` command, you can use
14 columns a CSV file as arguments to jp commands.
15
16 To set the command, you'll write in sequence what to do with each column of your data.
17 For each column you can:
18
19 - specify a short option name using ``-s ARGUMENTS, --short ARGUMENTS`` (short options are
20 the ones with a single ``-``)
21 - specify a long option name using ``-l ARGUMENTS, --long ARGUMENTS`` (long options are
22 the ones with two ``-``)
23 - specify a positional argument using ``-p ARGUMENTS, --positional ARGUMENTS``
24 - indicate to use the column data with ``stdin`` using ``-i, --stdin``
25 - ignore the column if it's not used in the jp command, using ``-x, --ignore``
26
27 After each column specification, you may use a filter to manage the value. So far the
28 following filters are available:
29
30 ``-S, --split``
31 This will split the value (on any whitespace character, discarding empty values) and
32 repeat the option which each item. This is useful when you are using an option which can
33 be repeated (like ``-t TAG, --tag TAG`` with ``jp blog set``).
34
35 ``-E ARGUMENTS, --empty ARGUMENTS``
36 Indicate what to do if the column value is empty (by default en empty string is used).
37 You can use either ``skip`` to skip the whole row, or ``ignore`` so the option will not
38 be set at all (which is different from the default which will still set the option but
39 with an empty string).
40
41 CSV file is read from stdin, and by default unicode is expected. You may force an encoding
42 by using ``--encoding ENCODING``.
43
44 By default all the rows are read, but you may want to ignore first rows (if they are used
45 for columns title, or if you have already handled part of the list). To do that, use the
46 ``-r ROW, --row ROW`` option.
47
48 When you test your command, it is better to do a dry run to see what will happen. The
49 ``-D, --debug`` option is here for that: if you set it, the commands won't be actually
50 run, but the exact command which would be executed will be printed on screen. You should
51 always use this option first until you're sure that what you want will be executed.
52
53 You may add verbosity level to help debugging. With a verbosity level of 2 (i.e. ``-vv``)
54 the value read from CSV will be printed.
55
56 By default stdout and stderr of each launched command is ignored, but you can log them to
57 files using respectively ``--log LOG`` and ``--log-err LOG_ERR`` where ``LOG`` and
58 ``LOG_ERR`` are paths to a log file to create.
59
60 Once all the sequence and options are set, you write the jp command that you want to use,
61 with all the needed static option (i.e. options which must be used each time).
62
63
64 example
65 -------
66
67 Louise as a spreadsheet with a table like this:
68
69 ============================ ============ ============= ===============
70 title body internal data tags
71 ============================ ============ ============= ===============
72 Some title a body ABC jp demo
73 Something else another body XYZ jp demo
74 Third one third body VWZ special_tag jp
75 This one doesn't have a body 123 jp demo numbers
76 last one last body 456 jp demo numbers
77 ============================ ============ ============= ===============
78
79 She wants to use it as input data to create blog posts.
80
81 She first saves the file using CSV format, let's say to ``~/blog_input.csv``.
82
83 Then she checks ``jp blog set --help`` to get name of options to use. She'll need to use
84 ``--title`` for title, ``stdin`` for the body and ``-t`` for tags. Louise wants to
85 activate comments, so she also wants to use ``-C`` for all posts, and a tag to says it's
86 coming from the spreadsheet (using ``-t spreadsheet``) .
87
88 The first row of the table is used for columns headers, so she'll start at row 1 with ``-r
89 1``.
90
91 There is one row without body, Louise want to skip any row without body so she'll use the
92 ``-E skip`` filter, just after specifying the body row.
93
94 Reading column by column, the sequence is like this:
95
96 ``-l title``
97 a title which goes to the ``--title`` long option of ``jp blog``
98 ``-i -E skip``
99 a body which goes to the stdin of ``jp blog``. If the body is empty, the ``-E skip``
100 filter tells to skip the whole row.
101 ``-x``
102 the ``internal data`` column is not used, so it is ignored
103 ``-s t -S``
104 the last column are the tags, so the ``-t`` short option is used. There are several of
105 them separated by spaces, so the ``-S`` filter is used to split the values.
106
107 First she'll use the ``-D, --debug`` to check that the commands which will be executed are
108 the expected one::
109
110 $ jp input csv -D -r 1 -l title -i -E skip -x -s t -S blog set -C -t spreadsheet < ~/blog_input.csv
111
112 Everything seems fine, so she'll actually launch the command by running the same command
113 line but without the ``-D`` option::
114
115 $ jp input csv -r 1 -l title -i -E skip -x -s t -S blog set -C -t spreadsheet < ~/blog_input.csv
116
117 She could also have used ``--log`` and ``--log-err`` to check the logs of each command::
118
119 $ jp input csv -r 1 -l title -i -E skip -x -s t -S --log /tmp/jp_blog_stdout.log --log-err /tmp/jp_blog_stderr.log blog set -C -t spreadsheet < ~/blog_input.csv