libervia-backend: sat_frontends/jp/cmd

annotate sat_frontends/jp/cmd_input.py @ 2787:298408833ec2

memory (sqlite): optimizations indexes were missing on foreign keys for "message", "subject" and "thread" tables, seriously impacting performances. In addition to those indexes, two indexes have been added to speed ordering by timestamp on "history", and one on "files" table. history.rowid is not used anymore as there is an index on (profile_id, received_timestamp) which will speed up the query. Primary keys order has been changed to use automatic index in most common cases (filtering by profile_id then namespace).

author	Goffi <goffi@goffi.org>
date	Sat, 19 Jan 2019 22:49:32 +0100
parents	003b8b4b56a7
children	ab2696e34d29

rev	line source
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python2
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	2 # -- coding: utf-8 --
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	3
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # jp: a SàT command line tool
2771 003b8b4b56a7 date update Goffi <goffi@goffi.org> parents: 2624 diff changeset	5 # Copyright (C) 2009-2019 Jérôme Poisson (goffi@goffi.org)
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	6
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	10 # (at your option) any later version.
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	11
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	16
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	19
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	20
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	21 import base
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	22 from sat.core.i18n import _
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	23 from sat.core import exceptions
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	24 from sat_frontends.jp.constants import Const as C
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	25 from sat.tools.common.ansi import ANSI as A
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	26 import subprocess
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	27 import argparse
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	28 import sys
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	29
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	30 __commands__ = ["Input"]
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	31 OPT_STDIN = "stdin"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	32 OPT_SHORT = "short"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	33 OPT_LONG = "long"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	34 OPT_POS = "positional"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	35 OPT_IGNORE = "ignore"
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	36 OPT_TYPES = (OPT_STDIN, OPT_SHORT, OPT_LONG, OPT_POS, OPT_IGNORE)
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	37 OPT_EMPTY_SKIP = "skip"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	38 OPT_EMPTY_IGNORE = "ignore"
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	39 OPT_EMPTY_CHOICES = (OPT_EMPTY_SKIP, OPT_EMPTY_IGNORE)
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	40
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	41
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 class InputCommon(base.CommandBase):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	43 def __init__(self, host, name, help):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	44 base.CommandBase.__init__(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	45 self, host, name, use_verbose=True, use_profile=False, help=help
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	46 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	47 self.idx = 0
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 self.reset()
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	49
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	50 def reset(self):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 self.args_idx = 0
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	52 self._stdin = []
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	53 self._opts = []
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	54 self._pos = []
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	55 self._values_ori = []
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	56
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	57 def add_parser_options(self):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	58 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	59 "--encoding", default="utf-8", help=_(u"encoding of the input data")
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	60 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	61 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	62 "-i",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	63 "--stdin",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	64 action="append_const",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	65 const=(OPT_STDIN, None),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	66 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	67 help=_(u"standard input"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	68 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	69 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	70 "-s",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	71 "--short",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	72 type=self.opt(OPT_SHORT),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	73 action="append",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	74 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	75 help=_(u"short option"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	76 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	77 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	78 "-l",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	79 "--long",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	80 type=self.opt(OPT_LONG),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	81 action="append",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	82 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	83 help=_(u"long option"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	84 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	85 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	86 "-p",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	87 "--positional",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	88 type=self.opt(OPT_POS),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	89 action="append",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	90 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	91 help=_(u"positional argument"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	92 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	93 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	94 "-x",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	95 "--ignore",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	96 action="append_const",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	97 const=(OPT_IGNORE, None),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	98 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	99 help=_(u"ignore value"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	100 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	101 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	102 "-D",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	103 "--debug",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	104 action="store_true",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	105 help=_(u"don't actually run commands but echo what would be launched"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	106 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	107 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	108 "--log", type=argparse.FileType("wb"), help=_(u"log stdout to FILE")
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	109 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	110 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	111 "--log-err", type=argparse.FileType("wb"), help=_(u"log stderr to FILE")
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	112 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	113 self.parser.add_argument("command", nargs=argparse.REMAINDER)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	114
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	115 def opt(self, type_):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	116 return lambda s: (type_, s)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	117
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	118 def addValue(self, value):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	119 """add a parsed value according to arguments sequence"""
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	120 self._values_ori.append(value)
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	121 arguments = self.args.arguments
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	122 try:
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	123 arg_type, arg_name = arguments[self.args_idx]
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	124 except IndexError:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	125 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	126 _(u"arguments in input data and in arguments sequence don't match"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	127 error=True,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	128 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	129 self.host.quit(C.EXIT_DATA_ERROR)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 self.args_idx += 1
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	131 while self.args_idx < len(arguments):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	132 next_arg = arguments[self.args_idx]
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	133 if next_arg[0] not in OPT_TYPES:
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	134 # value will not be used if False or None, so we skip filter
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	135 if value not in (False, None):
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	136 # we have a filter
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	137 filter_type, filter_arg = arguments[self.args_idx]
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	138 value = self.filter(filter_type, filter_arg, value)
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	139 else:
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	140 break
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	141 self.args_idx += 1
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	142
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	143 if value is None:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	144 # we ignore this argument
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	145 return
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	146
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	147 if value is False:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	148 # we skip the whole row
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	149 if self.args.debug:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	150 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	151 A.color(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	152 C.A_SUBHEADER,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	153 _(u"values: "),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	154 A.RESET,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	155 u", ".join(self._values_ori),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	156 ),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	157 2,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	158 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	159 self.disp(A.color(A.BOLD, _(u"SKIPPING\n")))
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	160 self.reset()
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	161 self.idx += 1
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	162 raise exceptions.CancelError
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	163
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	164 if not isinstance(value, list):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	165 value = [value]
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	166
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	167 for v in value:
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	168 if arg_type == OPT_STDIN:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	169 self._stdin.append(v.encode("utf-8"))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	170 elif arg_type == OPT_SHORT:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	171 self._opts.append("-{}".format(arg_name))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	172 self._opts.append(v.encode("utf-8"))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	173 elif arg_type == OPT_LONG:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	174 self._opts.append("--{}".format(arg_name))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	175 self._opts.append(v.encode("utf-8"))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	176 elif arg_type == OPT_POS:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	177 self._pos.append(v.encode("utf-8"))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	178 elif arg_type == OPT_IGNORE:
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	179 pass
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	180 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	181 self.parser.error(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	182 _(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	183 u"Invalid argument, an option type is expected, got {type_}:{name}"
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	184 ).format(type_=arg_type, name=arg_name)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	185 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	186
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	187 def runCommand(self):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	188 """run requested command with parsed arguments"""
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	189 if self.args_idx != len(self.args.arguments):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	190 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	191 _(u"arguments in input data and in arguments sequence don't match"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	192 error=True,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	193 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	194 self.host.quit(C.EXIT_DATA_ERROR)
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	195 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	196 A.color(C.A_HEADER, _(u"command {idx}").format(idx=self.idx)),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	197 no_lf=not self.args.debug,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	198 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	199 stdin = "".join(self._stdin)
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	200 if self.args.debug:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	201 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	202 A.color(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	203 C.A_SUBHEADER, _(u"values: "), A.RESET, u", ".join(self._values_ori)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	204 ),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	205 2,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	206 )
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	207
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 if stdin:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	209 self.disp(A.color(C.A_SUBHEADER, u"--- STDIN ---"))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	210 self.disp(stdin.decode("utf-8"))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	211 self.disp(A.color(C.A_SUBHEADER, u"-------------"))
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	212 self.disp(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	213 u"{indent}{prog} {static} {options} {positionals}".format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	214 indent=4 * u" ",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	215 prog=sys.argv[0],
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	216 static=" ".join(self.args.command).decode("utf-8"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	217 options=u" ".join([o.decode("utf-8") for o in self._opts]),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	218 positionals=u" ".join([p.decode("utf-8") for p in self._pos]),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	219 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	220 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	221 self.disp(u"\n")
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	222 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	223 self.disp(u" (" + u", ".join(self._values_ori) + u")", 2, no_lf=True)
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	224 args = [sys.argv[0]] + self.args.command + self._opts + self._pos
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	225 p = subprocess.Popen(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	226 args,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	227 stdin=subprocess.PIPE,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	228 stdout=subprocess.PIPE,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	229 stderr=subprocess.PIPE,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	230 )
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	231 (stdout, stderr) = p.communicate(stdin)
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	232 log = self.args.log
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	233 log_err = self.args.log_err
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	234 log_tpl = "{command}\n{buff}\n\n"
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	235 if log:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	236 log.write(log_tpl.format(command=" ".join(args), buff=stdout))
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	237 if log_err:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	238 log_err.write(log_tpl.format(command=" ".join(args), buff=stderr))
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	239 ret = p.wait()
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	240 if ret == 0:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	241 self.disp(A.color(C.A_SUCCESS, _(u"OK")))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	242 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	243 self.disp(A.color(C.A_FAILURE, _(u"FAILED")))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	244
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	245 self.reset()
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 self.idx += 1
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	247
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	248 def filter(self, filter_type, filter_arg, value):
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	249 """change input value
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	250
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	251 @param filter_type(unicode): name of the filter
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	252 @param filter_arg(unicode, None): argument of the filter
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	253 @param value(unicode): value to filter
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	254 @return (unicode, False, None): modified value
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	255 False to skip the whole row
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	256 None to ignore this argument (but continue row with other ones)
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	257 """
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	258 raise NotImplementedError
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	259
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	260
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	261 class Csv(InputCommon):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	262 def __init__(self, host):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	263 super(Csv, self).__init__(host, "csv", _(u"comma-separated values"))
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	264
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	265 def add_parser_options(self):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	266 InputCommon.add_parser_options(self)
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	267 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	268 "-r",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	269 "--row",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	270 type=int,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	271 default=0,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	272 help=_(u"starting row (previous ones will be ignored)"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	273 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	274 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	275 "-S",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	276 "--split",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	277 action="append_const",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	278 const=("split", None),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	279 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	280 help=_(u"split value in several options"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	281 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	282 self.parser.add_argument(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	283 "-E",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	284 "--empty",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	285 action="append",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	286 type=self.opt("empty"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	287 dest="arguments",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	288 help=_(u"action to do on empty value ({choices})").format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	289 choices=u", ".join(OPT_EMPTY_CHOICES)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	290 ),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	291 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	292
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	293 def filter(self, filter_type, filter_arg, value):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	294 if filter_type == "split":
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	295 return value.split()
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	296 elif filter_type == "empty":
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	297 if filter_arg == OPT_EMPTY_IGNORE:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	298 return value if value else None
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	299 elif filter_arg == OPT_EMPTY_SKIP:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	300 return value if value else False
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	301 else:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	302 self.parser.error(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	303 _(u"--empty value must be one of {choices}").format(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	304 choices=u", ".join(OPT_EMPTY_CHOICES)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	305 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	306 )
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	307
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	308 super(Csv, self).filter(filter_type, filter_arg, value)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	309
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	310 def start(self):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	311 import csv
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	312
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	313 reader = csv.reader(sys.stdin)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	314 for idx, row in enumerate(reader):
2282 d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	315 try:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	316 if idx < self.args.row:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	317 continue
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	318 for value in row:
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	319 self.addValue(value.decode(self.args.encoding))
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	320 self.runCommand()
d8e48c850ad2 jp (input): log improvments + empty filter: Goffi <goffi@goffi.org> parents: 2278 diff changeset	321 except exceptions.CancelError:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	322 # this row has been cancelled, we skip it
2278 489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	323 continue
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	324
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	325
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	326 class Input(base.CommandBase):
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	327 subcommands = (Csv,)
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	328
489efbda377c jp (input): input command first draft: Goffi <goffi@goffi.org> parents: diff changeset	329 def __init__(self, host):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	330 super(Input, self).__init__(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	331 host,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	332 "input",
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	333 use_profile=False,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	334 help=_(u"launch command with external input"),
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	335 )

Mercurial > libervia-backend

annotate sat_frontends/jp/cmd_input.py @ 2787:298408833ec2