libervia-backend: libervia/backend/plugins/plugin_tickets

annotate libervia/backend/plugins/plugin_tickets_import.py @ 4212:5f2d496c633f

core: get rid of `pickle`: Use of `pickle` to serialise data was a technical legacy that was causing trouble to store in database, to update (if a class was serialised, a change could break update), and to security (pickle can lead to code execution). This patch remove all use of Pickle in favour in JSON, notably: - for caching data, a Pydantic model is now used instead - for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation - in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and `to_dict` method have been implemented to do serialisation. - new methods to (de)serialise data can now be specified with Identity data types. It is notably used to (de)serialise `path` of avatars. A migration script has been created to convert data (for upgrade or downgrade), with special care for XEP-0373 case. Depending of size of database, this migration script can be long to run. rel 443

author	Goffi <goffi@goffi.org>
date	Fri, 23 Feb 2024 13:31:04 +0100
parents	4b842c1fb686
children

rev	line source
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	1 #!/usr/bin/env python3
3137 559a625a236b fixed shebangs Goffi <goffi@goffi.org> parents: 3136 diff changeset	2
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	3
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 # SàT plugin for import external ticketss
3479 be6d91572633 date update Goffi <goffi@goffi.org> parents: 3452 diff changeset	5 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	6
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	10 # (at your option) any later version.
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	11
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	16
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	19
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	20 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	21 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	22 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	23 from libervia.backend.core.log import getLogger
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	24
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	25 log = getLogger(__name__)
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	26 from twisted.internet import defer
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	27 from libervia.backend.tools.common import uri
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	28 from libervia.backend.tools import utils
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	29
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	30
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	31 PLUGIN_INFO = {
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	32 C.PI_NAME: "tickets import",
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	33 C.PI_IMPORT_NAME: "TICKETS_IMPORT",
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	34 C.PI_TYPE: C.PLUG_TYPE_IMPORT,
3452 bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	35 C.PI_DEPENDENCIES: ["IMPORT", "XEP-0060", "XEP-0277", "XEP-0346"],
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	36 C.PI_MAIN: "TicketsImportPlugin",
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	37 C.PI_HANDLER: "no",
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	38 C.PI_DESCRIPTION: _(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	39 """Tickets import management:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	40 This plugin manage the different tickets importers which can register to it, and handle generic importing tasks."""
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	41 ),
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 }
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	43
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	44 OPT_MAPPING = "mapping"
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	45 FIELDS_LIST = ("labels", "cc_emails") # fields which must have a list as value
ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	46 FIELDS_DATE = ("created", "updated")
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	47
3452 bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	48 NS_TICKETS = "fdp/submitted/org.salut-a-toi.tickets:0"
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	49
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	50
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 class TicketsImportPlugin(object):
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	52 BOOL_OPTIONS = ()
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	53 JSON_OPTIONS = (OPT_MAPPING,)
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	54 OPT_DEFAULTS = {}
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	55
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	56 def __init__(self, host):
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	57 log.info(_("plugin Tickets import initialization"))
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	58 self.host = host
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	59 self._importers = {}
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	60 self._p = host.plugins["XEP-0060"]
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	61 self._m = host.plugins["XEP-0277"]
3452 bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	62 self._s = host.plugins["XEP-0346"]
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	63 host.plugins["IMPORT"].initialize(self, "tickets")
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	64
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	65 @defer.inlineCallbacks
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	66 def import_item(
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	67 self, client, item_import_data, session, options, return_data, service, node
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	68 ):
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	69 """
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	70
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	71 @param item_import_data(dict): no key is mandatory, but if a key doesn't exists in dest form, it will be ignored.
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	72 Following names are recommendations which should be used where suitable in importers.
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	73 except if specified in description, values are unicode
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	74 'id': unique id (must be unique in the node) of the ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	75 'title': title (or short description/summary) of the ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	76 'body': main description of the ticket
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	77 'created': date of creation (unix time)
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	78 'updated': date of last update (unix time)
2471 544c4d2fec45 plugins schema, merge_requests, tickets: factorisation Goffi <goffi@goffi.org>* parents: 2437 diff changeset	79 'author': full name of reporter
544c4d2fec45 plugins schema, merge_requests, tickets: factorisation Goffi <goffi@goffi.org>* parents: 2437 diff changeset	80 'author_jid': jid of reporter
544c4d2fec45 plugins schema, merge_requests, tickets: factorisation Goffi <goffi@goffi.org>* parents: 2437 diff changeset	81 'author_email': email of reporter
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	82 'assigned_to_name': full name of person working on it
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	83 'assigned_to_email': email of person working on it
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	84 'cc_emails': list of emails subscribed to the ticket
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	85 'priority': priority of the ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	86 'severity': severity of the ticket
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	87 'labels': list of unicode values to use as label
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	88 'product': product concerned by this ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	89 'component': part of the product concerned by this ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	90 'version': version of the product/component concerned by this ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	91 'platform': platform converned by this ticket
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	92 'os': operating system concerned by this ticket
2382 a49a19f06e38 plugin import ticket, import ticket bugzilla: some normalization in values: Goffi <goffi@goffi.org> parents: 2372 diff changeset	93 'status': current status of the ticket, values:
a49a19f06e38 plugin import ticket, import ticket bugzilla: some normalization in values: Goffi <goffi@goffi.org> parents: 2372 diff changeset	94 - "queued": ticket is waiting
a49a19f06e38 plugin import ticket, import ticket bugzilla: some normalization in values: Goffi <goffi@goffi.org> parents: 2372 diff changeset	95 - "started": progress is ongoing
a49a19f06e38 plugin import ticket, import ticket bugzilla: some normalization in values: Goffi <goffi@goffi.org> parents: 2372 diff changeset	96 - "review": ticket is fixed and waiting for review
a49a19f06e38 plugin import ticket, import ticket bugzilla: some normalization in values: Goffi <goffi@goffi.org> parents: 2372 diff changeset	97 - "closed": ticket is finished or invalid
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	98 'milestone': target milestone for this ticket
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	99 'comments': list of microblog data (comment metadata, check [XEP_0277.send] data argument)
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	100 @param options(dict, None): Below are the generic options,
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	101 tickets importer can have specific ones. All options are serialized unicode values
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	102 generic options:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	103 - OPT_MAPPING (json): dict of imported ticket key => exported ticket key
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	104 e.g.: if you want to map "component" to "labels", you can specify:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	105 {'component': 'labels'}
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	106 If you specify several import ticket key to the same dest key,
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	107 the values will be joined with line feeds
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	108 """
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	109 if "comments_uri" in item_import_data:
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	110 raise exceptions.DataError(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	111 _("comments_uri key will be generated and must not be used by importer")
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	112 )
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	113 for key in FIELDS_LIST:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	114 if not isinstance(item_import_data.get(key, []), list):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	115 raise exceptions.DataError(_("{key} must be a list").format(key=key))
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	116 for key in FIELDS_DATE:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	117 try:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	118 item_import_data[key] = utils.xmpp_date(item_import_data[key])
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	119 except KeyError:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	120 continue
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	121 if session["root_node"] is None:
ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	122 session["root_node"] = NS_TICKETS
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	123 if not "schema" in session:
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	124 session["schema"] = yield self._s.get_schema_form(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	125 client, service, node or session["root_node"]
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	126 )
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	127 defer.returnValue(item_import_data)
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	128
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	129 @defer.inlineCallbacks
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	130 def import_sub_items(self, client, item_import_data, ticket_data, session, options):
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	131 # TODO: force "open" permission (except if private, check below)
f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	132 # TODO: handle "private" metadata, to have non public access for node
2393 8ed4ac10cb5e plugin tickets import: create comments nodes with open access/publishing Goffi <goffi@goffi.org> parents: 2390 diff changeset	133 # TODO: node access/publish model should be customisable
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	134 comments = ticket_data.get("comments", [])
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	135 service = yield self._m.get_comments_service(client)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	136 node = self._m.get_comments_node(session["root_node"] + "_" + ticket_data["id"])
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	137 node_options = {
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	138 self._p.OPT_ACCESS_MODEL: self._p.ACCESS_OPEN,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	139 self._p.OPT_PERSIST_ITEMS: 1,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	140 self._p.OPT_DELIVER_PAYLOADS: 1,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	141 self._p.OPT_SEND_ITEM_SUBSCRIBE: 1,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	142 self._p.OPT_PUBLISH_MODEL: self._p.ACCESS_OPEN,
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	143 }
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	144 yield self._p.create_if_new_node(client, service, node, options=node_options)
524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	145 ticket_data["comments_uri"] = uri.build_xmpp_uri(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	146 "pubsub", subtype="microblog", path=service.full(), node=node
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	147 )
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	148 for comment in comments:
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	149 if "updated" not in comment and "published" in comment:
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	150 # we don't want an automatic update date
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	151 comment["updated"] = comment["published"]
2390 f57a8eaec8ed plugins import, tickets import, bugzilla import: comments handling: Goffi <goffi@goffi.org> parents: 2382 diff changeset	152 yield self._m.send(client, comment, service, node)
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	153
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	154 def publish_item(self, client, ticket_data, service, node, session):
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	155 if node is None:
95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	156 node = NS_TICKETS
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	157 id_ = ticket_data.pop("id", None)
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	158 log.debug(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	159 "uploading item [{id}]: {title}".format(
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	160 id=id_, title=ticket_data.get("title", "")
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	161 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	162 )
3452 bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	163 return defer.ensureDeferred(
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	164 self._s.send_data_form_item(
3452 bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	165 client, service, node, ticket_data, session["schema"], id_
bb0225aaf4e6 plugin XEP-0346: "Form Discovery and Publishing" implementation: Goffi <goffi@goffi.org> parents: 3137 diff changeset	166 )
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	167 )
2372 95a41c5f67c0 plugin tickets import: specialized importer for tickets, first draft: Goffi <goffi@goffi.org> parents: diff changeset	168
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 3931 diff changeset	169 def item_filters(self, client, ticket_data, session, options):
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	170 mapping = options.get(OPT_MAPPING)
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	171 if mapping is not None:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	172 if not isinstance(mapping, dict):
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	173 raise exceptions.DataError(_("mapping option must be a dictionary"))
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	174
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	175 for source, dest in mapping.items():
ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	176 if not isinstance(source, str) or not isinstance(dest, str):
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	177 raise exceptions.DataError(
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	178 _(
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	179 "keys and values of mapping must be sources and destinations ticket fields"
2624 56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	180 )
56f94936df1e code style reformatting using black Goffi <goffi@goffi.org> parents: 2562 diff changeset	181 )
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	182 if source in ticket_data:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	183 value = ticket_data.pop(source)
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	184 if dest in FIELDS_LIST:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	185 values = ticket_data[dest] = ticket_data.get(dest, [])
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	186 values.append(value)
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	187 else:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	188 if dest in ticket_data:
3028 ab2696e34d29 Python 3 port: Goffi <goffi@goffi.org> parents: 2771 diff changeset	189 ticket_data[dest] = ticket_data[dest] + "\n" + value
2396 66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	190 else:
66baa687c682 plugins tickets import, jp (ticket/import): implemented mapping: Goffi <goffi@goffi.org> parents: 2393 diff changeset	191 ticket_data[dest] = value

Mercurial > libervia-backend

annotate libervia/backend/plugins/plugin_tickets_import.py @ 4212:5f2d496c633f