libervia-backend: sat/plugins/plugin_misc

annotate sat/plugins/plugin_misc_attach.py @ 4002:5245b675f7ad

plugin XEP-0313: don't wait for MAM to be retrieved in connection workflow: MAM retrieval can be long, and can be done after connection, message just need to be sorted when being inserted (i.e. frontends must do insort). To avoid blocking connection for too long and result in bad UX and timeout risk, one2one MAM message are not retrieved in background.

author	Goffi <goffi@goffi.org>
date	Fri, 10 Mar 2023 17:22:45 +0100
parents	0ff265725489
children	78b5f356900c

rev	line source
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	2
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	3 # SàT plugin for attaching files
3479 be6d91572633 date update Goffi <goffi@goffi.org> parents: 3223 diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	5
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	18
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	19 from collections import namedtuple
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	20 import mimetypes
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	21 from pathlib import Path
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	22 import shutil
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	23 import tempfile
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	24 from typing import Callable, Optional
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	25
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	26 from twisted.internet import defer
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	27
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	28 from sat.core import exceptions
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	29 from sat.core.constants import Const as C
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	30 from sat.core.core_types import SatXMPPEntity
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	31 from sat.core.i18n import _
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	32 from sat.core.log import getLogger
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	33 from sat.tools import utils
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	34 from sat.tools import image
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	35
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	36
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	37 log = getLogger(__name__)
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	38
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	39
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	40 PLUGIN_INFO = {
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	41 C.PI_NAME: "File Attach",
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 C.PI_IMPORT_NAME: "ATTACH",
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	43 C.PI_TYPE: C.PLUG_TYPE_MISC,
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	44 C.PI_MODES: C.PLUG_MODE_BOTH,
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	45 C.PI_DEPENDENCIES: ["UPLOAD"],
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	46 C.PI_MAIN: "AttachPlugin",
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	47 C.PI_HANDLER: "no",
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 C.PI_DESCRIPTION: _("""Attachments handler"""),
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	49 }
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	50
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	51
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	52 AttachmentHandler = namedtuple('AttachmentHandler', ['can_handle', 'attach', 'priority'])
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	53
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	54
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	55 class AttachPlugin:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	56
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	57 def __init__(self, host):
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	58 log.info(_("plugin Attach initialization"))
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	59 self.host = host
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	60 self._u = host.plugins["UPLOAD"]
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	61 host.trigger.add("sendMessage", self._sendMessageTrigger)
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	62 self._attachments_handlers = {'clear': [], 'encrypted': []}
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	63 self.register(self.defaultCanHandle, self.defaultAttach, False, -1000)
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	64
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	65 def register(self, can_handle, attach, encrypted=False, priority=0):
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	66 """Register an attachments handler
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	67
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	68 @param can_handle(callable, coroutine, Deferred): a method which must return True
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	69 if this plugin can handle the upload, otherwise next ones will be tried.
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	70 This method will get client and mess_data as arguments, before the XML is
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	71 generated
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	72 @param attach(callable, coroutine, Deferred): attach the file
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	73 this method will get client and mess_data as arguments, after XML is
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	74 generated. Upload operation must be handled
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	75 hint: "UPLOAD" plugin can be used
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	76 @param encrypted(bool): True if the handler manages encrypted files
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	77 A handler can be registered twice if it handle both encrypted and clear
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	78 attachments
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	79 @param priority(int): priority of this handler, handler with higher priority will
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	80 be tried first
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	81 """
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	82 handler = AttachmentHandler(can_handle, attach, priority)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	83 handlers = (
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	84 self._attachments_handlers['encrypted']
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	85 if encrypted else self._attachments_handlers['clear']
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	86 )
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	87 if handler in handlers:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	88 raise exceptions.InternalError(
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	89 'Attachment handler has been registered twice, this should never happen'
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	90 )
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	91
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	92 handlers.append(handler)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	93 handlers.sort(key=lambda h: h.priority, reverse=True)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	94 log.debug(f"new attachments handler: {handler}")
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	95
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	96 async def attachFiles(self, client, data):
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	97 """Main method to attach file
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	98
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	99 It will do generic pre-treatment, and call the suitable attachments handler
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	100 """
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	101 # we check attachment for pre-treatment like large image resizing
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	102 # media_type will be added if missing (and if it can be guessed from path)
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	103 attachments = data["extra"][C.MESS_KEY_ATTACHMENTS]
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	104 tmp_dirs_to_clean = []
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	105 for attachment in attachments:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	106 if attachment.get(C.MESS_KEY_ATTACHMENTS_RESIZE, False):
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	107 path = Path(attachment["path"])
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	108 try:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	109 media_type = attachment[C.MESS_KEY_ATTACHMENTS_MEDIA_TYPE]
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	110 except KeyError:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	111 media_type = mimetypes.guess_type(path, strict=False)[0]
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	112 if media_type is None:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	113 log.warning(
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	114 _("Can't resize attachment of unknown type: {attachment}")
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	115 .format(attachment=attachment))
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	116 continue
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	117 attachment[C.MESS_KEY_ATTACHMENTS_MEDIA_TYPE] = media_type
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	118
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	119 main_type = media_type.split('/')[0]
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	120 if main_type == "image":
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	121 report = image.check(self.host, path)
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	122 if report['too_large']:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	123 tmp_dir = Path(tempfile.mkdtemp())
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	124 tmp_dirs_to_clean.append(tmp_dir)
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	125 new_path = tmp_dir / path.name
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	126 await image.resize(
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	127 path, report["recommended_size"], dest=new_path)
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	128 attachment["path"] = new_path
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	129 log.info(
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	130 _("Attachment {path!r} has been resized at {new_path!r}")
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	131 .format(path=str(path), new_path=str(new_path)))
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	132 else:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	133 log.warning(
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	134 _("Can't resize attachment of type {main_type!r}: {attachment}")
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	135 .format(main_type=main_type, attachment=attachment))
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	136
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	137 if client.encryption.isEncryptionRequested(data):
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	138 handlers = self._attachments_handlers['encrypted']
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	139 else:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	140 handlers = self._attachments_handlers['clear']
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	141
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	142 for handler in handlers:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	143 can_handle = await utils.asDeferred(handler.can_handle, client, data)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	144 if can_handle:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	145 break
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	146 else:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	147 raise exceptions.NotFound(
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	148 _("No plugin can handle attachment with {destinee}").format(
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	149 destinee = data['to']
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	150 ))
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	151
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	152 await utils.asDeferred(handler.attach, client, data)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	153
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	154 for dir_path in tmp_dirs_to_clean:
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	155 log.debug(f"Cleaning temporary directory at {dir_path}")
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	156 shutil.rmtree(dir_path)
163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	157
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	158 return data
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	159
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	160 async def upload_files(
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	161 self,
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	162 client: SatXMPPEntity,
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	163 data: dict,
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	164 upload_cb: Optional[Callable] = None
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	165 ):
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	166 """Upload file, and update attachments
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	167
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	168 invalid attachments will be removed
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	169 @param client:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	170 @param data(dict): message data
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	171 @param upload_cb(coroutine, Deferred, None): method to use for upload
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	172 if None, upload method from UPLOAD plugin will be used.
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	173 Otherwise, following kwargs will be used with the cb:
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	174 - client
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	175 - filepath
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	176 - filename
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	177 - options
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	178 the method must return a tuple similar to UPLOAD plugin's upload method,
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	179 it must contain:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	180 - progress_id
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	181 - a deferred which fire download URL
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	182 """
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	183 if upload_cb is None:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	184 upload_cb = self._u.upload
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	185
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	186 uploads_d = []
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	187 to_delete = []
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	188 attachments = data["extra"]["attachments"]
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	189
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	190 for attachment in attachments:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	191 try:
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	192 # we pop path because we don't want it to be stored, as the file can be
3202 2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	193 # only in a temporary location
2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	194 path = Path(attachment.pop("path"))
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	195 except KeyError:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	196 log.warning("no path in attachment: {attachment}")
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	197 to_delete.append(attachment)
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	198 continue
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	199
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	200 if "url" in attachment:
3202 2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	201 url = attachment.pop('url')
2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	202 log.warning(
2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	203 f"unexpected URL in attachment: {url!r}\nattachment: {attachment}"
2e892f9f54f6 plugin attachment: remove "path" from attachment once used: Goffi <goffi@goffi.org> parents: 3192 diff changeset	204 )
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	205
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	206 try:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	207 name = attachment["name"]
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 except KeyError:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	209 name = attachment["name"] = path.name
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	210
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	211 attachment["size"] = path.stat().st_size
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	212
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	213 extra = {
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	214 "attachment": attachment
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	215 }
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	216 progress_id = attachment.pop("progress_id", None)
3182 f2bb57348587 plugin attach, XEP-0363: progress id can now be specified: Goffi <goffi@goffi.org> parents: 3181 diff changeset	217 if progress_id:
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	218 extra["progress_id"] = progress_id
3192 883fb4981958 plugin attach: disable TLS check if "check_certificate" is disabled Goffi <goffi@goffi.org> parents: 3182 diff changeset	219 check_certificate = self.host.memory.getParamA(
883fb4981958 plugin attach: disable TLS check if "check_certificate" is disabled Goffi <goffi@goffi.org> parents: 3182 diff changeset	220 "check_certificate", "Connection", profile_key=client.profile)
883fb4981958 plugin attach: disable TLS check if "check_certificate" is disabled Goffi <goffi@goffi.org> parents: 3182 diff changeset	221 if not check_certificate:
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	222 extra['ignore_tls_errors'] = True
3192 883fb4981958 plugin attach: disable TLS check if "check_certificate" is disabled Goffi <goffi@goffi.org> parents: 3182 diff changeset	223 log.warning(
883fb4981958 plugin attach: disable TLS check if "check_certificate" is disabled Goffi <goffi@goffi.org> parents: 3182 diff changeset	224 _("certificate check disabled for upload, this is dangerous!"))
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	225
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	226 __, upload_d = await upload_cb(
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	227 client=client,
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 filepath=path,
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	229 filename=name,
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	230 extra=extra,
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	231 )
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	232 uploads_d.append(upload_d)
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	233
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	234 for attachment in to_delete:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	235 attachments.remove(attachment)
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	236
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	237 upload_results = await defer.DeferredList(uploads_d)
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	238 for idx, (success, ret) in enumerate(upload_results):
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	239 attachment = attachments[idx]
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	240
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	241 if not success:
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	242 # ret is a failure here
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	243 log.warning(f"error while uploading {attachment}: {ret}")
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	244 continue
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	245
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 attachment["url"] = ret
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	247
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	248 return data
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	249
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	250 def _attachFiles(self, data, client):
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	251 return defer.ensureDeferred(self.attachFiles(client, data))
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	252
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	253 def _sendMessageTrigger(
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	254 self, client, mess_data, pre_xml_treatments, post_xml_treatments):
5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	255 if mess_data['extra'].get(C.MESS_KEY_ATTACHMENTS):
3223 163014f09bf4 plugin attach: handle large images resizing: Goffi <goffi@goffi.org> parents: 3219 diff changeset	256 post_xml_treatments.addCallback(self._attachFiles, client=client)
3181 5ff2cf7f0aba plugin attach: first draft: Goffi <goffi@goffi.org> parents: diff changeset	257 return True
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	258
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	259 async def defaultCanHandle(self, client, data):
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	260 return True
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	261
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	262 async def defaultAttach(self, client, data):
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	263 await self.upload_files(client, data)
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	264 # TODO: handle xhtml-im
3922 0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	265 body_elt = data["xml"].body
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	266 if body_elt is None:
0ff265725489 plugin XEP-0447: handle attachment and download: Goffi <goffi@goffi.org> parents: 3859 diff changeset	267 body_elt = data["xml"].addElement("body")
3219 2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	268 attachments = data["extra"][C.MESS_KEY_ATTACHMENTS]
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	269 if attachments:
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	270 body_links = '\n'.join(a['url'] for a in attachments)
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	271 if str(body_elt).strip():
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	272 # if there is already a body, we add a line feed before the first link
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	273 body_elt.addContent('\n')
2ba602aef90e plugin attach, aesgcm: attachments refactoring: Goffi <goffi@goffi.org> parents: 3202 diff changeset	274 body_elt.addContent(body_links)

Mercurial > libervia-backend

annotate sat/plugins/plugin_misc_attach.py @ 4002:5245b675f7ad