Mercurial > libervia-backend

annotate libervia/backend/tools/stream.py @ 4219:1b5cf2ee1d86

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
plugin XEP-0384, XEP-0391: download missing devices list: when a peer jid was not in our roster, devices list was not retrieved, resulting in failed en/decryption. This patch does check it and download missing devices list in necessary. There is no subscription managed yet, so the list won't be updated in case of new devices, this should be addressed at some point.
author Goffi <goffi@goffi.org>
date Tue, 05 Mar 2024 17:31:36 +0100
parents 7c5654c54fed
children 0d7bb4df2343
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
3480
7550ae9cfbac Renamed the project from "Salut à Toi" to "Libervia":
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
3 # Libervia: an XMPP client
3479
be6d91572633 date update
Goffi <goffi@goffi.org>
parents: 3182
diff changeset
4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 """ interfaces """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
20
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
21 from argparse import OPTIONAL
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
22 from pathlib import Path
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
23 from typing import Callable, Optional, Union
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
24 import uuid
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
25 import os
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
26 from zope import interface
4071
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
27 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
28 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
29 from libervia.backend.core.core_types import SatXMPPEntity
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend`
Goffi <goffi@goffi.org>
parents: 4037
diff changeset
30 from libervia.backend.core.log import getLogger
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 from twisted.protocols import basic
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 from twisted.internet import interfaces
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
33
4073
7c5654c54fed refactoring: rename `core.sat_main` to `core.main`
Goffi <goffi@goffi.org>
parents: 4072
diff changeset
34 from libervia.backend.core.main import LiberviaBackend
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
35
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 log = getLogger(__name__)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
37
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
38
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 class IStreamProducer(interface.Interface):
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
40 def start_stream(consumer):
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 """start producing the stream
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
42
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
43 @return (D): deferred fired when stream is finished
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 """
2866
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
45 pass
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
46
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
47
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
48 class SatFile:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 """A file-like object to have high level files manipulation"""
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
50
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 # TODO: manage "with" statement
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
53 def __init__(
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
54 self,
4072
040095a5dc7f refactoring: rename `SAT` class to `LiberviaBackend`
Goffi <goffi@goffi.org>
parents: 4071
diff changeset
55 host: LiberviaBackend,
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
56 client: SatXMPPEntity,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
57 path: Union[str, Path],
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
58 mode: str = "rb",
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
59 uid: Optional[str] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
60 size: Optional[int] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
61 data_cb: Optional[Callable] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
62 auto_end_signals: bool = True,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
63 check_size_with_read: bool = False,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
64 pre_close_cb: Optional[Callable]=None
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
65 ) -> None:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 @param host: %(doc_host)s
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
68 @param path(Path, str): path to the file to get or write to
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 @param mode(str): same as for built-in "open" function
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 @param uid(unicode, None): unique id identifing this progressing element
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
71 This uid will be used with self.host.progress_get
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 will be automaticaly generated if None
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 @param size(None, int): size of the file (when known in advance)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 @param data_cb(None, callable): method to call on each data read/write
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
75 can be used to do processing like calculating hash.
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
76 if data_cb return a non None value, it will be used instead of the
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
77 data read/to write
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
78 @param auto_end_signals(bool): if True, progress_finished and progress_error signals
2866
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
79 are automatically sent.
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
80 if False, you'll have to call self.progress_finished and self.progress_error
2866
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
81 yourself.
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
82 progress_started signal is always sent automatically
3924
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
83 @param check_size_with_read(bool): if True, size will be checked using number of
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
84 bytes read or written. This is useful when data_cb modifiy len of file.
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
85 @param pre_close_cb:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
86 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 self.host = host
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
88 self.profile = client.profile
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
89 self.uid = uid or str(uuid.uuid4())
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
90 self._file = open(path, mode)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
91 self.size = size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 self.data_cb = data_cb
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
93 self.auto_end_signals = auto_end_signals
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
94 self.pre_close_cb = pre_close_cb
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
95 metadata = self.get_progress_metadata()
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
96 self.host.register_progress_cb(
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
97 self.uid, self.get_progress, metadata, profile=client.profile
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
98 )
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
99 self.host.bridge.progress_started(self.uid, metadata, client.profile)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
100
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
101 self._transfer_count = 0 if check_size_with_read else None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
102
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
103 @property
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
104 def check_size_with_read(self):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
105 return self._transfer_count is not None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
106
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
107 @check_size_with_read.setter
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
108 def check_size_with_read(self, value):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
109 if value and self._transfer_count is None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
110 self._transfer_count = 0
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
111 else:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
112 self._transfer_count = None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
113
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
114 def check_size(self):
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 """Check that current size correspond to given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
116
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 must be used when the transfer is supposed to be finished
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
118 @return (bool): True if the position is the same as given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 @raise exceptions.NotFound: size has not be specified
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
120 """
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
121 if self.check_size_with_read:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
122 position = self._transfer_count
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
123 else:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
124 position = self._file.tell()
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
125 if self.size is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 raise exceptions.NotFound
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
127 return position == self.size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
128
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
129 def close(self, progress_metadata=None, error=None):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 """Close the current file
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
131
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
132 @param progress_metadata(None, dict): metadata to send with _onProgressFinished
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
133 message
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
134 @param error(None, unicode): set to an error message if progress was not
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
135 successful
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
136 mutually exclusive with progress_metadata
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
137 error can happen even if error is None, if current size differ from given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
138 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
139 if self._file.closed:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
140 return # avoid double close (which is allowed) error
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
141 if self.pre_close_cb is not None:
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
142 self.pre_close_cb()
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
143 if error is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
144 try:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
145 size_ok = self.check_size()
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
146 except exceptions.NotFound:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
147 size_ok = True
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
148 if not size_ok:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
149 error = "declared and actual size mismatch"
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
150 log.warning(error)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
151 progress_metadata = None
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
152
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
153 self._file.close()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
154
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
155 if self.auto_end_signals:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
156 if error is None:
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
157 self.progress_finished(progress_metadata)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
158 else:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
159 assert progress_metadata is None
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
160 self.progress_error(error)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
161
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
162 self.host.remove_progress_cb(self.uid, self.profile)
3924
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
163 if error is not None:
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
164 log.error(f"file {self._file} closed with an error: {error}")
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
165
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
166 @property
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
167 def closed(self):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
168 return self._file.closed
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
169
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
170 def progress_finished(self, metadata=None):
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
171 if metadata is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
172 metadata = {}
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
173 self.host.bridge.progress_finished(self.uid, metadata, self.profile)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
174
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
175 def progress_error(self, error):
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
176 self.host.bridge.progress_error(self.uid, error, self.profile)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
177
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
178 def flush(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
179 self._file.flush()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
180
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
181 def write(self, buf):
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
182 if self.data_cb is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
183 ret = self.data_cb(buf)
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
184 if ret is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
185 buf = ret
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
186 if self._transfer_count is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
187 self._transfer_count += len(buf)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
188 self._file.write(buf)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
189
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
190 def read(self, size=-1):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
191 read = self._file.read(size)
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
192 if self.data_cb is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
193 ret = self.data_cb(read)
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
194 if ret is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
195 read = ret
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
196 if self._transfer_count is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
197 self._transfer_count += len(read)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
198 return read
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
199
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
200 def seek(self, offset, whence=os.SEEK_SET):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
201 self._file.seek(offset, whence)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
202
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
203 def tell(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
204 return self._file.tell()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
205
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
206 @property
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
207 def mode(self):
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
208 return self._file.mode
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
209
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
210 def get_progress_metadata(self):
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
211 """Return progression metadata as given to progress_started
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
212
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
213 @return (dict): metadata (check bridge for documentation)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
214 """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
215 metadata = {"type": C.META_TYPE_FILE}
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
216
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
217 mode = self._file.mode
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
218 if "+" in mode:
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
219 pass # we have no direction in read/write modes
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
220 elif mode in ("r", "rb"):
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
221 metadata["direction"] = "out"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
222 elif mode in ("w", "wb"):
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
223 metadata["direction"] = "in"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
224 elif "U" in mode:
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
225 metadata["direction"] = "out"
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
226 else:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
227 raise exceptions.InternalError
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
228
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
229 metadata["name"] = self._file.name
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
230
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
231 return metadata
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
232
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
233 def get_progress(self, progress_id, profile):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
234 ret = {"position": self._file.tell()}
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
235 if self.size:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
236 ret["size"] = self.size
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
237 return ret
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
238
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
239
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
240 @interface.implementer(IStreamProducer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
241 @interface.implementer(interfaces.IConsumer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
242 class FileStreamObject(basic.FileSender):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
243 def __init__(self, host, client, path, **kwargs):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
244 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
245
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 A SatFile will be created and put in self.file_obj
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 @param path(unicode): path to the file
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 @param **kwargs: kw arguments to pass to SatFile
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
249 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
250 self.file_obj = SatFile(host, client, path, **kwargs)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
251
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 def registerProducer(self, producer, streaming):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
253 pass
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
254
4037
524856bd7b19 massive refactoring to switch from camelCase to snake_case:
Goffi <goffi@goffi.org>
parents: 3968
diff changeset
255 def start_stream(self, consumer):
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 return self.beginFileTransfer(self.file_obj, consumer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
257
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
258 def write(self, data):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
259 self.file_obj.write(data)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
260
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
261 def close(self, *args, **kwargs):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
262 self.file_obj.close(*args, **kwargs)