Mercurial > libervia-backend

annotate sat/tools/stream.py @ 3983:31c3d6652115

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
component AP gateway: ignore actor delection notifications: When a `Delete` activity was received and the object was the emitting actor itself, the signature checking was failing if the actor was unknown (due to the impossibility to retrieve the actor public key, as it is no more accessible). To avoid that, those notifications are ignored for now. In the future they should clean the cache linked to this actor.
author Goffi <goffi@goffi.org>
date Tue, 15 Nov 2022 18:15:16 +0100
parents 0dd79c6cc1d2
children 524856bd7b19
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
3480
7550ae9cfbac Renamed the project from "Salut à Toi" to "Libervia":
Goffi <goffi@goffi.org>
parents: 3479
diff changeset
3 # Libervia: an XMPP client
3479
be6d91572633 date update
Goffi <goffi@goffi.org>
parents: 3182
diff changeset
4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 """ interfaces """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
20
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
21 from argparse import OPTIONAL
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
22 from pathlib import Path
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
23 from typing import Callable, Optional, Union
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
24 import uuid
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
25 import os
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
26 from zope import interface
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
27 from sat.core import exceptions
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 from sat.core.constants import Const as C
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
29 from sat.core.core_types import SatXMPPEntity
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 from sat.core.log import getLogger
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 from twisted.protocols import basic
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 from twisted.internet import interfaces
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
33
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
34 from sat.core.sat_main import SAT
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
35
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 log = getLogger(__name__)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
37
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
38
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 class IStreamProducer(interface.Interface):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
40 def startStream(consumer):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 """start producing the stream
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
42
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
43 @return (D): deferred fired when stream is finished
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 """
2866
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
45 pass
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
46
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
47
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
48 class SatFile:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 """A file-like object to have high level files manipulation"""
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
50
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 # TODO: manage "with" statement
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
53 def __init__(
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
54 self,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
55 host: SAT,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
56 client: SatXMPPEntity,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
57 path: Union[str, Path],
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
58 mode: str = "rb",
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
59 uid: Optional[str] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
60 size: Optional[int] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
61 data_cb: Optional[Callable] = None,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
62 auto_end_signals: bool = True,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
63 check_size_with_read: bool = False,
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
64 pre_close_cb: Optional[Callable]=None
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
65 ) -> None:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 @param host: %(doc_host)s
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
68 @param path(Path, str): path to the file to get or write to
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 @param mode(str): same as for built-in "open" function
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 @param uid(unicode, None): unique id identifing this progressing element
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 This uid will be used with self.host.progressGet
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 will be automaticaly generated if None
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 @param size(None, int): size of the file (when known in advance)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 @param data_cb(None, callable): method to call on each data read/write
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
75 can be used to do processing like calculating hash.
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
76 if data_cb return a non None value, it will be used instead of the
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
77 data read/to write
2866
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
78 @param auto_end_signals(bool): if True, progressFinished and progressError signals
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
79 are automatically sent.
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
80 if False, you'll have to call self.progressFinished and self.progressError
8ce5748bfe97 plugin XEP-0363: updated to namespace "urn:xmpp:http:upload:0", handle headers
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
81 yourself.
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
82 progressStarted signal is always sent automatically
3924
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
83 @param check_size_with_read(bool): if True, size will be checked using number of
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
84 bytes read or written. This is useful when data_cb modifiy len of file.
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
85 @param pre_close_cb:
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
86 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 self.host = host
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
88 self.profile = client.profile
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
89 self.uid = uid or str(uuid.uuid4())
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
90 self._file = open(path, mode)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
91 self.size = size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 self.data_cb = data_cb
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
93 self.auto_end_signals = auto_end_signals
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
94 self.pre_close_cb = pre_close_cb
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
95 metadata = self.getProgressMetadata()
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
96 self.host.registerProgressCb(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
97 self.uid, self.getProgress, metadata, profile=client.profile
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
98 )
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
99 self.host.bridge.progressStarted(self.uid, metadata, client.profile)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
100
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
101 self._transfer_count = 0 if check_size_with_read else None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
102
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
103 @property
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
104 def check_size_with_read(self):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
105 return self._transfer_count is not None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
106
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
107 @check_size_with_read.setter
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
108 def check_size_with_read(self, value):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
109 if value and self._transfer_count is None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
110 self._transfer_count = 0
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
111 else:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
112 self._transfer_count = None
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
113
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
114 def checkSize(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 """Check that current size correspond to given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
116
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 must be used when the transfer is supposed to be finished
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
118 @return (bool): True if the position is the same as given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 @raise exceptions.NotFound: size has not be specified
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
120 """
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
121 if self.check_size_with_read:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
122 position = self._transfer_count
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
123 else:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
124 position = self._file.tell()
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
125 if self.size is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 raise exceptions.NotFound
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
127 return position == self.size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
128
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
129 def close(self, progress_metadata=None, error=None):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 """Close the current file
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
131
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
132 @param progress_metadata(None, dict): metadata to send with _onProgressFinished
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
133 message
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
134 @param error(None, unicode): set to an error message if progress was not
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
135 successful
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
136 mutually exclusive with progress_metadata
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
137 error can happen even if error is None, if current size differ from given size
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
138 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
139 if self._file.closed:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
140 return # avoid double close (which is allowed) error
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
141 if self.pre_close_cb is not None:
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
142 self.pre_close_cb()
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
143 if error is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
144 try:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
145 size_ok = self.checkSize()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
146 except exceptions.NotFound:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
147 size_ok = True
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
148 if not size_ok:
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2866
diff changeset
149 error = "declared and actual size mismatch"
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
150 log.warning(error)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
151 progress_metadata = None
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
152
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
153 self._file.close()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
154
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
155 if self.auto_end_signals:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
156 if error is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
157 self.progressFinished(progress_metadata)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
158 else:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
159 assert progress_metadata is None
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
160 self.progressError(error)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
161
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
162 self.host.removeProgressCb(self.uid, self.profile)
3924
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
163 if error is not None:
e4631d073c8b tools (stream): log error when a SatFile is closed:
Goffi <goffi@goffi.org>
parents: 3480
diff changeset
164 log.error(f"file {self._file} closed with an error: {error}")
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
165
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
166 @property
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
167 def closed(self):
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
168 return self._file.closed
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
169
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
170 def progressFinished(self, metadata=None):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
171 if metadata is None:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
172 metadata = {}
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
173 self.host.bridge.progressFinished(self.uid, metadata, self.profile)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
174
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
175 def progressError(self, error):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
176 self.host.bridge.progressError(self.uid, error, self.profile)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
177
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
178 def flush(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
179 self._file.flush()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
180
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
181 def write(self, buf):
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
182 if self.data_cb is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
183 ret = self.data_cb(buf)
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
184 if ret is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
185 buf = ret
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
186 if self._transfer_count is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
187 self._transfer_count += len(buf)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
188 self._file.write(buf)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
189
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
190 def read(self, size=-1):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
191 read = self._file.read(size)
3087
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
192 if self.data_cb is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
193 ret = self.data_cb(read)
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
194 if ret is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
195 read = ret
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
196 if self._transfer_count is not None:
a51f7fce1e2c tools (stream): data modification on SatFile:
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
197 self._transfer_count += len(read)
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
198 return read
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
199
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
200 def seek(self, offset, whence=os.SEEK_SET):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
201 self._file.seek(offset, whence)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
202
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
203 def tell(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
204 return self._file.tell()
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
205
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
206 @property
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
207 def mode(self):
3968
0dd79c6cc1d2 tools (strem): fix `mode` + add `pre_close_cb` callback:
Goffi <goffi@goffi.org>
parents: 3924
diff changeset
208 return self._file.mode
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
209
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
210 def getProgressMetadata(self):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
211 """Return progression metadata as given to progressStarted
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
212
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
213 @return (dict): metadata (check bridge for documentation)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
214 """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
215 metadata = {"type": C.META_TYPE_FILE}
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
216
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
217 mode = self._file.mode
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
218 if "+" in mode:
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
219 pass # we have no direction in read/write modes
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
220 elif mode in ("r", "rb"):
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
221 metadata["direction"] = "out"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
222 elif mode in ("w", "wb"):
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
223 metadata["direction"] = "in"
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
224 elif "U" in mode:
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
225 metadata["direction"] = "out"
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
226 else:
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
227 raise exceptions.InternalError
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
228
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
229 metadata["name"] = self._file.name
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
230
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
231 return metadata
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
232
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
233 def getProgress(self, progress_id, profile):
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
234 ret = {"position": self._file.tell()}
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
235 if self.size:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
236 ret["size"] = self.size
2489
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
237 return ret
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
238
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
239
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
240 @interface.implementer(IStreamProducer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
241 @interface.implementer(interfaces.IConsumer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
242 class FileStreamObject(basic.FileSender):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
243 def __init__(self, host, client, path, **kwargs):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
244 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
245
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 A SatFile will be created and put in self.file_obj
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 @param path(unicode): path to the file
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 @param **kwargs: kw arguments to pass to SatFile
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
249 """
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
250 self.file_obj = SatFile(host, client, path, **kwargs)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
251
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 def registerProducer(self, producer, streaming):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
253 pass
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
254
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
255 def startStream(self, consumer):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 return self.beginFileTransfer(self.file_obj, consumer)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
257
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
258 def write(self, data):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
259 self.file_obj.write(data)
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
260
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
261 def close(self, *args, **kwargs):
e2a7bb875957 plugin pipe/stream, file transfert: refactoring and improvments:
Goffi <goffi@goffi.org>
parents:
diff changeset
262 self.file_obj.close(*args, **kwargs)