Mercurial > libervia-backend

annotate libervia/backend/tools/common/files_utils.py @ 4212:5f2d496c633f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
core: get rid of `pickle`: Use of `pickle` to serialise data was a technical legacy that was causing trouble to store in database, to update (if a class was serialised, a change could break update), and to security (pickle can lead to code execution). This patch remove all use of Pickle in favour in JSON, notably: - for caching data, a Pydantic model is now used instead - for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation - in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and `to_dict` method have been implemented to do serialisation. - new methods to (de)serialise data can now be specified with Identity data types. It is notably used to (de)serialise `path` of avatars. A migration script has been created to convert data (for upgrade or downgrade), with special care for XEP-0373 case. Depending of size of database, this migration script can be long to run. rel 443
author Goffi <goffi@goffi.org>
date Fri, 23 Feb 2024 13:31:04 +0100
parents 4b842c1fb686
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3028
ab2696e34d29 Python 3 port:
Goffi <goffi@goffi.org>
parents: 2771
diff changeset
1 #!/usr/bin/env python3
3137
559a625a236b fixed shebangs
Goffi <goffi@goffi.org>
parents: 3136
diff changeset
2
3368
e86b71b1aa31 core: minor typos, docstring/comments update
Goffi <goffi@goffi.org>
parents: 3246
diff changeset
3 # SaT: an XMPP client
3479
be6d91572633 date update
Goffi <goffi@goffi.org>
parents: 3368
diff changeset
4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 """tools to help manipulating files"""
3091
2e1c3d33099b tools (common/files_utils): get_unique_name now uses and returns Path instances.
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
20 from pathlib import Path
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
21
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
22
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 def get_unique_name(path):
3091
2e1c3d33099b tools (common/files_utils): get_unique_name now uses and returns Path instances.
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
24 """Generate a path with a name not conflicting with existing file
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
25
3091
2e1c3d33099b tools (common/files_utils): get_unique_name now uses and returns Path instances.
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
26 @param path(str, Path): path to the file to create
2e1c3d33099b tools (common/files_utils): get_unique_name now uses and returns Path instances.
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
27 @return (Path): unique path (can be the same as path if there is no conflict)
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 """
3131
d6da17f6e4ce tools (common/files_utils): fixed get_unique_name when `path` is a str and not a Path
Goffi <goffi@goffi.org>
parents: 3091
diff changeset
29 ori_path = path = Path(path)
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 idx = 1
3091
2e1c3d33099b tools (common/files_utils): get_unique_name now uses and returns Path instances.
Goffi <goffi@goffi.org>
parents: 3028
diff changeset
31 while path.exists():
3246
5ba0b1cdd45b tools (common/files_utils): put extra suffix before file extension
Goffi <goffi@goffi.org>
parents: 3137
diff changeset
32 path = ori_path.with_name(f"{ori_path.stem}_{idx}{ori_path.suffix}")
2593
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 idx += 1
d78eff6b4487 tools (common): new files_utils module:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 return path