Mercurial > libervia-backend

comparison libervia/backend/memory/sqla.py @ 4212:5f2d496c633f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
core: get rid of `pickle`: Use of `pickle` to serialise data was a technical legacy that was causing trouble to store in database, to update (if a class was serialised, a change could break update), and to security (pickle can lead to code execution). This patch remove all use of Pickle in favour in JSON, notably: - for caching data, a Pydantic model is now used instead - for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation - in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and `to_dict` method have been implemented to do serialisation. - new methods to (de)serialise data can now be specified with Identity data types. It is notably used to (de)serialise `path` of avatars. A migration script has been created to convert data (for upgrade or downgrade), with special care for XEP-0373 case. Depending of size of database, this migration script can be long to run. rel 443
author Goffi <goffi@goffi.org>
date Fri, 23 Feb 2024 13:31:04 +0100
parents 6a0066ea5c97
children 0d7bb4df2343
comparison
equal deleted inserted replaced
4211:be89ab1cbca4 4212:5f2d496c633f
18 18
19 import asyncio 19 import asyncio
20 from asyncio.subprocess import PIPE 20 from asyncio.subprocess import PIPE
21 import copy 21 import copy
22 from datetime import datetime 22 from datetime import datetime
23 import json
23 from pathlib import Path 24 from pathlib import Path
24 import sys 25 import sys
25 import time 26 import time
26 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union 27 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union
27 28
212 213
213 db_config = sqla_config.get_db_config() 214 db_config = sqla_config.get_db_config()
214 engine = create_async_engine( 215 engine = create_async_engine(
215 db_config["url"], 216 db_config["url"],
216 future=True, 217 future=True,
218 json_serializer=lambda obj: json.dumps(obj, ensure_ascii=False)
217 ) 219 )
218 220
219 new_base = not db_config["path"].exists() 221 new_base = not db_config["path"].exists()
220 if new_base: 222 if new_base:
221 log.info(_("The database is new, creating the tables")) 223 log.info(_("The database is new, creating the tables"))