diff libervia/backend/memory/sqla.py @ 4212:5f2d496c633f

core: get rid of `pickle`: Use of `pickle` to serialise data was a technical legacy that was causing trouble to store in database, to update (if a class was serialised, a change could break update), and to security (pickle can lead to code execution). This patch remove all use of Pickle in favour in JSON, notably: - for caching data, a Pydantic model is now used instead - for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation - in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and `to_dict` method have been implemented to do serialisation. - new methods to (de)serialise data can now be specified with Identity data types. It is notably used to (de)serialise `path` of avatars. A migration script has been created to convert data (for upgrade or downgrade), with special care for XEP-0373 case. Depending of size of database, this migration script can be long to run. rel 443
author Goffi <goffi@goffi.org>
date Fri, 23 Feb 2024 13:31:04 +0100
parents 6a0066ea5c97
children 0d7bb4df2343
line wrap: on
line diff
--- a/libervia/backend/memory/sqla.py	Fri Feb 16 18:46:06 2024 +0100
+++ b/libervia/backend/memory/sqla.py	Fri Feb 23 13:31:04 2024 +0100
@@ -20,6 +20,7 @@
 from asyncio.subprocess import PIPE
 import copy
 from datetime import datetime
+import json
 from pathlib import Path
 import sys
 import time
@@ -214,6 +215,7 @@
         engine = create_async_engine(
             db_config["url"],
             future=True,
+            json_serializer=lambda obj: json.dumps(obj, ensure_ascii=False)
         )
 
         new_base = not db_config["path"].exists()