libervia-backend: sat/memory/sqla.py annotate

author	Goffi <goffi@goffi.org>
date	Thu, 03 Jun 2021 15:20:47 +0200
parents
children	71516731d0aa

rev	line source
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	2
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia: an XMPP client
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	5
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	10
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	15
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	18
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	19 import time
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	20 from typing import Dict, List, Tuple, Iterable, Any, Callable, Optional
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	21 from urllib.parse import quote
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	22 from pathlib import Path
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	23 from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	24 from sqlalchemy.exc import IntegrityError, NoResultFound
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	25 from sqlalchemy.orm import sessionmaker, subqueryload, contains_eager
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	26 from sqlalchemy.future import select
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	27 from sqlalchemy.engine import Engine
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	28 from sqlalchemy import update, delete, and_, or_, event
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	29 from sqlalchemy.sql.functions import coalesce, sum as sum_
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	30 from sqlalchemy.dialects.sqlite import insert
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	31 from twisted.internet import defer
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	32 from twisted.words.protocols.jabber import jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	33 from sat.core.i18n import _
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	34 from sat.core import exceptions
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	35 from sat.core.log import getLogger
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	36 from sat.core.constants import Const as C
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	37 from sat.core.core_types import SatXMPPEntity
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	38 from sat.tools.utils import aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	39 from sat.memory.sqla_mapping import (
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	40 NOT_IN_EXTRA,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	41 Base,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	42 Profile,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	43 Component,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	44 History,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	45 Message,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	46 Subject,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	47 Thread,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	48 ParamGen,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	49 ParamInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	50 PrivateGen,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	51 PrivateInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	52 PrivateGenBin,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	53 PrivateIndBin,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	54 File
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	55 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	56
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	57
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	58 log = getLogger(__name__)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	59
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	60
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	61 @event.listens_for(Engine, "connect")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	62 def set_sqlite_pragma(dbapi_connection, connection_record):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	63 cursor = dbapi_connection.cursor()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	64 cursor.execute("PRAGMA foreign_keys=ON")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	65 cursor.close()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	66
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	67
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	68 class Storage:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	69
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	70 def __init__(self, db_filename, sat_version):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	71 self.initialized = defer.Deferred()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	72 self.filename = Path(db_filename)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	73 # we keep cache for the profiles (key: profile name, value: profile id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	74 # profile id to name
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	75 self.profiles: Dict[int, str] = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	76 # profile id to component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	77 self.components: Dict[int, str] = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	78
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	79 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	80 async def initialise(self):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	81 log.info(_("Connecting database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	82 engine = create_async_engine(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	83 f"sqlite+aiosqlite:///{quote(str(self.filename))}",
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	84 future=True
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	85 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	86 self.session = sessionmaker(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	87 engine, expire_on_commit=False, class_=AsyncSession
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	88 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	89 new_base = not self.filename.exists()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	90 if new_base:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	91 log.info(_("The database is new, creating the tables"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	92 # the dir may not exist if it's not the XDG recommended one
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	93 self.filename.parent.mkdir(0o700, True, True)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	94 async with engine.begin() as conn:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	95 await conn.run_sync(Base.metadata.create_all)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	96
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	97 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	98 result = await session.execute(select(Profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	99 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	100 self.profiles[p.name] = p.id
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	101 result = await session.execute(select(Component))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	102 for c in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	103 self.components[c.profile_id] = c.entry_point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	104
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	105 self.initialized.callback(None)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	106
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	107 ## Profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	108
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	109 def getProfilesList(self) -> List[str]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	110 """"Return list of all registered profiles"""
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	111 return list(self.profiles.keys())
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	112
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	113 def hasProfile(self, profile_name: str) -> bool:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	114 """return True if profile_name exists
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	115
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	116 @param profile_name: name of the profile to check
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	117 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	118 return profile_name in self.profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	119
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	120 def profileIsComponent(self, profile_name: str) -> bool:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	121 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	122 return self.profiles[profile_name] in self.components
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	123 except KeyError:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	124 raise exceptions.NotFound("the requested profile doesn't exists")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	125
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	126 def getEntryPoint(self, profile_name: str) -> str:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	127 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	128 return self.components[self.profiles[profile_name]]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	129 except KeyError:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	130 raise exceptions.NotFound("the requested profile doesn't exists or is not a component")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	131
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	132 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	133 async def createProfile(self, name: str, component_ep: Optional[str] = None) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	134 """Create a new profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	135
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	136 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	137 @param component: if not None, must point to a component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	138 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	139 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	140 profile = Profile(name=name)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	141 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	142 session.add(profile)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	143 self.profiles[profile.id] = profile.name
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	144 if component_ep is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	145 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	146 component = Component(profile=profile, entry_point=component_ep)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	147 session.add(component)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	148 self.components[profile.id] = component_ep
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	149 return profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	150
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	151 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	152 async def deleteProfile(self, name: str) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	153 """Delete profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	154
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	155 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	156 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	157 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	158 result = await session.execute(select(Profile).where(Profile.name == name))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	159 profile = result.scalar()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	160 await session.delete(profile)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	161 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	162 del self.profiles[profile.id]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	163 if profile.id in self.components:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	164 del self.components[profile.id]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	165 log.info(_("Profile {name!r} deleted").format(name = name))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	166
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	167 ## Params
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	168
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	169 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	170 async def loadGenParams(self, params_gen: dict) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	171 """Load general parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	172
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	173 @param params_gen: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	174 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	175 log.debug(_("loading general parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	176 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	177 result = await session.execute(select(ParamGen))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	178 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	179 params_gen[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	180
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	181 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	182 async def loadIndParams(self, params_ind: dict, profile: str) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	183 """Load individual parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	184
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	185 @param params_ind: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	186 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	187 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	188 log.debug(_("loading individual parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	189 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	190 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	191 select(ParamInd).where(ParamInd.profile_id == self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	192 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	193 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	194 params_ind[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	195
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	196 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	197 async def getIndParam(self, category: str, name: str, profile: str) -> Optional[str]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	198 """Ask database for the value of one specific individual parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	199
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	200 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	201 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	202 @param profile: %(doc_profile)s
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	203 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	204 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	205 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	206 select(ParamInd.value)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	207 .filter_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	208 category=category,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	209 name=name,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	210 profile_id=self.profiles[profile]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	211 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	212 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	213 return result.scalar_one_or_none()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	214
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	215 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	216 async def getIndParamValues(self, category: str, name: str) -> Dict[str, str]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	217 """Ask database for the individual values of a parameter for all profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	218
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	219 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	220 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	221 @return dict: profile => value map
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	222 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	223 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	224 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	225 select(ParamInd)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	226 .filter_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	227 category=category,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	228 name=name
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	229 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	230 .options(subqueryload(ParamInd.profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	231 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	232 return {param.profile.name: param.value for param in result.scalars()}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	233
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	234 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	235 async def setGenParam(self, category: str, name: str, value: Optional[str]) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	236 """Save the general parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	237
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	238 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	239 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	240 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	241 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	242 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	243 stmt = insert(ParamGen).values(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	244 category=category,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	245 name=name,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	246 value=value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	247 ).on_conflict_do_update(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	248 index_elements=(ParamGen.category, ParamGen.name),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	249 set_={
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	250 ParamGen.value: value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	251 }
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	252 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	253 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	254 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	255
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	256 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	257 async def setIndParam(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	258 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	259 category:str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	260 name: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	261 value: Optional[str],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	262 profile: str
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	263 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	264 """Save the individual parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	265
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	266 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	267 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	268 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	269 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	270 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	271 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	272 stmt = insert(ParamInd).values(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	273 category=category,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	274 name=name,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	275 profile_id=self.profiles[profile],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	276 value=value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	277 ).on_conflict_do_update(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	278 index_elements=(ParamInd.category, ParamInd.name, ParamInd.profile_id),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	279 set_={
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	280 ParamInd.value: value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	281 }
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	282 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	283 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	284 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	285
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	286 def _jid_filter(self, jid_: jid.JID, dest: bool = False):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	287 """Generate condition to filter on a JID, using relevant columns
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	288
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	289 @param dest: True if it's the destinee JID, otherwise it's the source one
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	290 @param jid_: JID to filter by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	291 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	292 if jid_.resource:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	293 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	294 return and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	295 History.dest == jid_.userhost(),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	296 History.dest_res == jid_.resource
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	297 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	298 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	299 return and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	300 History.source == jid_.userhost(),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	301 History.source_res == jid_.resource
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	302 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	303 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	304 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	305 return History.dest == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	306 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	307 return History.source == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	308
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	309 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	310 async def historyGet(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	311 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	312 from_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	313 to_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	314 limit: Optional[int] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	315 between: bool = True,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	316 filters: Optional[Dict[str, str]] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	317 profile: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	318 ) -> List[Tuple[
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	319 str, int, str, str, Dict[str, str], Dict[str, str], str, str, str]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	320 ]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	321 """Retrieve messages in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	322
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	323 @param from_jid: source JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	324 @param to_jid: dest JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	325 @param limit: maximum number of messages to get:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	326 - 0 for no message (returns the empty list)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	327 - None for unlimited
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	328 @param between: confound source and dest (ignore the direction)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	329 @param filters: pattern to filter the history results
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	330 @return: list of messages as in [messageNew], minus the profile which is already
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	331 known.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	332 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	333 # we have to set a default value to profile because it's last argument
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	334 # and thus follow other keyword arguments with default values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	335 # but None should not be used for it
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	336 assert profile is not None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	337 if limit == 0:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	338 return []
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	339 if filters is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	340 filters = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	341
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	342 stmt = (
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	343 select(History)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	344 .filter_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	345 profile_id=self.profiles[profile]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	346 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	347 .outerjoin(History.messages)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	348 .outerjoin(History.subjects)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	349 .outerjoin(History.thread)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	350 .options(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	351 contains_eager(History.messages),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	352 contains_eager(History.subjects),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	353 contains_eager(History.thread),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	354 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	355 .order_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	356 # timestamp may be identical for 2 close messages (specially when delay is
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	357 # used) that's why we order ties by received_timestamp. We'll reverse the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	358 # order when returning the result. We use DESC here so LIMIT keep the last
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	359 # messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	360 History.timestamp.desc(),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	361 History.received_timestamp.desc()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	362 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	363 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	364
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	365
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	366 if not from_jid and not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	367 # no jid specified, we want all one2one communications
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	368 pass
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	369 elif between:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	370 if not from_jid or not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	371 # we only have one jid specified, we check all messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	372 # from or to this jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	373 jid_ = from_jid or to_jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	374 stmt = stmt.where(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	375 or_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	376 self._jid_filter(jid_),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	377 self._jid_filter(jid_, dest=True)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	378 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	379 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	380 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	381 # we have 2 jids specified, we check all communications between
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	382 # those 2 jids
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	383 stmt = stmt.where(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	384 or_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	385 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	386 self._jid_filter(from_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	387 self._jid_filter(to_jid, dest=True),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	388 ),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	389 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	390 self._jid_filter(to_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	391 self._jid_filter(from_jid, dest=True),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	392 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	393 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	394 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	395 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	396 # we want one communication in specific direction (from somebody or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	397 # to somebody).
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	398 if from_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	399 stmt = stmt.where(self._jid_filter(from_jid))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	400 if to_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	401 stmt = stmt.where(self._jid_filter(to_jid, dest=True))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	402
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	403 if filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	404 if 'timestamp_start' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	405 stmt = stmt.where(History.timestamp >= float(filters['timestamp_start']))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	406 if 'before_uid' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	407 # orignially this query was using SQLITE's rowid. This has been changed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	408 # to use coalesce(received_timestamp, timestamp) to be SQL engine independant
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	409 stmt = stmt.where(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	410 coalesce(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	411 History.received_timestamp,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	412 History.timestamp
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	413 ) < (
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	414 select(coalesce(History.received_timestamp, History.timestamp))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	415 .filter_by(uid=filters["before_uid"])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	416 ).scalar_subquery()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	417 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	418 if 'body' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	419 # TODO: use REGEXP (function to be defined) instead of GLOB: https://www.sqlite.org/lang_expr.html
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	420 stmt = stmt.where(Message.message.like(f"%{filters['body']}%"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	421 if 'search' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	422 search_term = f"%{filters['search']}%"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	423 stmt = stmt.where(or_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	424 Message.message.like(search_term),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	425 History.source_res.like(search_term)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	426 ))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	427 if 'types' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	428 types = filters['types'].split()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	429 stmt = stmt.where(History.type.in_(types))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	430 if 'not_types' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	431 types = filters['not_types'].split()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	432 stmt = stmt.where(History.type.not_in(types))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	433 if 'last_stanza_id' in filters:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	434 # this request get the last message with a "stanza_id" that we
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	435 # have in history. This is mainly used to retrieve messages sent
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	436 # while we were offline, using MAM (XEP-0313).
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	437 if (filters['last_stanza_id'] is not True
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	438 or limit != 1):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	439 raise ValueError("Unexpected values for last_stanza_id filter")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	440 stmt = stmt.where(History.stanza_id.is_not(None))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	441
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	442 if limit is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	443 stmt = stmt.limit(limit)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	444
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	445 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	446 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	447
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	448 result = result.scalars().unique().all()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	449 result.reverse()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	450 return [h.as_tuple() for h in result]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	451
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	452 @aio
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	453 async def addToHistory(self, data: dict, profile: str) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	454 """Store a new message in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	455
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	456 @param data: message data as build by SatMessageProtocol.onMessage
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	457 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	458 extra = {k: v for k, v in data["extra"].items() if k not in NOT_IN_EXTRA}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	459 messages = [Message(message=mess, language=lang)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	460 for lang, mess in data["message"].items()]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	461 subjects = [Subject(subject=mess, language=lang)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	462 for lang, mess in data["subject"].items()]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	463 if "thread" in data["extra"]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	464 thread = Thread(thread_id=data["extra"]["thread"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	465 parent_id=data["extra"].get["thread_parent"])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	466 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	467 thread = None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	468 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	469 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	470 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	471 session.add(History(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	472 uid=data["uid"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	473 stanza_id=data["extra"].get("stanza_id"),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	474 update_uid=data["extra"].get("update_uid"),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	475 profile_id=self.profiles[profile],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	476 source_jid=data["from"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	477 dest_jid=data["to"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	478 timestamp=data["timestamp"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	479 received_timestamp=data.get("received_timestamp"),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	480 type=data["type"],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	481 extra=extra,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	482 messages=messages,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	483 subjects=subjects,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	484 thread=thread,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	485 ))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	486 except IntegrityError as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	487 if "unique" in str(e.orig).lower():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	488 log.debug(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	489 f"message {data['uid']!r} is already in history, not storing it again"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	490 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	491 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	492 log.error(f"Can't store message {data['uid']!r} in history: {e}")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	493 except Exception as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	494 log.critical(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	495 f"Can't store message, unexpected exception (uid: {data['uid']}): {e}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	496 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	497
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	498 ## Private values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	499
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	500 def _getPrivateClass(self, binary, profile):

3537

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

1 #!/usr/bin/env python3

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

2

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

3 # Libervia: an XMPP client

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

5

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

6 # This program is free software: you can redistribute it and/or modify

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

7 # it under the terms of the GNU Affero General Public License as published by

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

8 # the Free Software Foundation, either version 3 of the License, or

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

9 # (at your option) any later version.

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

10

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

11 # This program is distributed in the hope that it will be useful,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

12 # but WITHOUT ANY WARRANTY; without even the implied warranty of

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

14 # GNU Affero General Public License for more details.

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

15

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

16 # You should have received a copy of the GNU Affero General Public License

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

17 # along with this program. If not, see <http://www.gnu.org/licenses/>.

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

18

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

19 import time

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

20 from typing import Dict, List, Tuple, Iterable, Any, Callable, Optional

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

21 from urllib.parse import quote

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

22 from pathlib import Path

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

23 from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

24 from sqlalchemy.exc import IntegrityError, NoResultFound

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

25 from sqlalchemy.orm import sessionmaker, subqueryload, contains_eager

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

26 from sqlalchemy.future import select

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

27 from sqlalchemy.engine import Engine

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

28 from sqlalchemy import update, delete, and_, or_, event

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

29 from sqlalchemy.sql.functions import coalesce, sum as sum_

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

30 from sqlalchemy.dialects.sqlite import insert

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

31 from twisted.internet import defer

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

32 from twisted.words.protocols.jabber import jid

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

33 from sat.core.i18n import _

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

34 from sat.core import exceptions

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

35 from sat.core.log import getLogger

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

36 from sat.core.constants import Const as C

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

37 from sat.core.core_types import SatXMPPEntity

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

38 from sat.tools.utils import aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

39 from sat.memory.sqla_mapping import (

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

40 NOT_IN_EXTRA,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

41 Base,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

42 Profile,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

43 Component,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

44 History,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

45 Message,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

46 Subject,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

47 Thread,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

48 ParamGen,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

49 ParamInd,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

50 PrivateGen,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

51 PrivateInd,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

52 PrivateGenBin,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

53 PrivateIndBin,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

54 File

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

55 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

56

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

57

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

58 log = getLogger(__name__)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

59

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

60

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

61 @event.listens_for(Engine, "connect")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

62 def set_sqlite_pragma(dbapi_connection, connection_record):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

63 cursor = dbapi_connection.cursor()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

64 cursor.execute("PRAGMA foreign_keys=ON")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

65 cursor.close()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

66

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

67

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

68 class Storage:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

69

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

70 def __init__(self, db_filename, sat_version):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

71 self.initialized = defer.Deferred()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

72 self.filename = Path(db_filename)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

73 # we keep cache for the profiles (key: profile name, value: profile id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

74 # profile id to name

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

75 self.profiles: Dict[int, str] = {}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

76 # profile id to component entry point

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

77 self.components: Dict[int, str] = {}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

78

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

79 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

80 async def initialise(self):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

81 log.info(_("Connecting database"))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

82 engine = create_async_engine(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

83 f"sqlite+aiosqlite:///{quote(str(self.filename))}",

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

84 future=True

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

85 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

86 self.session = sessionmaker(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

87 engine, expire_on_commit=False, class_=AsyncSession

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

88 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

89 new_base = not self.filename.exists()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

90 if new_base:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

91 log.info(_("The database is new, creating the tables"))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

92 # the dir may not exist if it's not the XDG recommended one

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

93 self.filename.parent.mkdir(0o700, True, True)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

94 async with engine.begin() as conn:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

95 await conn.run_sync(Base.metadata.create_all)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

96

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

97 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

98 result = await session.execute(select(Profile))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

99 for p in result.scalars():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

100 self.profiles[p.name] = p.id

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

101 result = await session.execute(select(Component))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

102 for c in result.scalars():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

103 self.components[c.profile_id] = c.entry_point

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

104

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

105 self.initialized.callback(None)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

106

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

107 ## Profiles

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

108

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

109 def getProfilesList(self) -> List[str]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

110 """"Return list of all registered profiles"""

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

111 return list(self.profiles.keys())

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

112

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

113 def hasProfile(self, profile_name: str) -> bool:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

114 """return True if profile_name exists

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

115

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

116 @param profile_name: name of the profile to check

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

117 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

118 return profile_name in self.profiles

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

119

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

120 def profileIsComponent(self, profile_name: str) -> bool:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

121 try:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

122 return self.profiles[profile_name] in self.components

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

123 except KeyError:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

124 raise exceptions.NotFound("the requested profile doesn't exists")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

125

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

126 def getEntryPoint(self, profile_name: str) -> str:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

127 try:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

128 return self.components[self.profiles[profile_name]]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

129 except KeyError:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

130 raise exceptions.NotFound("the requested profile doesn't exists or is not a component")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

131

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

132 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

133 async def createProfile(self, name: str, component_ep: Optional[str] = None) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

134 """Create a new profile

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

135

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

136 @param name: name of the profile

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

137 @param component: if not None, must point to a component entry point

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

138 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

139 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

140 profile = Profile(name=name)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

141 async with session.begin():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

142 session.add(profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

143 self.profiles[profile.id] = profile.name

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

144 if component_ep is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

145 async with session.begin():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

146 component = Component(profile=profile, entry_point=component_ep)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

147 session.add(component)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

148 self.components[profile.id] = component_ep

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

149 return profile

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

150

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

151 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

152 async def deleteProfile(self, name: str) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

153 """Delete profile

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

154

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

155 @param name: name of the profile

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

156 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

157 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

158 result = await session.execute(select(Profile).where(Profile.name == name))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

159 profile = result.scalar()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

160 await session.delete(profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

161 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

162 del self.profiles[profile.id]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

163 if profile.id in self.components:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

164 del self.components[profile.id]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

165 log.info(_("Profile {name!r} deleted").format(name = name))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

166

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

167 ## Params

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

168

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

169 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

170 async def loadGenParams(self, params_gen: dict) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

171 """Load general parameters

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

172

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

173 @param params_gen: dictionary to fill

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

174 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

175 log.debug(_("loading general parameters from database"))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

176 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

177 result = await session.execute(select(ParamGen))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

178 for p in result.scalars():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

179 params_gen[(p.category, p.name)] = p.value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

180

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

181 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

182 async def loadIndParams(self, params_ind: dict, profile: str) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

183 """Load individual parameters

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

184

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

185 @param params_ind: dictionary to fill

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

186 @param profile: a profile which *must* exist

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

187 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

188 log.debug(_("loading individual parameters from database"))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

189 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

190 result = await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

191 select(ParamInd).where(ParamInd.profile_id == self.profiles[profile])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

192 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

193 for p in result.scalars():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

194 params_ind[(p.category, p.name)] = p.value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

195

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

196 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

197 async def getIndParam(self, category: str, name: str, profile: str) -> Optional[str]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

198 """Ask database for the value of one specific individual parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

199

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

200 @param category: category of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

201 @param name: name of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

202 @param profile: %(doc_profile)s

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

203 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

204 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

205 result = await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

206 select(ParamInd.value)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

207 .filter_by(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

208 category=category,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

209 name=name,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

210 profile_id=self.profiles[profile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

211 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

212 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

213 return result.scalar_one_or_none()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

214

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

215 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

216 async def getIndParamValues(self, category: str, name: str) -> Dict[str, str]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

217 """Ask database for the individual values of a parameter for all profiles

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

218

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

219 @param category: category of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

220 @param name: name of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

221 @return dict: profile => value map

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

222 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

223 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

224 result = await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

225 select(ParamInd)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

226 .filter_by(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

227 category=category,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

228 name=name

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

229 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

230 .options(subqueryload(ParamInd.profile))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

231 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

232 return {param.profile.name: param.value for param in result.scalars()}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

233

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

234 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

235 async def setGenParam(self, category: str, name: str, value: Optional[str]) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

236 """Save the general parameters in database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

237

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

238 @param category: category of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

239 @param name: name of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

240 @param value: value to set

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

241 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

242 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

243 stmt = insert(ParamGen).values(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

244 category=category,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

245 name=name,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

246 value=value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

247 ).on_conflict_do_update(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

248 index_elements=(ParamGen.category, ParamGen.name),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

249 set_={

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

250 ParamGen.value: value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

251 }

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

252 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

253 await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

254 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

255

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

256 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

257 async def setIndParam(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

258 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

259 category:str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

260 name: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

261 value: Optional[str],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

262 profile: str

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

263 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

264 """Save the individual parameters in database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

265

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

266 @param category: category of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

267 @param name: name of the parameter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

268 @param value: value to set

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

269 @param profile: a profile which *must* exist

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

270 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

271 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

272 stmt = insert(ParamInd).values(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

273 category=category,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

274 name=name,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

275 profile_id=self.profiles[profile],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

276 value=value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

277 ).on_conflict_do_update(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

278 index_elements=(ParamInd.category, ParamInd.name, ParamInd.profile_id),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

279 set_={

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

280 ParamInd.value: value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

281 }

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

282 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

283 await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

284 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

285

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

286 def _jid_filter(self, jid_: jid.JID, dest: bool = False):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

287 """Generate condition to filter on a JID, using relevant columns

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

288

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

289 @param dest: True if it's the destinee JID, otherwise it's the source one

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

290 @param jid_: JID to filter by

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

291 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

292 if jid_.resource:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

293 if dest:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

294 return and_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

295 History.dest == jid_.userhost(),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

296 History.dest_res == jid_.resource

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

297 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

298 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

299 return and_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

300 History.source == jid_.userhost(),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

301 History.source_res == jid_.resource

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

302 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

303 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

304 if dest:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

305 return History.dest == jid_.userhost()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

306 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

307 return History.source == jid_.userhost()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

308

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

309 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

310 async def historyGet(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

311 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

312 from_jid: Optional[jid.JID],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

313 to_jid: Optional[jid.JID],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

314 limit: Optional[int] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

315 between: bool = True,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

316 filters: Optional[Dict[str, str]] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

317 profile: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

318 ) -> List[Tuple[

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

319 str, int, str, str, Dict[str, str], Dict[str, str], str, str, str]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

320 ]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

321 """Retrieve messages in history

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

322

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

323 @param from_jid: source JID (full, or bare for catchall)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

324 @param to_jid: dest JID (full, or bare for catchall)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

325 @param limit: maximum number of messages to get:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

326 - 0 for no message (returns the empty list)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

327 - None for unlimited

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

328 @param between: confound source and dest (ignore the direction)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

329 @param filters: pattern to filter the history results

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

330 @return: list of messages as in [messageNew], minus the profile which is already

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

331 known.

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

332 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

333 # we have to set a default value to profile because it's last argument

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

334 # and thus follow other keyword arguments with default values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

335 # but None should not be used for it

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

336 assert profile is not None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

337 if limit == 0:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

338 return []

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

339 if filters is None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

340 filters = {}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

341

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

342 stmt = (

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

343 select(History)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

344 .filter_by(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

345 profile_id=self.profiles[profile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

346 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

347 .outerjoin(History.messages)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

348 .outerjoin(History.subjects)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

349 .outerjoin(History.thread)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

350 .options(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

351 contains_eager(History.messages),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

352 contains_eager(History.subjects),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

353 contains_eager(History.thread),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

354 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

355 .order_by(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

356 # timestamp may be identical for 2 close messages (specially when delay is

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

357 # used) that's why we order ties by received_timestamp. We'll reverse the

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

358 # order when returning the result. We use DESC here so LIMIT keep the last

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

359 # messages

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

360 History.timestamp.desc(),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

361 History.received_timestamp.desc()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

362 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

363 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

364

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

365

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

366 if not from_jid and not to_jid:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

367 # no jid specified, we want all one2one communications

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

368 pass

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

369 elif between:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

370 if not from_jid or not to_jid:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

371 # we only have one jid specified, we check all messages

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

372 # from or to this jid

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

373 jid_ = from_jid or to_jid

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

374 stmt = stmt.where(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

375 or_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

376 self._jid_filter(jid_),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

377 self._jid_filter(jid_, dest=True)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

378 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

379 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

380 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

381 # we have 2 jids specified, we check all communications between

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

382 # those 2 jids

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

383 stmt = stmt.where(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

384 or_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

385 and_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

386 self._jid_filter(from_jid),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

387 self._jid_filter(to_jid, dest=True),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

388 ),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

389 and_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

390 self._jid_filter(to_jid),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

391 self._jid_filter(from_jid, dest=True),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

392 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

393 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

394 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

395 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

396 # we want one communication in specific direction (from somebody or

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

397 # to somebody).

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

398 if from_jid is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

399 stmt = stmt.where(self._jid_filter(from_jid))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

400 if to_jid is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

401 stmt = stmt.where(self._jid_filter(to_jid, dest=True))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

402

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

403 if filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

404 if 'timestamp_start' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

405 stmt = stmt.where(History.timestamp >= float(filters['timestamp_start']))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

406 if 'before_uid' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

407 # orignially this query was using SQLITE's rowid. This has been changed

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

408 # to use coalesce(received_timestamp, timestamp) to be SQL engine independant

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

409 stmt = stmt.where(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

410 coalesce(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

411 History.received_timestamp,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

412 History.timestamp

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

413 ) < (

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

414 select(coalesce(History.received_timestamp, History.timestamp))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

415 .filter_by(uid=filters["before_uid"])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

416 ).scalar_subquery()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

417 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

418 if 'body' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

419 # TODO: use REGEXP (function to be defined) instead of GLOB: https://www.sqlite.org/lang_expr.html

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

420 stmt = stmt.where(Message.message.like(f"%{filters['body']}%"))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

421 if 'search' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

422 search_term = f"%{filters['search']}%"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

423 stmt = stmt.where(or_(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

424 Message.message.like(search_term),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

425 History.source_res.like(search_term)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

426 ))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

427 if 'types' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

428 types = filters['types'].split()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

429 stmt = stmt.where(History.type.in_(types))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

430 if 'not_types' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

431 types = filters['not_types'].split()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

432 stmt = stmt.where(History.type.not_in(types))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

433 if 'last_stanza_id' in filters:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

434 # this request get the last message with a "stanza_id" that we

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

435 # have in history. This is mainly used to retrieve messages sent

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

436 # while we were offline, using MAM (XEP-0313).

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

437 if (filters['last_stanza_id'] is not True

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

438 or limit != 1):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

439 raise ValueError("Unexpected values for last_stanza_id filter")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

440 stmt = stmt.where(History.stanza_id.is_not(None))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

441

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

442 if limit is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

443 stmt = stmt.limit(limit)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

444

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

445 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

446 result = await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

447

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

448 result = result.scalars().unique().all()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

449 result.reverse()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

450 return [h.as_tuple() for h in result]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

451

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

452 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

453 async def addToHistory(self, data: dict, profile: str) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

454 """Store a new message in history

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

455

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

456 @param data: message data as build by SatMessageProtocol.onMessage

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

457 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

458 extra = {k: v for k, v in data["extra"].items() if k not in NOT_IN_EXTRA}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

459 messages = [Message(message=mess, language=lang)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

460 for lang, mess in data["message"].items()]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

461 subjects = [Subject(subject=mess, language=lang)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

462 for lang, mess in data["subject"].items()]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

463 if "thread" in data["extra"]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

464 thread = Thread(thread_id=data["extra"]["thread"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

465 parent_id=data["extra"].get["thread_parent"])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

466 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

467 thread = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

468 try:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

469 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

470 async with session.begin():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

471 session.add(History(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

472 uid=data["uid"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

473 stanza_id=data["extra"].get("stanza_id"),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

474 update_uid=data["extra"].get("update_uid"),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

475 profile_id=self.profiles[profile],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

476 source_jid=data["from"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

477 dest_jid=data["to"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

478 timestamp=data["timestamp"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

479 received_timestamp=data.get("received_timestamp"),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

480 type=data["type"],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

481 extra=extra,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

482 messages=messages,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

483 subjects=subjects,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

484 thread=thread,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

485 ))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

486 except IntegrityError as e:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

487 if "unique" in str(e.orig).lower():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

488 log.debug(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

489 f"message {data['uid']!r} is already in history, not storing it again"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

490 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

491 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

492 log.error(f"Can't store message {data['uid']!r} in history: {e}")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

493 except Exception as e:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

494 log.critical(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

495 f"Can't store message, unexpected exception (uid: {data['uid']}): {e}"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

496 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

497

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

498 ## Private values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

499

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

500 def _getPrivateClass(self, binary, profile):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

501 """Get ORM class to use for private values"""

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

502 if profile is None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

503 return PrivateGenBin if binary else PrivateGen

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

504 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

505 return PrivateIndBin if binary else PrivateInd

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

506

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

507

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

508 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

509 async def getPrivates(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

510 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

511 namespace:str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

512 keys: Optional[Iterable[str]] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

513 binary: bool = False,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

514 profile: Optional[str] = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

515 ) -> Dict[str, Any]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

516 """Get private value(s) from databases

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

517

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

518 @param namespace: namespace of the values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

519 @param keys: keys of the values to get None to get all keys/values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

520 @param binary: True to deserialise binary values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

521 @param profile: profile to use for individual values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

522 None to use general values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

523 @return: gotten keys/values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

524 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

525 if keys is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

526 keys = list(keys)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

527 log.debug(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

528 f"getting {'general' if profile is None else 'individual'}"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

529 f"{' binary' if binary else ''} private values from database for namespace "

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

530 f"{namespace}{f' with keys {keys!r}' if keys is not None else ''}"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

531 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

532 cls = self._getPrivateClass(binary, profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

533 stmt = select(cls).filter_by(namespace=namespace)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

534 if keys:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

535 stmt = stmt.where(cls.key.in_(list(keys)))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

536 if profile is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

537 stmt = stmt.filter_by(profile_id=self.profiles[profile])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

538 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

539 result = await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

540 return {p.key: p.value for p in result.scalars()}

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

541

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

542 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

543 async def setPrivateValue(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

544 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

545 namespace: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

546 key:str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

547 value: Any,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

548 binary: bool = False,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

549 profile: Optional[str] = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

550 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

551 """Set a private value in database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

552

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

553 @param namespace: namespace of the values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

554 @param key: key of the value to set

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

555 @param value: value to set

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

556 @param binary: True if it's a binary values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

557 binary values need to be serialised, used for everything but strings

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

558 @param profile: profile to use for individual value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

559 if None, it's a general value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

560 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

561 cls = self._getPrivateClass(binary, profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

562

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

563 values = {

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

564 "namespace": namespace,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

565 "key": key,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

566 "value": value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

567 }

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

568 index_elements = [cls.namespace, cls.key]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

569

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

570 if profile is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

571 values["profile_id"] = self.profiles[profile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

572 index_elements.append(cls.profile_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

573

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

574 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

575 await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

576 insert(cls).values(**values).on_conflict_do_update(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

577 index_elements=index_elements,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

578 set_={

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

579 cls.value: value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

580 }

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

581 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

582 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

583 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

584

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

585 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

586 async def delPrivateValue(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

587 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

588 namespace: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

589 key: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

590 binary: bool = False,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

591 profile: Optional[str] = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

592 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

593 """Delete private value from database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

594

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

595 @param category: category of the privateeter

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

596 @param key: key of the private value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

597 @param binary: True if it's a binary values

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

598 @param profile: profile to use for individual value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

599 if None, it's a general value

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

600 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

601 cls = self._getPrivateClass(binary, profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

602

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

603 stmt = delete(cls).filter_by(namespace=namespace, key=key)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

604

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

605 if profile is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

606 stmt = stmt.filter_by(profile_id=self.profiles[profile])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

607

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

608 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

609 await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

610 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

611

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

612 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

613 async def delPrivateNamespace(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

614 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

615 namespace: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

616 binary: bool = False,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

617 profile: Optional[str] = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

618 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

619 """Delete all data from a private namespace

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

620

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

621 Be really cautious when you use this method, as all data with given namespace are

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

622 removed.

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

623 Params are the same as for delPrivateValue

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

624 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

625 cls = self._getPrivateClass(binary, profile)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

626

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

627 stmt = delete(cls).filter_by(namespace=namespace)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

628

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

629 if profile is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

630 stmt = stmt.filter_by(profile_id=self.profiles[profile])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

631

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

632 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

633 await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

634 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

635

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

636 ## Files

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

637

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

638 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

639 async def getFiles(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

640 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

641 client: Optional[SatXMPPEntity],

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

642 file_id: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

643 version: Optional[str] = '',

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

644 parent: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

645 type_: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

646 file_hash: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

647 hash_algo: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

648 name: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

649 namespace: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

650 mime_type: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

651 public_id: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

652 owner: Optional[jid.JID] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

653 access: Optional[dict] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

654 projection: Optional[List[str]] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

655 unique: bool = False

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

656 ) -> List[dict]:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

657 """Retrieve files with with given filters

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

658

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

659 @param file_id: id of the file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

660 None to ignore

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

661 @param version: version of the file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

662 None to ignore

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

663 empty string to look for current version

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

664 @param parent: id of the directory containing the files

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

665 None to ignore

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

666 empty string to look for root files/directories

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

667 @param projection: name of columns to retrieve

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

668 None to retrieve all

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

669 @param unique: if True will remove duplicates

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

670 other params are the same as for [setFile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

671 @return: files corresponding to filters

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

672 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

673 if projection is None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

674 projection = [

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

675 'id', 'version', 'parent', 'type', 'file_hash', 'hash_algo', 'name',

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

676 'size', 'namespace', 'media_type', 'media_subtype', 'public_id',

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

677 'created', 'modified', 'owner', 'access', 'extra'

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

678 ]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

679

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

680 stmt = select(*[getattr(File, f) for f in projection])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

681

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

682 if unique:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

683 stmt = stmt.distinct()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

684

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

685 if client is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

686 stmt = stmt.filter_by(profile_id=self.profiles[client.profile])

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

687 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

688 if public_id is None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

689 raise exceptions.InternalError(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

690 "client can only be omitted when public_id is set"

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

691 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

692 if file_id is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

693 stmt = stmt.filter_by(id=file_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

694 if version is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

695 stmt = stmt.filter_by(version=version)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

696 if parent is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

697 stmt = stmt.filter_by(parent=parent)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

698 if type_ is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

699 stmt = stmt.filter_by(type=type_)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

700 if file_hash is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

701 stmt = stmt.filter_by(file_hash=file_hash)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

702 if hash_algo is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

703 stmt = stmt.filter_by(hash_algo=hash_algo)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

704 if name is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

705 stmt = stmt.filter_by(name=name)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

706 if namespace is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

707 stmt = stmt.filter_by(namespace=namespace)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

708 if mime_type is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

709 if '/' in mime_type:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

710 media_type, media_subtype = mime_type.split("/", 1)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

711 stmt = stmt.filter_by(media_type=media_type, media_subtype=media_subtype)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

712 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

713 stmt = stmt.filter_by(media_type=mime_type)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

714 if public_id is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

715 stmt = stmt.filter_by(public_id=public_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

716 if owner is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

717 stmt = stmt.filter_by(owner=owner)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

718 if access is not None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

719 raise NotImplementedError('Access check is not implemented yet')

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

720 # a JSON comparison is needed here

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

721

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

722 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

723 result = await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

724

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

725 return [dict(r) for r in result]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

726

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

727 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

728 async def setFile(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

729 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

730 client: SatXMPPEntity,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

731 name: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

732 file_id: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

733 version: str = "",

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

734 parent: str = "",

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

735 type_: str = C.FILE_TYPE_FILE,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

736 file_hash: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

737 hash_algo: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

738 size: int = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

739 namespace: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

740 mime_type: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

741 public_id: Optional[str] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

742 created: Optional[float] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

743 modified: Optional[float] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

744 owner: Optional[jid.JID] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

745 access: Optional[dict] = None,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

746 extra: Optional[dict] = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

747 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

748 """Set a file metadata

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

749

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

750 @param client: client owning the file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

751 @param name: name of the file (must not contain "/")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

752 @param file_id: unique id of the file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

753 @param version: version of this file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

754 @param parent: id of the directory containing this file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

755 Empty string if it is a root file/directory

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

756 @param type_: one of:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

757 - file

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

758 - directory

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

759 @param file_hash: unique hash of the payload

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

760 @param hash_algo: algorithm used for hashing the file (usually sha-256)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

761 @param size: size in bytes

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

762 @param namespace: identifier (human readable is better) to group files

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

763 for instance, namespace could be used to group files in a specific photo album

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

764 @param mime_type: media type of the file, or None if not known/guessed

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

765 @param public_id: ID used to server the file publicly via HTTP

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

766 @param created: UNIX time of creation

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

767 @param modified: UNIX time of last modification, or None to use created date

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

768 @param owner: jid of the owner of the file (mainly useful for component)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

769 @param access: serialisable dictionary with access rules. See [memory.memory] for details

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

770 @param extra: serialisable dictionary of any extra data

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

771 will be encoded to json in database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

772 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

773 if mime_type is None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

774 media_type = media_subtype = None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

775 elif '/' in mime_type:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

776 media_type, media_subtype = mime_type.split('/', 1)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

777 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

778 media_type, media_subtype = mime_type, None

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

779

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

780 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

781 async with session.begin():

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

782 session.add(File(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

783 id=file_id,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

784 version=version.strip(),

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

785 parent=parent,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

786 type=type_,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

787 file_hash=file_hash,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

788 hash_algo=hash_algo,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

789 name=name,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

790 size=size,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

791 namespace=namespace,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

792 media_type=media_type,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

793 media_subtype=media_subtype,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

794 public_id=public_id,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

795 created=time.time() if created is None else created,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

796 modified=modified,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

797 owner=owner,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

798 access=access,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

799 extra=extra,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

800 profile_id=self.profiles[client.profile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

801 ))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

802

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

803 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

804 async def fileGetUsedSpace(self, client: SatXMPPEntity, owner: jid.JID) -> int:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

805 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

806 result = await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

807 select(sum_(File.size)).filter_by(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

808 owner=owner,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

809 type=C.FILE_TYPE_FILE,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

810 profile_id=self.profiles[client.profile]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

811 ))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

812 return result.scalar_one_or_none() or 0

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

813

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

814 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

815 async def fileDelete(self, file_id: str) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

816 """Delete file metadata from the database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

817

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

818 @param file_id: id of the file to delete

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

819 NOTE: file itself must still be removed, this method only handle metadata in

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

820 database

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

821 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

822 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

823 await session.execute(delete(File).filter_by(id=file_id))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

824 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

825

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

826 @aio

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

827 async def fileUpdate(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

828 self,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

829 file_id: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

830 column: str,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

831 update_cb: Callable[[dict], None]

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

832 ) -> None:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

833 """Update a column value using a method to avoid race conditions

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

834

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

835 the older value will be retrieved from database, then update_cb will be applied to

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

836 update it, and file will be updated checking that older value has not been changed

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

837 meanwhile by an other user. If it has changed, it tries again a couple of times

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

838 before failing

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

839 @param column: column name (only "access" or "extra" are allowed)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

840 @param update_cb: method to update the value of the colum

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

841 the method will take older value as argument, and must update it in place

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

842 update_cb must not care about serialization,

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

843 it get the deserialized data (i.e. a Python object) directly

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

844 @raise exceptions.NotFound: there is not file with this id

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

845 """

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

846 if column not in ('access', 'extra'):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

847 raise exceptions.InternalError('bad column name')

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

848 orm_col = getattr(File, column)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

849

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

850 for i in range(5):

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

851 async with self.session() as session:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

852 try:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

853 value = (await session.execute(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

854 select(orm_col).filter_by(id=file_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

855 )).scalar_one()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

856 except NoResultFound:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

857 raise exceptions.NotFound

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

858 update_cb(value)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

859 stmt = update(orm_col).filter_by(id=file_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

860 if not value:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

861 # because JsonDefaultDict convert NULL to an empty dict, we have to

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

862 # test both for empty dict and None when we have and empty dict

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

863 stmt = stmt.where((orm_col == None) | (orm_col == value))

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

864 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

865 stmt = stmt.where(orm_col == value)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

866 result = await session.execute(stmt)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

867 await session.commit()

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

868

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

869 if result.rowcount == 1:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

870 break

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

871

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

872 log.warning(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

873 _("table not updated, probably due to race condition, trying again "

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

874 "({tries})").format(tries=i+1)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

875 )

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

876

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

877 else:

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

878 raise exceptions.DatabaseError(

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

879 _("Can't update file {file_id} due to race condition")

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

880 .format(file_id=file_id)

f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy

Goffi <goffi@goffi.org>

parents:

diff changeset

881 )

Mercurial > libervia-backend

annotate sat/memory/sqla.py @ 3537:f9a5b810f14d