libervia-backend: libervia/backend/memory/sqla.py annotate

annotate libervia/backend/memory/sqla.py @ 4202:b26339343076

core: use a user specific directory for PID file: default location of pid file is now specific to logged user, this allow to run several instances of Libervia by different users on the same machine without PID conflicts.

author	Goffi <goffi@goffi.org>
date	Sun, 14 Jan 2024 17:48:02 +0100
parents	6a0066ea5c97
children	5f2d496c633f

rev	line source
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	2
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia: an XMPP client
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	5
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	10
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	15
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	18
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	19 import asyncio
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	20 from asyncio.subprocess import PIPE
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	21 import copy
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	22 from datetime import datetime
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	23 from pathlib import Path
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	24 import sys
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	25 import time
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	26 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	27
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	28 from alembic import config as al_config, script as al_script
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	29 from alembic.runtime import migration as al_migration
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	30 from sqlalchemy import and_, delete, event, func, or_, update
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	31 from sqlalchemy import Integer, literal_column, text
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	32 from sqlalchemy.dialects.sqlite import insert
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	33 from sqlalchemy.engine import Connection, Engine
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	34 from sqlalchemy.exc import IntegrityError, NoResultFound
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	35 from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, create_async_engine
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	36 from sqlalchemy.future import select
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	37 from sqlalchemy.orm import (
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	38 contains_eager,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	39 joinedload,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	40 selectinload,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	41 sessionmaker,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	42 subqueryload,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	43 )
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	44 from sqlalchemy.orm.attributes import Mapped
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	45 from sqlalchemy.orm.decl_api import DeclarativeMeta
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	46 from sqlalchemy.sql.functions import coalesce, count, now, sum as sum_
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	47 from twisted.internet import defer
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	48 from twisted.words.protocols.jabber import jid
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	49 from twisted.words.xish import domish
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	50
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	51 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	52 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	53 from libervia.backend.core.core_types import SatXMPPEntity
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	54 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	55 from libervia.backend.core.log import getLogger
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	56 from libervia.backend.memory import migration
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	57 from libervia.backend.memory import sqla_config
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	58 from libervia.backend.memory.sqla_mapping import (
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	59 Base,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	60 Component,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	61 File,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	62 History,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	63 Message,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	64 NOT_IN_EXTRA,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	65 Notification,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	66 NotificationPriority,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	67 NotificationStatus,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	68 NotificationType,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	69 ParamGen,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	70 ParamInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	71 PrivateGen,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	72 PrivateGenBin,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	73 PrivateInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	74 PrivateIndBin,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	75 Profile,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	76 PubsubItem,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	77 PubsubNode,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	78 Subject,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	79 SyncState,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	80 Thread,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	81 get_profile_by_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	82 profiles,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	83 )
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	84 from libervia.backend.tools.common import uri
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	85 from libervia.backend.tools.utils import aio, as_future
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	86
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	87
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	88 log = getLogger(__name__)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	89 migration_path = Path(migration.__file__).parent
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	90 #: mapping of Libervia search query operators to SQLAlchemy method name
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	91 OP_MAP = {
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	92 "==": "__eq__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	93 "eq": "__eq__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	94 "!=": "__ne__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	95 "ne": "__ne__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	96 ">": "__gt__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	97 "gt": "__gt__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	98 "<": "__le__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	99 "le": "__le__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	100 "between": "between",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	101 "in": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	102 "not_in": "not_in",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	103 "overlap": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	104 "ioverlap": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	105 "disjoint": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	106 "idisjoint": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	107 "like": "like",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	108 "ilike": "ilike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	109 "not_like": "notlike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	110 "not_ilike": "notilike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	111 }
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	112
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	113
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	114 @event.listens_for(Engine, "connect")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	115 def set_sqlite_pragma(dbapi_connection, connection_record):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	116 cursor = dbapi_connection.cursor()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	117 cursor.execute("PRAGMA foreign_keys=ON")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	118 cursor.close()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	119
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	120
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	121 class Storage:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	122
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	123 def __init__(self):
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	124 self.initialized = defer.Deferred()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	125 # we keep cache for the profiles (key: profile name, value: profile id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	126 # profile id to component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	127 self.components: Dict[int, str] = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	128
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	129 @property
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	130 def profiles(self):
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	131 return profiles
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	132
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	133 def get_profile_by_id(self, profile_id):
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	134 return get_profile_by_id(profile_id)
3665 72b0e4053ab0 core (memory/slqa): implement `getProfileById` Goffi <goffi@goffi.org> parents: 3664 diff changeset	135
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	136 async def migrate_apply(self, *args: str, log_output: bool = False) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	137 """Do a migration command
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	138
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	139 Commands are applied by running Alembic in a subprocess.
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	140 Arguments are alembic executables commands
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	141
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	142 @param log_output: manage stdout and stderr:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	143 - if False, stdout and stderr are buffered, and logged only in case of error
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	144 - if True, stdout and stderr will be logged during the command execution
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	145 @raise exceptions.DatabaseError: something went wrong while running the
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	146 process
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	147 """
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	148 stdout, stderr = 2 * (None,) if log_output else 2 * (PIPE,)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	149 proc = await asyncio.create_subprocess_exec(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	150 sys.executable,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	151 "-m",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	152 "alembic",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	153 *args,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	154 stdout=stdout,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	155 stderr=stderr,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	156 cwd=migration_path,
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	157 )
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	158 log_out, log_err = await proc.communicate()
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	159 if proc.returncode != 0:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	160 msg = _("Can't {operation} database (exit code {exit_code})").format(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	161 operation=args[0], exit_code=proc.returncode
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	162 )
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	163 if log_out or log_err:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	164 msg += f":\nstdout: {log_out.decode()}\nstderr: {log_err.decode()}"
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	165 log.error(msg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	166
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	167 raise exceptions.DatabaseError(msg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	168
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	169 async def create_db(self, engine: AsyncEngine, db_config: dict) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	170 """Create a new database
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	171
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	172 The database is generated from SQLAlchemy model, then stamped by Alembic
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	173 """
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	174 # the dir may not exist if it's not the XDG recommended one
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	175 db_config["path"].parent.mkdir(0o700, True, True)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	176 async with engine.begin() as conn:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	177 await conn.run_sync(Base.metadata.create_all)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	178
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	179 log.debug("stamping the database")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	180 await self.migrate_apply("stamp", "head")
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	181 log.debug("stamping done")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	182
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	183 def _check_db_is_up_to_date(self, conn: Connection) -> bool:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	184 al_ini_path = migration_path / "alembic.ini"
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	185 al_cfg = al_config.Config(al_ini_path)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	186 directory = al_script.ScriptDirectory.from_config(al_cfg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	187 context = al_migration.MigrationContext.configure(conn)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	188 return set(context.get_current_heads()) == set(directory.get_heads())
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	189
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	190 def _sqlite_set_journal_mode_wal(self, conn: Connection) -> None:
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	191 """Check if journal mode is WAL, and set it if necesssary"""
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	192 result = conn.execute(text("PRAGMA journal_mode"))
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	193 if result.scalar() != "wal":
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	194 log.info("WAL mode not activated, activating it")
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	195 conn.execute(text("PRAGMA journal_mode=WAL"))
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	196
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	197 async def check_and_update_db(self, engine: AsyncEngine, db_config: dict) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	198 """Check that database is up-to-date, and update if necessary"""
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	199 async with engine.connect() as conn:
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	200 up_to_date = await conn.run_sync(self._check_db_is_up_to_date)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	201 if up_to_date:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	202 log.debug("Database is up-to-date")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	203 else:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	204 log.info("Database needs to be updated")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	205 log.info("updating…")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	206 await self.migrate_apply("upgrade", "head", log_output=True)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	207 log.info("Database is now up-to-date")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	208
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	209 @aio
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	210 async def initialise(self) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	211 log.info(_("Connecting database"))
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	212
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	213 db_config = sqla_config.get_db_config()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	214 engine = create_async_engine(
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	215 db_config["url"],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	216 future=True,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	217 )
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	218
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	219 new_base = not db_config["path"].exists()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	220 if new_base:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	221 log.info(_("The database is new, creating the tables"))
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	222 await self.create_db(engine, db_config)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	223 else:
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	224 await self.check_and_update_db(engine, db_config)
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	225
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	226 async with engine.connect() as conn:
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	227 await conn.run_sync(self._sqlite_set_journal_mode_wal)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	228
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	229 self.session = sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	230
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	231 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	232 result = await session.execute(select(Profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	233 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	234 self.profiles[p.name] = p.id
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	235 result = await session.execute(select(Component))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	236 for c in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	237 self.components[c.profile_id] = c.entry_point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	238
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	239 self.initialized.callback(None)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	240
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	241 ## Generic
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	242
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	243 @aio
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	244 async def get(
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	245 self,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	246 client: SatXMPPEntity,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	247 db_cls: DeclarativeMeta,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	248 db_id_col: Mapped,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	249 id_value: Any,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	250 joined_loads=None,
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	251 ) -> Optional[DeclarativeMeta]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	252 stmt = select(db_cls).where(db_id_col == id_value)
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	253 if client is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	254 stmt = stmt.filter_by(profile_id=self.profiles[client.profile])
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	255 if joined_loads is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	256 for joined_load in joined_loads:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	257 stmt = stmt.options(joinedload(joined_load))
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	258 async with self.session() as session:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	259 result = await session.execute(stmt)
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	260 if joined_loads is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	261 result = result.unique()
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	262 return result.scalar_one_or_none()
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	263
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	264 @aio
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	265 async def add(self, db_obj: DeclarativeMeta) -> None:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	266 """Add an object to database"""
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	267 async with self.session() as session:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	268 async with session.begin():
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	269 session.add(db_obj)
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	270
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	271 @aio
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	272 async def delete(
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	273 self,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	274 db_obj: Union[DeclarativeMeta, List[DeclarativeMeta]],
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	275 session_add: Optional[List[DeclarativeMeta]] = None,
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	276 ) -> None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	277 """Delete an object from database
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	278
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	279 @param db_obj: object to delete or list of objects to delete
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	280 @param session_add: other objects to add to session.
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	281 This is useful when parents of deleted objects needs to be updated too, or if
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	282 other objects needs to be updated in the same transaction.
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	283 """
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	284 if not db_obj:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	285 return
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	286 if not isinstance(db_obj, list):
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	287 db_obj = [db_obj]
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	288 async with self.session() as session:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	289 async with session.begin():
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	290 if session_add is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	291 for obj in session_add:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	292 session.add(obj)
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	293 for obj in db_obj:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	294 await session.delete(obj)
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	295 await session.commit()
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	296
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	297 ## Profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	298
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	299 def get_profiles_list(self) -> List[str]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	300 """ "Return list of all registered profiles"""
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	301 return list(self.profiles.keys())
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	302
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	303 def has_profile(self, profile_name: str) -> bool:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	304 """return True if profile_name exists
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	305
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	306 @param profile_name: name of the profile to check
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	307 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	308 return profile_name in self.profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	309
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	310 def profile_is_component(self, profile_name: str) -> bool:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	311 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	312 return self.profiles[profile_name] in self.components
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	313 except KeyError:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	314 raise exceptions.NotFound("the requested profile doesn't exists")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	315
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	316 def get_entry_point(self, profile_name: str) -> str:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	317 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	318 return self.components[self.profiles[profile_name]]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	319 except KeyError:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	320 raise exceptions.NotFound(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	321 "the requested profile doesn't exists or is not a component"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	322 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	323
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	324 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	325 async def create_profile(self, name: str, component_ep: Optional[str] = None) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	326 """Create a new profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	327
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	328 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	329 @param component: if not None, must point to a component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	330 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	331 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	332 profile = Profile(name=name)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	333 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	334 session.add(profile)
3638 257135d5c5c2 core (memory/sqla): fix key/value inversion in self.profiles Goffi <goffi@goffi.org> parents: 3621 diff changeset	335 self.profiles[profile.name] = profile.id
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	336 if component_ep is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	337 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	338 component = Component(profile=profile, entry_point=component_ep)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	339 session.add(component)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	340 self.components[profile.id] = component_ep
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	341 return profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	342
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	343 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	344 async def delete_profile(self, name: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	345 """Delete profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	346
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	347 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	348 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	349 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	350 result = await session.execute(select(Profile).where(Profile.name == name))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	351 profile = result.scalar()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	352 await session.delete(profile)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	353 await session.commit()
3638 257135d5c5c2 core (memory/sqla): fix key/value inversion in self.profiles Goffi <goffi@goffi.org> parents: 3621 diff changeset	354 del self.profiles[profile.name]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	355 if profile.id in self.components:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	356 del self.components[profile.id]
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	357 log.info(_("Profile {name!r} deleted").format(name=name))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	358
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	359 ## Params
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	360
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	361 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	362 async def load_gen_params(self, params_gen: dict) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	363 """Load general parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	364
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	365 @param params_gen: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	366 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	367 log.debug(_("loading general parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	368 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	369 result = await session.execute(select(ParamGen))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	370 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	371 params_gen[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	372
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	373 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	374 async def load_ind_params(self, params_ind: dict, profile: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	375 """Load individual parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	376
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	377 @param params_ind: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	378 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	379 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	380 log.debug(_("loading individual parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	381 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	382 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	383 select(ParamInd).where(ParamInd.profile_id == self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	384 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	385 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	386 params_ind[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	387
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	388 @aio
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	389 async def get_ind_param(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	390 self, category: str, name: str, profile: str
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	391 ) -> Optional[str]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	392 """Ask database for the value of one specific individual parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	393
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	394 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	395 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	396 @param profile: %(doc_profile)s
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	397 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	398 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	399 result = await session.execute(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	400 select(ParamInd.value).filter_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	401 category=category, name=name, profile_id=self.profiles[profile]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	402 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	403 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	404 return result.scalar_one_or_none()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	405
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	406 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	407 async def get_ind_param_values(self, category: str, name: str) -> Dict[str, str]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	408 """Ask database for the individual values of a parameter for all profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	409
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	410 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	411 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	412 @return dict: profile => value map
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	413 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	414 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	415 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	416 select(ParamInd)
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	417 .filter_by(category=category, name=name)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	418 .options(subqueryload(ParamInd.profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	419 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	420 return {param.profile.name: param.value for param in result.scalars()}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	421
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	422 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	423 async def set_gen_param(self, category: str, name: str, value: Optional[str]) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	424 """Save the general parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	425
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	426 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	427 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	428 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	429 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	430 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	431 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	432 insert(ParamGen)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	433 .values(category=category, name=name, value=value)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	434 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	435 index_elements=(ParamGen.category, ParamGen.name),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	436 set_={ParamGen.value: value},
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	437 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	438 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	439 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	440 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	441
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	442 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	443 async def set_ind_param(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	444 self, category: str, name: str, value: Optional[str], profile: str
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	445 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	446 """Save the individual parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	447
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	448 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	449 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	450 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	451 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	452 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	453 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	454 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	455 insert(ParamInd)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	456 .values(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	457 category=category,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	458 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	459 profile_id=self.profiles[profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	460 value=value,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	461 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	462 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	463 index_elements=(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	464 ParamInd.category,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	465 ParamInd.name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	466 ParamInd.profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	467 ),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	468 set_={ParamInd.value: value},
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	469 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	470 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	471 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	472 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	473
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	474 def _jid_filter(self, jid_: jid.JID, dest: bool = False):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	475 """Generate condition to filter on a JID, using relevant columns
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	476
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	477 @param dest: True if it's the destinee JID, otherwise it's the source one
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	478 @param jid_: JID to filter by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	479 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	480 if jid_.resource:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	481 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	482 return and_(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	483 History.dest == jid_.userhost(), History.dest_res == jid_.resource
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	484 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	485 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	486 return and_(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	487 History.source == jid_.userhost(), History.source_res == jid_.resource
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	488 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	489 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	490 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	491 return History.dest == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	492 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	493 return History.source == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	494
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	495 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	496 async def history_get(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	497 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	498 from_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	499 to_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	500 limit: Optional[int] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	501 between: bool = True,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	502 filters: Optional[Dict[str, str]] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	503 profile: Optional[str] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	504 ) -> List[Tuple[str, int, str, str, Dict[str, str], Dict[str, str], str, str, str]]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	505 """Retrieve messages in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	506
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	507 @param from_jid: source JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	508 @param to_jid: dest JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	509 @param limit: maximum number of messages to get:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	510 - 0 for no message (returns the empty list)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	511 - None for unlimited
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	512 @param between: confound source and dest (ignore the direction)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	513 @param filters: pattern to filter the history results
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	514 @return: list of messages as in [message_new], minus the profile which is already
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	515 known.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	516 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	517 # we have to set a default value to profile because it's last argument
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	518 # and thus follow other keyword arguments with default values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	519 # but None should not be used for it
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	520 assert profile is not None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	521 if limit == 0:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	522 return []
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	523 if filters is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	524 filters = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	525
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	526 stmt = (
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	527 select(History)
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	528 .filter_by(profile_id=self.profiles[profile])
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	529 .outerjoin(History.messages)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	530 .outerjoin(History.subjects)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	531 .outerjoin(History.thread)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	532 .options(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	533 contains_eager(History.messages),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	534 contains_eager(History.subjects),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	535 contains_eager(History.thread),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	536 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	537 .order_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	538 # timestamp may be identical for 2 close messages (specially when delay is
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	539 # used) that's why we order ties by received_timestamp. We'll reverse the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	540 # order when returning the result. We use DESC here so LIMIT keep the last
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	541 # messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	542 History.timestamp.desc(),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	543 History.received_timestamp.desc(),
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	544 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	545 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	546
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	547 if not from_jid and not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	548 # no jid specified, we want all one2one communications
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	549 pass
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	550 elif between:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	551 if not from_jid or not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	552 # we only have one jid specified, we check all messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	553 # from or to this jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	554 jid_ = from_jid or to_jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	555 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	556 or_(self._jid_filter(jid_), self._jid_filter(jid_, dest=True))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	557 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	558 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	559 # we have 2 jids specified, we check all communications between
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	560 # those 2 jids
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	561 stmt = stmt.where(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	562 or_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	563 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	564 self._jid_filter(from_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	565 self._jid_filter(to_jid, dest=True),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	566 ),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	567 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	568 self._jid_filter(to_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	569 self._jid_filter(from_jid, dest=True),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	570 ),
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	571 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	572 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	573 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	574 # we want one communication in specific direction (from somebody or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	575 # to somebody).
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	576 if from_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	577 stmt = stmt.where(self._jid_filter(from_jid))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	578 if to_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	579 stmt = stmt.where(self._jid_filter(to_jid, dest=True))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	580
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	581 if filters:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	582 if "timestamp_start" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	583 stmt = stmt.where(History.timestamp >= float(filters["timestamp_start"]))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	584 if "before_uid" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	585 # orignially this query was using SQLITE's rowid. This has been changed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	586 # to use coalesce(received_timestamp, timestamp) to be SQL engine independant
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	587 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	588 coalesce(History.received_timestamp, History.timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	589 < (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	590 select(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	591 coalesce(History.received_timestamp, History.timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	592 ).filter_by(uid=filters["before_uid"])
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	593 ).scalar_subquery()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	594 )
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	595 if "body" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	596 # TODO: use REGEXP (function to be defined) instead of GLOB: https://www.sqlite.org/lang_expr.html
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	597 stmt = stmt.where(Message.message.like(f"%{filters['body']}%"))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	598 if "search" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	599 search_term = f"%{filters['search']}%"
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	600 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	601 or_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	602 Message.message.like(search_term),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	603 History.source_res.like(search_term),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	604 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	605 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	606 if "types" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	607 types = filters["types"].split()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	608 stmt = stmt.where(History.type.in_(types))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	609 if "not_types" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	610 types = filters["not_types"].split()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	611 stmt = stmt.where(History.type.not_in(types))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	612 if "last_stanza_id" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	613 # this request get the last message with a "stanza_id" that we
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	614 # have in history. This is mainly used to retrieve messages sent
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	615 # while we were offline, using MAM (XEP-0313).
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	616 if filters["last_stanza_id"] is not True or limit != 1:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	617 raise ValueError("Unexpected values for last_stanza_id filter")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	618 stmt = stmt.where(History.stanza_id.is_not(None))
4160 6a0066ea5c97 core (memory/sqla): add `id` filter in `history_get` Goffi <goffi@goffi.org> parents: 4130 diff changeset	619 if "id" in filters:
6a0066ea5c97 core (memory/sqla): add `id` filter in `history_get` Goffi <goffi@goffi.org> parents: 4130 diff changeset	620 stmt = stmt.where(History.uid == filters["id"])
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	621 if "origin_id" in filters:
3797 cc653b2685f0 core (memory/sqla), plugin XEP-0359: always add `origin-id`, and store: Goffi <goffi@goffi.org> parents: 3754 diff changeset	622 stmt = stmt.where(History.origin_id == filters["origin_id"])
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	623
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	624 if limit is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	625 stmt = stmt.limit(limit)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	626
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	627 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	628 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	629
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	630 result = result.scalars().unique().all()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	631 result.reverse()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	632 return [h.as_tuple() for h in result]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	633
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	634 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	635 async def add_to_history(self, data: dict, profile: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	636 """Store a new message in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	637
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	638 @param data: message data as build by SatMessageProtocol.onMessage
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	639 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	640 extra = {k: v for k, v in data["extra"].items() if k not in NOT_IN_EXTRA}
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	641 messages = [
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	642 Message(message=mess, language=lang) for lang, mess in data["message"].items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	643 ]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	644 subjects = [
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	645 Subject(subject=mess, language=lang) for lang, mess in data["subject"].items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	646 ]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	647 if "thread" in data["extra"]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	648 thread = Thread(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	649 thread_id=data["extra"]["thread"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	650 parent_id=data["extra"].get["thread_parent"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	651 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	652 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	653 thread = None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	654 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	655 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	656 async with session.begin():
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	657 session.add(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	658 History(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	659 uid=data["uid"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	660 origin_id=data["extra"].get("origin_id"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	661 stanza_id=data["extra"].get("stanza_id"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	662 update_uid=data["extra"].get("update_uid"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	663 profile_id=self.profiles[profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	664 source_jid=data["from"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	665 dest_jid=data["to"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	666 timestamp=data["timestamp"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	667 received_timestamp=data.get("received_timestamp"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	668 type=data["type"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	669 extra=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	670 messages=messages,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	671 subjects=subjects,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	672 thread=thread,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	673 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	674 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	675 except IntegrityError as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	676 if "unique" in str(e.orig).lower():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	677 log.debug(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	678 f"message {data['uid']!r} is already in history, not storing it again"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	679 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	680 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	681 log.error(f"Can't store message {data['uid']!r} in history: {e}")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	682 except Exception as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	683 log.critical(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	684 f"Can't store message, unexpected exception (uid: {data['uid']}): {e}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	685 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	686
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	687 ## Private values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	688
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	689 def _get_private_class(self, binary, profile):
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	690 """Get ORM class to use for private values"""
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	691 if profile is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	692 return PrivateGenBin if binary else PrivateGen
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	693 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	694 return PrivateIndBin if binary else PrivateInd
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	695
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	696 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	697 async def get_privates(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	698 self,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	699 namespace: str,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	700 keys: Optional[Iterable[str]] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	701 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	702 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	703 ) -> Dict[str, Any]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	704 """Get private value(s) from databases
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	705
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	706 @param namespace: namespace of the values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	707 @param keys: keys of the values to get None to get all keys/values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	708 @param binary: True to deserialise binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	709 @param profile: profile to use for individual values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	710 None to use general values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	711 @return: gotten keys/values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	712 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	713 if keys is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	714 keys = list(keys)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	715 log.debug(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	716 f"getting {'general' if profile is None else 'individual'}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	717 f"{' binary' if binary else ''} private values from database for namespace "
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	718 f"{namespace}{f' with keys {keys!r}' if keys is not None else ''}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	719 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	720 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	721 stmt = select(cls).filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	722 if keys:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	723 stmt = stmt.where(cls.key.in_(list(keys)))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	724 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	725 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	726 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	727 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	728 return {p.key: p.value for p in result.scalars()}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	729
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	730 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	731 async def set_private_value(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	732 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	733 namespace: str,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	734 key: str,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	735 value: Any,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	736 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	737 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	738 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	739 """Set a private value in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	740
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	741 @param namespace: namespace of the values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	742 @param key: key of the value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	743 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	744 @param binary: True if it's a binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	745 binary values need to be serialised, used for everything but strings
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	746 @param profile: profile to use for individual value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	747 if None, it's a general value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	748 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	749 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	750
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	751 values = {"namespace": namespace, "key": key, "value": value}
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	752 index_elements = [cls.namespace, cls.key]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	753
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	754 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	755 values["profile_id"] = self.profiles[profile]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	756 index_elements.append(cls.profile_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	757
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	758 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	759 await session.execute(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	760 insert(cls)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	761 .values(**values)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	762 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	763 index_elements=index_elements, set_={cls.value: value}
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	764 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	765 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	766 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	767
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	768 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	769 async def del_private_value(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	770 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	771 namespace: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	772 key: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	773 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	774 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	775 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	776 """Delete private value from database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	777
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	778 @param category: category of the privateeter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	779 @param key: key of the private value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	780 @param binary: True if it's a binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	781 @param profile: profile to use for individual value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	782 if None, it's a general value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	783 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	784 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	785
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	786 stmt = delete(cls).filter_by(namespace=namespace, key=key)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	787
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	788 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	789 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	790
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	791 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	792 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	793 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	794
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	795 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	796 async def del_private_namespace(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	797 self, namespace: str, binary: bool = False, profile: Optional[str] = None
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	798 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	799 """Delete all data from a private namespace
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	800
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	801 Be really cautious when you use this method, as all data with given namespace are
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	802 removed.
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	803 Params are the same as for del_private_value
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	804 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	805 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	806
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	807 stmt = delete(cls).filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	808
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	809 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	810 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	811
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	812 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	813 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	814 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	815
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	816 ## Files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	817
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	818 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	819 async def get_files(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	820 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	821 client: Optional[SatXMPPEntity],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	822 file_id: Optional[str] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	823 version: Optional[str] = "",
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	824 parent: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	825 type_: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	826 file_hash: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	827 hash_algo: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	828 name: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	829 namespace: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	830 mime_type: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	831 public_id: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	832 owner: Optional[jid.JID] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	833 access: Optional[dict] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	834 projection: Optional[List[str]] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	835 unique: bool = False,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	836 ) -> List[dict]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	837 """Retrieve files with with given filters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	838
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	839 @param file_id: id of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	840 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	841 @param version: version of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	842 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	843 empty string to look for current version
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	844 @param parent: id of the directory containing the files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	845 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	846 empty string to look for root files/directories
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	847 @param projection: name of columns to retrieve
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	848 None to retrieve all
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	849 @param unique: if True will remove duplicates
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	850 other params are the same as for [set_file]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	851 @return: files corresponding to filters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	852 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	853 if projection is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	854 projection = [
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	855 "id",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	856 "version",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	857 "parent",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	858 "type",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	859 "file_hash",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	860 "hash_algo",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	861 "name",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	862 "size",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	863 "namespace",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	864 "media_type",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	865 "media_subtype",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	866 "public_id",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	867 "created",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	868 "modified",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	869 "owner",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	870 "access",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	871 "extra",
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	872 ]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	873
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	874 stmt = select(*[getattr(File, f) for f in projection])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	875
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	876 if unique:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	877 stmt = stmt.distinct()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	878
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	879 if client is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	880 stmt = stmt.filter_by(profile_id=self.profiles[client.profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	881 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	882 if public_id is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	883 raise exceptions.InternalError(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	884 "client can only be omitted when public_id is set"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	885 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	886 if file_id is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	887 stmt = stmt.filter_by(id=file_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	888 if version is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	889 stmt = stmt.filter_by(version=version)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	890 if parent is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	891 stmt = stmt.filter_by(parent=parent)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	892 if type_ is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	893 stmt = stmt.filter_by(type=type_)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	894 if file_hash is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	895 stmt = stmt.filter_by(file_hash=file_hash)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	896 if hash_algo is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	897 stmt = stmt.filter_by(hash_algo=hash_algo)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	898 if name is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	899 stmt = stmt.filter_by(name=name)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	900 if namespace is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	901 stmt = stmt.filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	902 if mime_type is not None:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	903 if "/" in mime_type:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	904 media_type, media_subtype = mime_type.split("/", 1)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	905 stmt = stmt.filter_by(media_type=media_type, media_subtype=media_subtype)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	906 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	907 stmt = stmt.filter_by(media_type=mime_type)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	908 if public_id is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	909 stmt = stmt.filter_by(public_id=public_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	910 if owner is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	911 stmt = stmt.filter_by(owner=owner)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	912 if access is not None:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	913 raise NotImplementedError("Access check is not implemented yet")
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	914 # a JSON comparison is needed here
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	915
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	916 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	917 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	918
4092 74c66c0d93f3 core (memory/sqla): fix row to dict conversion in `get_files` Goffi <goffi@goffi.org> parents: 4071 diff changeset	919 return [r._asdict() for r in result]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	920
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	921 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	922 async def set_file(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	923 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	924 client: SatXMPPEntity,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	925 name: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	926 file_id: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	927 version: str = "",
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	928 parent: str = "",
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	929 type_: str = C.FILE_TYPE_FILE,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	930 file_hash: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	931 hash_algo: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	932 size: int = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	933 namespace: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	934 mime_type: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	935 public_id: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	936 created: Optional[float] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	937 modified: Optional[float] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	938 owner: Optional[jid.JID] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	939 access: Optional[dict] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	940 extra: Optional[dict] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	941 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	942 """Set a file metadata
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	943
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	944 @param client: client owning the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	945 @param name: name of the file (must not contain "/")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	946 @param file_id: unique id of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	947 @param version: version of this file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	948 @param parent: id of the directory containing this file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	949 Empty string if it is a root file/directory
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	950 @param type_: one of:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	951 - file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	952 - directory
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	953 @param file_hash: unique hash of the payload
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	954 @param hash_algo: algorithm used for hashing the file (usually sha-256)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	955 @param size: size in bytes
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	956 @param namespace: identifier (human readable is better) to group files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	957 for instance, namespace could be used to group files in a specific photo album
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	958 @param mime_type: media type of the file, or None if not known/guessed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	959 @param public_id: ID used to server the file publicly via HTTP
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	960 @param created: UNIX time of creation
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	961 @param modified: UNIX time of last modification, or None to use created date
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	962 @param owner: jid of the owner of the file (mainly useful for component)
3621 9e1a993ad1bf core (memory/sqla): minor docstring line lenght fix Goffi <goffi@goffi.org> parents: 3596 diff changeset	963 @param access: serialisable dictionary with access rules. See [memory.memory] for
9e1a993ad1bf core (memory/sqla): minor docstring line lenght fix Goffi <goffi@goffi.org> parents: 3596 diff changeset	964 details
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	965 @param extra: serialisable dictionary of any extra data
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	966 will be encoded to json in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	967 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	968 if mime_type is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	969 media_type = media_subtype = None
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	970 elif "/" in mime_type:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	971 media_type, media_subtype = mime_type.split("/", 1)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	972 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	973 media_type, media_subtype = mime_type, None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	974
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	975 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	976 async with session.begin():
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	977 session.add(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	978 File(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	979 id=file_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	980 version=version.strip(),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	981 parent=parent,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	982 type=type_,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	983 file_hash=file_hash,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	984 hash_algo=hash_algo,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	985 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	986 size=size,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	987 namespace=namespace,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	988 media_type=media_type,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	989 media_subtype=media_subtype,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	990 public_id=public_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	991 created=time.time() if created is None else created,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	992 modified=modified,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	993 owner=owner,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	994 access=access,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	995 extra=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	996 profile_id=self.profiles[client.profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	997 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	998 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	999
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1000 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1001 async def file_get_used_space(self, client: SatXMPPEntity, owner: jid.JID) -> int:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1002 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1003 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1004 select(sum_(File.size)).filter_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1005 owner=owner,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1006 type=C.FILE_TYPE_FILE,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1007 profile_id=self.profiles[client.profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1008 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1009 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1010 return result.scalar_one_or_none() or 0
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1011
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1012 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1013 async def file_delete(self, file_id: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1014 """Delete file metadata from the database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1015
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1016 @param file_id: id of the file to delete
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1017 NOTE: file itself must still be removed, this method only handle metadata in
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1018 database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1019 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1020 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1021 await session.execute(delete(File).filter_by(id=file_id))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1022 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1023
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1024 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1025 async def file_update(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1026 self, file_id: str, column: str, update_cb: Callable[[dict], None]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1027 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1028 """Update a column value using a method to avoid race conditions
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1029
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1030 the older value will be retrieved from database, then update_cb will be applied to
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1031 update it, and file will be updated checking that older value has not been changed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1032 meanwhile by an other user. If it has changed, it tries again a couple of times
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1033 before failing
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1034 @param column: column name (only "access" or "extra" are allowed)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1035 @param update_cb: method to update the value of the colum
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1036 the method will take older value as argument, and must update it in place
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1037 update_cb must not care about serialization,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1038 it get the deserialized data (i.e. a Python object) directly
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1039 @raise exceptions.NotFound: there is not file with this id
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1040 """
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1041 if column not in ("access", "extra"):
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1042 raise exceptions.InternalError("bad column name")
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1043 orm_col = getattr(File, column)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1044
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1045 for i in range(5):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1046 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1047 try:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1048 value = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1049 await session.execute(select(orm_col).filter_by(id=file_id))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1050 ).scalar_one()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1051 except NoResultFound:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1052 raise exceptions.NotFound
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1053 old_value = copy.deepcopy(value)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1054 update_cb(value)
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1055 stmt = update(File).filter_by(id=file_id).values({column: value})
bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1056 if not old_value:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1057 # because JsonDefaultDict convert NULL to an empty dict, we have to
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1058 # test both for empty dict and None when we have an empty dict
bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1059 stmt = stmt.where((orm_col == None) \| (orm_col == old_value))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1060 else:
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1061 stmt = stmt.where(orm_col == old_value)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1062 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1063 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1064
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1065 if result.rowcount == 1:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1066 break
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1067
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1068 log.warning(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1069 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1070 "table not updated, probably due to race condition, trying again "
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1071 "({tries})"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1072 ).format(tries=i + 1)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1073 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1074
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1075 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1076 raise exceptions.DatabaseError(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1077 _("Can't update file {file_id} due to race condition").format(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1078 file_id=file_id
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1079 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1080 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1081
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1082 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1083 async def get_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1084 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1085 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1086 service: jid.JID,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1087 name: str,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1088 with_items: bool = False,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1089 with_subscriptions: bool = False,
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1090 create: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1091 create_kwargs: Optional[dict] = None,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1092 ) -> Optional[PubsubNode]:
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1093 """Retrieve a PubsubNode from DB
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1094
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1095 @param service: service hosting the node
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1096 @param name: node's name
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1097 @param with_items: retrieve items in the same query
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1098 @param with_subscriptions: retrieve subscriptions in the same query
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1099 @param create: if the node doesn't exist in DB, create it
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1100 @param create_kwargs: keyword arguments to use with ``set_pubsub_node`` if the node
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1101 needs to be created.
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1102 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1103 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1104 stmt = select(PubsubNode).filter_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1105 service=service,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1106 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1107 profile_id=self.profiles[client.profile],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1108 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1109 if with_items:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1110 stmt = stmt.options(joinedload(PubsubNode.items))
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1111 if with_subscriptions:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1112 stmt = stmt.options(joinedload(PubsubNode.subscriptions))
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1113 result = await session.execute(stmt)
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1114 ret = result.unique().scalar_one_or_none()
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1115 if ret is None and create:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1116 # we auto-create the node
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1117 if create_kwargs is None:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1118 create_kwargs = {}
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1119 try:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1120 return await as_future(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1121 self.set_pubsub_node(client, service, name, **create_kwargs)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1122 )
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1123 except IntegrityError as e:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1124 if "unique" in str(e.orig).lower():
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1125 # the node may already exist, if it has been created just after
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1126 # get_pubsub_node above
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1127 log.debug("ignoring UNIQUE constraint error")
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1128 cached_node = await as_future(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1129 self.get_pubsub_node(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1130 client,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1131 service,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1132 name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1133 with_items=with_items,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1134 with_subscriptions=with_subscriptions,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1135 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1136 )
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1137 else:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1138 raise e
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1139 else:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1140 return ret
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1141
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1142 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1143 async def set_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1144 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1145 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1146 service: jid.JID,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1147 name: str,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1148 analyser: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1149 type_: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1150 subtype: Optional[str] = None,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1151 subscribed: bool = False,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1152 ) -> PubsubNode:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1153 node = PubsubNode(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1154 profile_id=self.profiles[client.profile],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1155 service=service,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1156 name=name,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1157 subscribed=subscribed,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1158 analyser=analyser,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1159 type_=type_,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1160 subtype=subtype,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1161 subscriptions=[],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1162 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1163 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1164 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1165 session.add(node)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1166 return node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1167
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1168 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1169 async def update_pubsub_node_sync_state(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1170 self, node: PubsubNode, state: SyncState
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1171 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1172 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1173 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1174 await session.execute(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1175 update(PubsubNode)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1176 .filter_by(id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1177 .values(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1178 sync_state=state,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1179 sync_state_updated=time.time(),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1180 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1181 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1182
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1183 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1184 async def delete_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1185 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1186 profiles: Optional[List[str]],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1187 services: Optional[List[jid.JID]],
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1188 names: Optional[List[str]],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1189 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1190 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1191
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1192 @param profiles: profile names from which nodes must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1193 None to remove nodes from ALL profiles
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1194 @param services: JIDs of pubsub services from which nodes must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1195 None to remove nodes from ALL services
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1196 @param names: names of nodes which must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1197 None to remove ALL nodes whatever is their names
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1198 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1199 stmt = delete(PubsubNode)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1200 if profiles is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1201 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1202 PubsubNode.profile.in_([self.profiles[p] for p in profiles])
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1203 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1204 if services is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1205 stmt = stmt.where(PubsubNode.service.in_(services))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1206 if names is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1207 stmt = stmt.where(PubsubNode.name.in_(names))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1208 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1209 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1210 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1211
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1212 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1213 async def cache_pubsub_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1214 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1215 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1216 node: PubsubNode,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1217 items: List[domish.Element],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1218 parsed_items: Optional[List[dict]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1219 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1220 """Add items to database, using an upsert taking care of "updated" field"""
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1221 if parsed_items is not None and len(items) != len(parsed_items):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1222 raise exceptions.InternalError(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1223 "parsed_items must have the same lenght as items"
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1224 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1225 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1226 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1227 for idx, item in enumerate(items):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1228 parsed = parsed_items[idx] if parsed_items else None
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1229 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1230 insert(PubsubItem)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1231 .values(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1232 node_id=node.id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1233 name=item["id"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1234 data=item,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1235 parsed=parsed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1236 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1237 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1238 index_elements=(PubsubItem.node_id, PubsubItem.name),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1239 set_={
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1240 PubsubItem.data: item,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1241 PubsubItem.parsed: parsed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1242 PubsubItem.updated: now(),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1243 },
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1244 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1245 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1246 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1247 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1248
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1249 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1250 async def delete_pubsub_items(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1251 self, node: PubsubNode, items_names: Optional[List[str]] = None
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1252 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1253 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1254
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1255 @param node: node from which items must be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1256 @param items_names: names of items to delete
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1257 if None, ALL items will be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1258 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1259 stmt = delete(PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1260 if node is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1261 if isinstance(node, list):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1262 stmt = stmt.where(PubsubItem.node_id.in_([n.id for n in node]))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1263 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1264 stmt = stmt.filter_by(node_id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1265 if items_names is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1266 stmt = stmt.where(PubsubItem.name.in_(items_names))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1267 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1268 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1269 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1270
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1271 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1272 async def purge_pubsub_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1273 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1274 services: Optional[List[jid.JID]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1275 names: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1276 types: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1277 subtypes: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1278 profiles: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1279 created_before: Optional[datetime] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1280 updated_before: Optional[datetime] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1281 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1282 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1283
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1284 @param node: node from which items must be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1285 @param items_names: names of items to delete
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1286 if None, ALL items will be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1287 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1288 stmt = delete(PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1289 node_fields = {
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1290 "service": services,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1291 "name": names,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1292 "type_": types,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1293 "subtype": subtypes,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1294 }
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1295 if profiles is not None:
658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1296 node_fields["profile_id"] = [self.profiles[p] for p in profiles]
658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1297
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1298 if any(x is not None for x in node_fields.values()):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1299 sub_q = select(PubsubNode.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1300 for col, values in node_fields.items():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1301 if values is None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1302 continue
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1303 sub_q = sub_q.where(getattr(PubsubNode, col).in_(values))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1304 stmt = stmt.where(PubsubItem.node_id.in_(sub_q)).execution_options(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1305 synchronize_session=False
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1306 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1307
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1308 if created_before is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1309 stmt = stmt.where(PubsubItem.created < created_before)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1310
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1311 if updated_before is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1312 stmt = stmt.where(PubsubItem.updated < updated_before)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1313
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1314 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1315 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1316 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1317
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1318 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1319 async def get_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1320 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1321 node: PubsubNode,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1322 max_items: Optional[int] = None,
3753 10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1323 item_ids: Optional[list[str]] = None,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1324 before: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1325 after: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1326 from_index: Optional[int] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1327 order_by: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1328 desc: bool = True,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1329 force_rsm: bool = False,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1330 ) -> Tuple[List[PubsubItem], dict]:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1331 """Get Pubsub Items from cache
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1332
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1333 @param node: retrieve items from this node (must be synchronised)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1334 @param max_items: maximum number of items to retrieve
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1335 @param before: get items which are before the item with this name in given order
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1336 empty string is not managed here, use desc order to reproduce RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1337 behaviour.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1338 @param after: get items which are after the item with this name in given order
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1339 @param from_index: get items with item index (as defined in RSM spec)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1340 starting from this number
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1341 @param order_by: sorting order of items (one of C.ORDER_BY_*)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1342 @param desc: direction or ordering
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1343 @param force_rsm: if True, force the use of RSM worklow.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1344 RSM workflow is automatically used if any of before, after or
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1345 from_index is used, but if only RSM max_items is used, it won't be
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1346 used by default. This parameter let's use RSM workflow in this
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1347 case. Note that in addition to RSM metadata, the result will not be
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1348 the same (max_items without RSM will returns most recent items,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1349 i.e. last items in modification order, while max_items with RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1350 will return the oldest ones (i.e. first items in modification
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1351 order).
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1352 to be used when max_items is used from RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1353 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1354
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1355 metadata = {
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1356 "service": node.service,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1357 "node": node.name,
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1358 "uri": uri.build_xmpp_uri(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1359 "pubsub",
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1360 path=node.service.full(),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1361 node=node.name,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1362 ),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1363 }
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1364 if max_items is None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1365 max_items = 20
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1366
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1367 use_rsm = any((before, after, from_index is not None))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1368 if force_rsm and not use_rsm:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1369 #
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1370 use_rsm = True
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1371 from_index = 0
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1372
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1373 stmt = select(PubsubItem).filter_by(node_id=node.id).limit(max_items)
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1374
3753 10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1375 if item_ids is not None:
10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1376 stmt = stmt.where(PubsubItem.name.in_(item_ids))
10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1377
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1378 if not order_by:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1379 order_by = [C.ORDER_BY_MODIFICATION]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1380
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1381 order = []
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1382 for order_type in order_by:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1383 if order_type == C.ORDER_BY_MODIFICATION:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1384 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1385 order.extend((PubsubItem.updated.desc(), PubsubItem.id.desc()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1386 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1387 order.extend((PubsubItem.updated.asc(), PubsubItem.id.asc()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1388 elif order_type == C.ORDER_BY_CREATION:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1389 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1390 order.append(PubsubItem.id.desc())
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1391 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1392 order.append(PubsubItem.id.asc())
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1393 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1394 raise exceptions.InternalError(f"Unknown order type {order_type!r}")
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1395
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1396 stmt = stmt.order_by(*order)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1397
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1398 if use_rsm:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1399 # CTE to have result row numbers
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1400 row_num_q = select(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1401 PubsubItem.id,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1402 PubsubItem.name,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1403 # row_number starts from 1, but RSM index must start from 0
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1404 (func.row_number().over(order_by=order) - 1).label("item_index"),
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1405 ).filter_by(node_id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1406
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1407 row_num_cte = row_num_q.cte()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1408
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1409 if max_items > 0:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1410 # as we can't simply use PubsubItem.id when we order by modification,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1411 # we need to use row number
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1412 item_name = before or after
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1413 row_num_limit_q = (
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1414 select(row_num_cte.c.item_index).where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1415 row_num_cte.c.name == item_name
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1416 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1417 ).scalar_subquery()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1418
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1419 stmt = (
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1420 select(row_num_cte.c.item_index, PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1421 .join(row_num_cte, PubsubItem.id == row_num_cte.c.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1422 .limit(max_items)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1423 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1424 if before:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1425 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1426 row_num_cte.c.item_index < row_num_limit_q
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1427 ).order_by(row_num_cte.c.item_index.desc())
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1428 elif after:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1429 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1430 row_num_cte.c.item_index > row_num_limit_q
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1431 ).order_by(row_num_cte.c.item_index.asc())
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1432 else:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1433 stmt = stmt.where(row_num_cte.c.item_index >= from_index).order_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1434 row_num_cte.c.item_index.asc()
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1435 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1436 # from_index is used
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1437
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1438 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1439 if max_items == 0:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1440 items = result = []
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1441 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1442 result = await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1443 result = result.all()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1444 if before:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1445 result.reverse()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1446 items = [row[-1] for row in result]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1447 rows_count = (
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1448 await session.execute(row_num_q.with_only_columns(count()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1449 ).scalar_one()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1450
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1451 try:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1452 index = result[0][0]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1453 except IndexError:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1454 index = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1455
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1456 try:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1457 first = result[0][1].name
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1458 except IndexError:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1459 first = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1460 last = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1461 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1462 last = result[-1][1].name
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1463
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1464 metadata["rsm"] = {
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1465 k: v
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1466 for k, v in {
3719 cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1467 "index": index,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1468 "count": rows_count,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1469 "first": first,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1470 "last": last,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1471 }.items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1472 if v is not None
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1473 }
3720 40a6374fcd44 memory (sqla): fix `complete` setting when `index` is `None` in `getItems` Goffi <goffi@goffi.org> parents: 3719 diff changeset	1474 metadata["complete"] = (index or 0) + len(result) == rows_count
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1475
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1476 return items, metadata
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1477
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1478 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1479 result = await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1480
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1481 result = result.scalars().all()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1482 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1483 result.reverse()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1484 return result, metadata
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1485
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1486 def _get_sqlite_path(self, path: List[Union[str, int]]) -> str:
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1487 """generate path suitable to query JSON element with SQLite"""
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1488 return f"${''.join(f'[{p}]' if isinstance(p, int) else f'.{p}' for p in path)}"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1489
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1490 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1491 async def search_pubsub_items(
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1492 self,
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1493 query: dict,
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1494 ) -> Tuple[List[PubsubItem]]:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1495 """Search for pubsub items in cache
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1496
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1497 @param query: search terms. Keys can be:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1498 :fts (str):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1499 Full-Text Search query. Currently SQLite FT5 engine is used, its query
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1500 syntax can be used, see `FTS5 Query documentation
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1501 <https://sqlite.org/fts5.html#full_text_query_syntax>`_
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1502 :profiles (list[str]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1503 filter on nodes linked to those profiles
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1504 :nodes (list[str]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1505 filter on nodes with those names
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1506 :services (list[jid.JID]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1507 filter on nodes from those services
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1508 :types (list[str\|None]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1509 filter on nodes with those types. None can be used to filter on nodes with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1510 no type set
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1511 :subtypes (list[str\|None]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1512 filter on nodes with those subtypes. None can be used to filter on nodes with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1513 no subtype set
3754 af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1514 :names (list[str]):
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1515 filter on items with those names
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1516 :parsed (list[dict]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1517 Filter on a parsed data field. The dict must contain 3 keys: ``path``
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1518 which is a list of str or int giving the path to the field of interest
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1519 (str for a dict key, int for a list index), ``operator`` with indicate the
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1520 operator to use to check the condition, and ``value`` which depends of
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1521 field type and operator.
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1522
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1523 See documentation for details on operators (it's currently explained at
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1524 ``doc/libervia-cli/pubsub_cache.rst`` in ``search`` command
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1525 documentation).
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1526
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1527 :order-by (list[dict]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1528 Indicates how to order results. The dict can contain either a ``order``
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1529 for a well-know order or a ``path`` for a parsed data field path
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1530 (``order`` and ``path`` can't be used at the same time), an an optional
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1531 ``direction`` which can be ``asc`` or ``desc``. See documentation for
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1532 details on well-known orders (it's currently explained at
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1533 ``doc/libervia-cli/pubsub_cache.rst`` in ``search`` command
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1534 documentation).
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1535
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1536 :index (int):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1537 starting index of items to return from the query result. It's translated
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1538 to SQL's OFFSET
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1539
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1540 :limit (int):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1541 maximum number of items to return. It's translated to SQL's LIMIT.
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1542
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1543 @result: found items (the ``node`` attribute will be filled with suitable
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1544 PubsubNode)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1545 """
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1546 # TODO: FTS and parsed data filters use SQLite specific syntax
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1547 # when other DB engines will be used, this will have to be adapted
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1548 stmt = select(PubsubItem)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1549
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1550 # Full-Text Search
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1551 fts = query.get("fts")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1552 if fts:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1553 fts_select = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1554 text("SELECT rowid, rank FROM pubsub_items_fts(:fts_query)")
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1555 .bindparams(fts_query=fts)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1556 .columns(rowid=Integer)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1557 .subquery()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1558 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1559 stmt = stmt.select_from(fts_select).outerjoin(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1560 PubsubItem, fts_select.c.rowid == PubsubItem.id
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1561 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1562
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1563 # node related filters
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1564 profiles = query.get("profiles")
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1565 if profiles or any(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1566 query.get(k) for k in ("nodes", "services", "types", "subtypes")
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1567 ):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1568 stmt = stmt.join(PubsubNode).options(contains_eager(PubsubItem.node))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1569 if profiles:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1570 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1571 stmt = stmt.where(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1572 PubsubNode.profile_id.in_(self.profiles[p] for p in profiles)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1573 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1574 except KeyError as e:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1575 raise exceptions.ProfileUnknownError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1576 f"This profile doesn't exist: {e.args[0]!r}"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1577 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1578 for key, attr in (
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1579 ("nodes", "name"),
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1580 ("services", "service"),
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1581 ("types", "type_"),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1582 ("subtypes", "subtype"),
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1583 ):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1584 value = query.get(key)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1585 if not value:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1586 continue
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1587 if key in ("types", "subtypes") and None in value:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1588 # NULL can't be used with SQL's IN, so we have to add a condition with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1589 # IS NULL, and use a OR if there are other values to check
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1590 value.remove(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1591 condition = getattr(PubsubNode, attr).is_(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1592 if value:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1593 condition = or_(getattr(PubsubNode, attr).in_(value), condition)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1594 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1595 condition = getattr(PubsubNode, attr).in_(value)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1596 stmt = stmt.where(condition)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1597 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1598 stmt = stmt.options(selectinload(PubsubItem.node))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1599
3754 af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1600 # names
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1601 names = query.get("names")
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1602 if names:
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1603 stmt = stmt.where(PubsubItem.name.in_(names))
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1604
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1605 # parsed data filters
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1606 parsed = query.get("parsed", [])
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1607 for filter_ in parsed:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1608 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1609 path = filter_["path"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1610 operator = filter_["op"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1611 value = filter_["value"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1612 except KeyError as e:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1613 raise ValueError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1614 f'missing mandatory key {e.args[0]!r} in "parsed" filter'
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1615 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1616 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1617 op_attr = OP_MAP[operator]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1618 except KeyError:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1619 raise ValueError(f"invalid operator: {operator!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1620 sqlite_path = self._get_sqlite_path(path)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1621 if operator in ("overlap", "ioverlap", "disjoint", "idisjoint"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1622 col = literal_column("json_each.value")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1623 if operator[0] == "i":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1624 col = func.lower(col)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1625 value = [str(v).lower() for v in value]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1626 condition = (
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1627 select(1)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1628 .select_from(func.json_each(PubsubItem.parsed, sqlite_path))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1629 .where(col.in_(value))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1630 ).scalar_subquery()
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1631 if operator in ("disjoint", "idisjoint"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1632 condition = condition.is_(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1633 stmt = stmt.where(condition)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1634 elif operator == "between":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1635 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1636 left, right = value
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1637 except (ValueError, TypeError):
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1638 raise ValueError(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1639 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1640 'invalid value for "between" filter, you must use a 2 items '
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1641 "array: {value!r}"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1642 ).format(value=value)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1643 )
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1644 col = func.json_extract(PubsubItem.parsed, sqlite_path)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1645 stmt = stmt.where(col.between(left, right))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1646 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1647 # we use func.json_extract instead of generic JSON way because SQLAlchemy
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1648 # add a JSON_QUOTE to the value, and we want SQL value
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1649 col = func.json_extract(PubsubItem.parsed, sqlite_path)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1650 stmt = stmt.where(getattr(col, op_attr)(value))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1651
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1652 # order
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1653 order_by = query.get("order-by") or [{"order": "creation"}]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1654
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1655 for order_data in order_by:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1656 order, path = order_data.get("order"), order_data.get("path")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1657 if order and path:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1658 raise ValueError(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1659 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1660 '"order" and "path" can\'t be used at the same time in '
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1661 '"order-by" data'
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1662 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1663 )
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1664 if order:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1665 if order == "creation":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1666 col = PubsubItem.id
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1667 elif order == "modification":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1668 col = PubsubItem.updated
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1669 elif order == "item_id":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1670 col = PubsubItem.name
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1671 elif order == "rank":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1672 if not fts:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1673 raise ValueError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1674 "'rank' order can only be used with Full-Text Search (fts)"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1675 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1676 col = literal_column("rank")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1677 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1678 raise NotImplementedError(f"Unknown {order!r} order")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1679 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1680 # we have a JSON path
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1681 # sqlite_path = self._get_sqlite_path(path)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1682 col = PubsubItem.parsed[path]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1683 direction = order_data.get("direction", "ASC").lower()
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1684 if not direction in ("asc", "desc"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1685 raise ValueError(f"Invalid order-by direction: {direction!r}")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1686 stmt = stmt.order_by(getattr(col, direction)())
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1687
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1688 # offset, limit
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1689 index = query.get("index")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1690 if index:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1691 stmt = stmt.offset(index)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1692 limit = query.get("limit")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1693 if limit:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1694 stmt = stmt.limit(limit)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1695
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1696 async with self.session() as session:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1697 result = await session.execute(stmt)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1698
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1699 return result.scalars().all()
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1700
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1701 # Notifications
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1702
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1703 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1704 async def add_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1705 self,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1706 client: Optional[SatXMPPEntity],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1707 type_: NotificationType,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1708 body_plain: str,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1709 body_rich: Optional[str] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1710 title: Optional[str] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1711 requires_action: bool = False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1712 priority: NotificationPriority = NotificationPriority.MEDIUM,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1713 expire_at: Optional[float] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1714 extra: Optional[dict] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1715 ) -> Notification:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1716 """Add a new notification to the DB.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1717
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1718 @param client: client associated with the notification. If None, the notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1719 will be global.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1720 @param type_: type of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1721 @param body_plain: plain text body.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1722 @param body_rich: rich text (XHTML) body.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1723 @param title: optional title.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1724 @param requires_action: True if the notification requires user action (e.g. a
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1725 dialog need to be answered).
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1726 @priority: how urgent the notification is
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1727 @param expire_at: expiration timestamp for the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1728 @param extra: additional data.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1729 @return: created Notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1730 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1731 profile_id = self.profiles[client.profile] if client else None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1732 notification = Notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1733 profile_id=profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1734 type=type_,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1735 body_plain=body_plain,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1736 body_rich=body_rich,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1737 requires_action=requires_action,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1738 priority=priority,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1739 expire_at=expire_at,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1740 title=title,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1741 extra_data=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1742 status=NotificationStatus.new,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1743 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1744 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1745 async with session.begin():
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1746 session.add(notification)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1747 return notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1748
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1749 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1750 async def update_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1751 self, client: SatXMPPEntity, notification_id: int, **kwargs
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1752 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1753 """Update an existing notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1754
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1755 @param client: client associated with the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1756 @param notification_id: ID of the notification to update.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1757 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1758 profile_id = self.profiles[client.profile]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1759 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1760 await session.execute(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1761 update(Notification)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1762 .where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1763 and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1764 Notification.profile_id == profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1765 Notification.id == notification_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1766 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1767 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1768 .values(**kwargs)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1769 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1770 await session.commit()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1771
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1772 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1773 async def get_notifications(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1774 self,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1775 client: SatXMPPEntity,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1776 type_: Optional[NotificationType] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1777 status: Optional[NotificationStatus] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1778 requires_action: Optional[bool] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1779 min_priority: Optional[int] = None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1780 ) -> List[Notification]:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1781 """Retrieve all notifications for a given profile with optional filters.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1782
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1783 @param client: client associated with the notifications.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1784 @param type_: filter by type of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1785 @param status: filter by status of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1786 @param requires_action: filter by notifications that require user action.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1787 @param min_priority: filter by minimum priority value.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1788 @return: list of matching Notification instances.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1789 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1790 profile_id = self.profiles[client.profile]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1791 filters = [or_(Notification.profile_id == profile_id, Notification.profile_id.is_(None))]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1792
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1793 if type_:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1794 filters.append(Notification.type == type_)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1795 if status:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1796 filters.append(Notification.status == status)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1797 if requires_action is not None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1798 filters.append(Notification.requires_action == requires_action)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1799 if min_priority:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1800 filters.append(Notification.priority >= min_priority)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1801
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1802 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1803 result = await session.execute(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1804 select(Notification)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1805 .where(and_(*filters))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1806 .order_by(Notification.id)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1807 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1808 return result.scalars().all()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1809
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1810 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1811 async def delete_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1812 self, client: Optional[SatXMPPEntity], notification_id: str
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1813 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1814 """Delete a notification by its profile and id.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1815
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1816 @param client: client associated with the notification. If None, profile_id will be NULL.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1817 @param notification_id: ID of the notification to delete.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1818 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1819 profile_id = self.profiles[client.profile] if client else None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1820 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1821 await session.execute(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1822 delete(Notification).where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1823 and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1824 Notification.profile_id == profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1825 Notification.id == int(notification_id),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1826 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1827 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1828 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1829 await session.commit()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1830
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1831 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1832 async def clean_expired_notifications(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1833 self, client: Optional[SatXMPPEntity], limit_timestamp: Optional[float] = None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1834 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1835 """Cleans expired notifications and older profile-specific notifications.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1836
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1837 - Removes all notifications where the expiration timestamp has passed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1838 irrespective of their profile.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1839 - If a limit_timestamp is provided, removes older notifications with a profile set
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1840 (i.e., not global notifications) that do not require user action. If client is
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1841 provided, only remove notification for this profile.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1842
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1843 @param client: if provided, only expire notification for this client (in addition
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1844 to truly expired notifications for everybody).
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1845 @param limit_timestamp: Timestamp limit for older notifications. If None, only
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1846 truly expired notifications are removed.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1847 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1848
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1849 # Delete truly expired notifications
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1850 expired_condition = Notification.expire_at < time.time()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1851
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1852 # Delete older profile-specific notifications (created before the limit_timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1853 if client is None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1854 profile_condition = Notification.profile_id.isnot(None)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1855 else:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1856 profile_condition = Notification.profile_id == self.profiles[client.profile]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1857 older_condition = and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1858 profile_condition,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1859 Notification.timestamp < limit_timestamp if limit_timestamp else False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1860 Notification.requires_action == False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1861 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1862
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1863 # Combine the conditions
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1864 conditions = or_(expired_condition, older_condition)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1865
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1866 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1867 await session.execute(delete(Notification).where(conditions))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1868 await session.commit()

Mercurial > libervia-backend

annotate libervia/backend/memory/sqla.py @ 4202:b26339343076