libervia-backend: libervia/backend/memory/sqla.py annotate

annotate libervia/backend/memory/sqla.py @ 4387:a6270030968d default tip

doc (components): Document the handling of mailing lists in Email Gateway: fix 462

author	Goffi <goffi@goffi.org>
date	Sun, 03 Aug 2025 23:45:48 +0200
parents	a1ac33fe6b97
children

rev	line source
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	2
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia: an XMPP client
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	5
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	10
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	15
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	18
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	19 import asyncio
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	20 from asyncio.subprocess import PIPE
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	21 import copy
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	22 from datetime import datetime
4212 5f2d496c633f core: get rid of `pickle`: Goffi <goffi@goffi.org> parents: 4160 diff changeset	23 import json
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	24 from pathlib import Path
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	25 import sys
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	26 import time
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	27 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, Union
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	28
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	29 from alembic import config as al_config, script as al_script
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	30 from alembic.runtime import migration as al_migration
4373 5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	31 from sqlalchemy import and_, delete, event, exists, func, or_, update
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	32 from sqlalchemy import Integer, literal_column, text
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	33 from sqlalchemy.dialects.sqlite import insert
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	34 from sqlalchemy.engine import Connection, Engine
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	35 from sqlalchemy.exc import IntegrityError, NoResultFound
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	36 from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, create_async_engine
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	37 from sqlalchemy.future import select
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	38 from sqlalchemy.orm import (
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	39 contains_eager,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	40 joinedload,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	41 selectinload,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	42 sessionmaker,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	43 subqueryload,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	44 )
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	45 from sqlalchemy.orm.attributes import Mapped
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	46 from sqlalchemy.orm.decl_api import DeclarativeMeta
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	47 from sqlalchemy.sql.functions import coalesce, count, now, sum as sum_
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	48 from twisted.internet import defer
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	49 from twisted.words.protocols.jabber import jid
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	50 from twisted.words.xish import domish
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	51
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	52 from libervia.backend.core import exceptions
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	53 from libervia.backend.core.constants import Const as C
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	54 from libervia.backend.core.core_types import SatXMPPEntity
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	55 from libervia.backend.core.i18n import _
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	56 from libervia.backend.core.log import getLogger
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	57 from libervia.backend.memory import migration
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	58 from libervia.backend.memory import sqla_config
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	59 from libervia.backend.memory.sqla_mapping import (
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	60 AccessModel,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	61 Base,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	62 Component,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	63 File,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	64 History,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	65 Message,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	66 NOT_IN_EXTRA,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	67 Notification,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	68 NotificationPriority,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	69 NotificationStatus,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	70 NotificationType,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	71 ParamGen,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	72 ParamInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	73 PrivateGen,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	74 PrivateGenBin,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	75 PrivateInd,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	76 PrivateIndBin,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	77 Profile,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	78 PublishModel,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	79 PubsubAffiliation,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	80 PubsubItem,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	81 PubsubNode,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	82 PubsubSub,
3893 045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	83 Subject,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	84 SyncState,
045af0eeda3f core, CLI (base), tools (common/data_format): typing/core readability Goffi <goffi@goffi.org> parents: 3862 diff changeset	85 Thread,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	86 get_profile_by_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	87 profiles,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	88 )
4071 4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	89 from libervia.backend.tools.common import uri
4b842c1fb686 refactoring: renamed `sat` package to `libervia.backend` Goffi <goffi@goffi.org> parents: 4037 diff changeset	90 from libervia.backend.tools.utils import aio, as_future
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	91
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	92
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	93 log = getLogger(__name__)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	94 migration_path = Path(migration.__file__).parent
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	95 #: mapping of Libervia search query operators to SQLAlchemy method name
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	96 OP_MAP = {
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	97 "==": "__eq__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	98 "eq": "__eq__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	99 "!=": "__ne__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	100 "ne": "__ne__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	101 ">": "__gt__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	102 "gt": "__gt__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	103 "<": "__le__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	104 "le": "__le__",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	105 "between": "between",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	106 "in": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	107 "not_in": "not_in",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	108 "overlap": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	109 "ioverlap": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	110 "disjoint": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	111 "idisjoint": "in_",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	112 "like": "like",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	113 "ilike": "ilike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	114 "not_like": "notlike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	115 "not_ilike": "notilike",
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	116 }
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	117
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	118
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	119 @event.listens_for(Engine, "connect")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	120 def set_sqlite_pragma(dbapi_connection, connection_record):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	121 cursor = dbapi_connection.cursor()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	122 cursor.execute("PRAGMA foreign_keys=ON")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	123 cursor.close()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	124
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	125
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	126 class Storage:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	127
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	128 def __init__(self):
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	129 self.initialized = defer.Deferred()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	130 # we keep cache for the profiles (key: profile name, value: profile id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	131 # profile id to component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	132 self.components: Dict[int, str] = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	133
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	134 @property
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	135 def profiles(self):
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	136 return profiles
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	137
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	138 def get_profile_by_id(self, profile_id):
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	139 return get_profile_by_id(profile_id)
3665 72b0e4053ab0 core (memory/slqa): implement `getProfileById` Goffi <goffi@goffi.org> parents: 3664 diff changeset	140
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	141 async def migrate_apply(self, *args: str, log_output: bool = False) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	142 """Do a migration command
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	143
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	144 Commands are applied by running Alembic in a subprocess.
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	145 Arguments are alembic executables commands
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	146
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	147 @param log_output: manage stdout and stderr:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	148 - if False, stdout and stderr are buffered, and logged only in case of error
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	149 - if True, stdout and stderr will be logged during the command execution
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	150 @raise exceptions.DatabaseError: something went wrong while running the
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	151 process
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	152 """
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	153 stdout, stderr = 2 * (None,) if log_output else 2 * (PIPE,)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	154 proc = await asyncio.create_subprocess_exec(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	155 sys.executable,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	156 "-m",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	157 "alembic",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	158 *args,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	159 stdout=stdout,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	160 stderr=stderr,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	161 cwd=migration_path,
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	162 )
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	163 log_out, log_err = await proc.communicate()
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	164 if proc.returncode != 0:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	165 msg = _("Can't {operation} database (exit code {exit_code})").format(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	166 operation=args[0], exit_code=proc.returncode
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	167 )
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	168 if log_out or log_err:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	169 msg += f":\nstdout: {log_out.decode()}\nstderr: {log_err.decode()}"
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	170 log.error(msg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	171
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	172 raise exceptions.DatabaseError(msg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	173
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	174 async def create_db(self, engine: AsyncEngine, db_config: dict) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	175 """Create a new database
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	176
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	177 The database is generated from SQLAlchemy model, then stamped by Alembic
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	178 """
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	179 # the dir may not exist if it's not the XDG recommended one
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	180 db_config["path"].parent.mkdir(0o700, True, True)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	181 async with engine.begin() as conn:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	182 await conn.run_sync(Base.metadata.create_all)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	183
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	184 log.debug("stamping the database")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	185 await self.migrate_apply("stamp", "head")
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	186 log.debug("stamping done")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	187
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	188 def _check_db_is_up_to_date(self, conn: Connection) -> bool:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	189 al_ini_path = migration_path / "alembic.ini"
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	190 al_cfg = al_config.Config(al_ini_path)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	191 directory = al_script.ScriptDirectory.from_config(al_cfg)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	192 context = al_migration.MigrationContext.configure(conn)
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	193 return set(context.get_current_heads()) == set(directory.get_heads())
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	194
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	195 def _sqlite_set_journal_mode_wal(self, conn: Connection) -> None:
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	196 """Check if journal mode is WAL, and set it if necesssary"""
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	197 result = conn.execute(text("PRAGMA journal_mode"))
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	198 if result.scalar() != "wal":
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	199 log.info("WAL mode not activated, activating it")
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	200 conn.execute(text("PRAGMA journal_mode=WAL"))
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	201
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	202 async def check_and_update_db(self, engine: AsyncEngine, db_config: dict) -> None:
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	203 """Check that database is up-to-date, and update if necessary"""
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	204 async with engine.connect() as conn:
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	205 up_to_date = await conn.run_sync(self._check_db_is_up_to_date)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	206 if up_to_date:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	207 log.debug("Database is up-to-date")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	208 else:
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	209 log.info("Database needs to be updated")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	210 log.info("updating…")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	211 await self.migrate_apply("upgrade", "head", log_output=True)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	212 log.info("Database is now up-to-date")
71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	213
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	214 @aio
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	215 async def initialise(self) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	216 log.info(_("Connecting database"))
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	217
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	218 db_config = sqla_config.get_db_config()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	219 engine = create_async_engine(
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	220 db_config["url"],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	221 future=True,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	222 json_serializer=lambda obj: json.dumps(obj, ensure_ascii=False),
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	223 )
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	224
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	225 new_base = not db_config["path"].exists()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	226 if new_base:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	227 log.info(_("The database is new, creating the tables"))
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	228 await self.create_db(engine, db_config)
3582 71516731d0aa core (memory/sqla): database migration using Alembic: Goffi <goffi@goffi.org> parents: 3537 diff changeset	229 else:
4003 1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	230 await self.check_and_update_db(engine, db_config)
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	231
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	232 async with engine.connect() as conn:
1a77e1f866f9 core (memory/sqla): activate Write-Ahead Logging: Goffi <goffi@goffi.org> parents: 3893 diff changeset	233 await conn.run_sync(self._sqlite_set_journal_mode_wal)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	234
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	235 self.session = sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
3583 16ade4ad63f3 core (memory/sqla_mapping): fix some technical debt: Goffi <goffi@goffi.org> parents: 3582 diff changeset	236
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	237 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	238 result = await session.execute(select(Profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	239 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	240 self.profiles[p.name] = p.id
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	241 result = await session.execute(select(Component))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	242 for c in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	243 self.components[c.profile_id] = c.entry_point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	244
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	245 self.initialized.callback(None)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	246
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	247 ## Generic
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	248
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	249 @aio
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	250 async def get(
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	251 self,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	252 client: SatXMPPEntity,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	253 db_cls: DeclarativeMeta,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	254 db_id_col: Mapped,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	255 id_value: Any,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	256 joined_loads=None,
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	257 ) -> Optional[DeclarativeMeta]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	258 stmt = select(db_cls).where(db_id_col == id_value)
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	259 if client is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	260 stmt = stmt.filter_by(profile_id=self.profiles[client.profile])
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	261 if joined_loads is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	262 for joined_load in joined_loads:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	263 stmt = stmt.options(joinedload(joined_load))
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	264 async with self.session() as session:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	265 result = await session.execute(stmt)
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	266 if joined_loads is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	267 result = result.unique()
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	268 return result.scalar_one_or_none()
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	269
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	270 @aio
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	271 async def add(self, db_obj: DeclarativeMeta) -> None:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	272 """Add an object to database"""
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	273 async with self.session() as session:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	274 async with session.begin():
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	275 session.add(db_obj)
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	276
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	277 @aio
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	278 async def delete(
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	279 self,
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	280 db_obj: Union[DeclarativeMeta, List[DeclarativeMeta]],
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	281 session_add: Optional[List[DeclarativeMeta]] = None,
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	282 ) -> None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	283 """Delete an object from database
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	284
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	285 @param db_obj: object to delete or list of objects to delete
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	286 @param session_add: other objects to add to session.
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	287 This is useful when parents of deleted objects needs to be updated too, or if
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	288 other objects needs to be updated in the same transaction.
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	289 """
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	290 if not db_obj:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	291 return
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	292 if not isinstance(db_obj, list):
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	293 db_obj = [db_obj]
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	294 async with self.session() as session:
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	295 async with session.begin():
3798 b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	296 if session_add is not None:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	297 for obj in session_add:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	298 session.add(obj)
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	299 for obj in db_obj:
b5013bada4b6 core (memory/sqla): generic `get` + `session_add` in `delete`: Goffi <goffi@goffi.org> parents: 3797 diff changeset	300 await session.delete(obj)
3596 2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	301 await session.commit()
2d97c695af05 core (memory/sqla): generic methods to add and delete ORM objects directly Goffi <goffi@goffi.org> parents: 3595 diff changeset	302
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	303 ## Profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	304
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	305 def get_profiles_list(self) -> List[str]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	306 """ "Return list of all registered profiles"""
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	307 return list(self.profiles.keys())
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	308
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	309 def has_profile(self, profile_name: str) -> bool:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	310 """return True if profile_name exists
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	311
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	312 @param profile_name: name of the profile to check
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	313 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	314 return profile_name in self.profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	315
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	316 def profile_is_component(self, profile_name: str) -> bool:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	317 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	318 return self.profiles[profile_name] in self.components
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	319 except KeyError:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	320 raise exceptions.NotFound("the requested profile doesn't exists")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	321
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	322 def get_entry_point(self, profile_name: str) -> str:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	323 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	324 return self.components[self.profiles[profile_name]]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	325 except KeyError:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	326 raise exceptions.NotFound(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	327 "the requested profile doesn't exists or is not a component"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	328 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	329
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	330 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	331 async def create_profile(self, name: str, component_ep: Optional[str] = None) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	332 """Create a new profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	333
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	334 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	335 @param component: if not None, must point to a component entry point
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	336 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	337 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	338 profile = Profile(name=name)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	339 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	340 session.add(profile)
3638 257135d5c5c2 core (memory/sqla): fix key/value inversion in self.profiles Goffi <goffi@goffi.org> parents: 3621 diff changeset	341 self.profiles[profile.name] = profile.id
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	342 if component_ep is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	343 async with session.begin():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	344 component = Component(profile=profile, entry_point=component_ep)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	345 session.add(component)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	346 self.components[profile.id] = component_ep
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	347 return profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	348
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	349 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	350 async def delete_profile(self, name: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	351 """Delete profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	352
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	353 @param name: name of the profile
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	354 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	355 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	356 result = await session.execute(select(Profile).where(Profile.name == name))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	357 profile = result.scalar()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	358 await session.delete(profile)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	359 await session.commit()
3638 257135d5c5c2 core (memory/sqla): fix key/value inversion in self.profiles Goffi <goffi@goffi.org> parents: 3621 diff changeset	360 del self.profiles[profile.name]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	361 if profile.id in self.components:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	362 del self.components[profile.id]
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	363 log.info(_("Profile {name!r} deleted").format(name=name))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	364
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	365 ## Params
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	366
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	367 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	368 async def load_gen_params(self, params_gen: dict) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	369 """Load general parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	370
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	371 @param params_gen: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	372 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	373 log.debug(_("loading general parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	374 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	375 result = await session.execute(select(ParamGen))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	376 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	377 params_gen[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	378
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	379 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	380 async def load_ind_params(self, params_ind: dict, profile: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	381 """Load individual parameters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	382
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	383 @param params_ind: dictionary to fill
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	384 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	385 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	386 log.debug(_("loading individual parameters from database"))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	387 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	388 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	389 select(ParamInd).where(ParamInd.profile_id == self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	390 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	391 for p in result.scalars():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	392 params_ind[(p.category, p.name)] = p.value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	393
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	394 @aio
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	395 async def get_ind_param(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	396 self, category: str, name: str, profile: str
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	397 ) -> Optional[str]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	398 """Ask database for the value of one specific individual parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	399
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	400 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	401 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	402 @param profile: %(doc_profile)s
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	403 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	404 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	405 result = await session.execute(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	406 select(ParamInd.value).filter_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	407 category=category, name=name, profile_id=self.profiles[profile]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	408 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	409 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	410 return result.scalar_one_or_none()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	411
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	412 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	413 async def get_ind_param_values(self, category: str, name: str) -> Dict[str, str]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	414 """Ask database for the individual values of a parameter for all profiles
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	415
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	416 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	417 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	418 @return dict: profile => value map
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	419 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	420 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	421 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	422 select(ParamInd)
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	423 .filter_by(category=category, name=name)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	424 .options(subqueryload(ParamInd.profile))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	425 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	426 return {param.profile.name: param.value for param in result.scalars()}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	427
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	428 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	429 async def set_gen_param(self, category: str, name: str, value: Optional[str]) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	430 """Save the general parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	431
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	432 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	433 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	434 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	435 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	436 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	437 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	438 insert(ParamGen)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	439 .values(category=category, name=name, value=value)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	440 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	441 index_elements=(ParamGen.category, ParamGen.name),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	442 set_={ParamGen.value: value},
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	443 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	444 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	445 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	446 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	447
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	448 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	449 async def set_ind_param(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	450 self, category: str, name: str, value: Optional[str], profile: str
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	451 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	452 """Save the individual parameters in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	453
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	454 @param category: category of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	455 @param name: name of the parameter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	456 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	457 @param profile: a profile which must exist
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	458 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	459 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	460 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	461 insert(ParamInd)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	462 .values(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	463 category=category,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	464 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	465 profile_id=self.profiles[profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	466 value=value,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	467 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	468 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	469 index_elements=(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	470 ParamInd.category,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	471 ParamInd.name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	472 ParamInd.profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	473 ),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	474 set_={ParamInd.value: value},
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	475 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	476 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	477 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	478 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	479
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	480 def _jid_filter(self, jid_: jid.JID, dest: bool = False):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	481 """Generate condition to filter on a JID, using relevant columns
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	482
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	483 @param dest: True if it's the destinee JID, otherwise it's the source one
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	484 @param jid_: JID to filter by
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	485 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	486 if jid_.resource:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	487 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	488 return and_(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	489 History.dest == jid_.userhost(), History.dest_res == jid_.resource
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	490 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	491 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	492 return and_(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	493 History.source == jid_.userhost(), History.source_res == jid_.resource
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	494 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	495 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	496 if dest:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	497 return History.dest == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	498 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	499 return History.source == jid_.userhost()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	500
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	501 @aio
4366 1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	502 async def get_history_from_xmpp_id(
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	503 self, client: SatXMPPEntity, message_id: str, message_type: str\|None
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	504 ) -> History \| None:
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	505 """Retrieves history row that match a specific message ID.
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	506
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	507 The retrieval criteria vary based on the message type.
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	508
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	509 @param message_type: The type of the message (one of C.MESS_TYPE_*)
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	510 @param message_id: the ID of the message, XEP-0359's stanza-id is used for
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	511 groupchat, origin-id from the same XEP otherwise.
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	512 @return: History instance with messages, subjects and threads, if any found, None
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	513 otherwise.
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	514 """
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	515 profile_id = self.profiles[client.profile]
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	516 async with self.session() as session:
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	517 if message_type == C.MESS_TYPE_GROUPCHAT:
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	518 query = select(History).where(
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	519 History.profile_id == profile_id, History.stanza_id == message_id
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	520 )
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	521 else:
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	522 query = select(History).where(
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	523 History.profile_id == profile_id, History.origin_id == message_id
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	524 )
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	525
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	526 query = (
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	527 query
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	528 .outerjoin(History.messages)
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	529 .outerjoin(History.subjects)
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	530 .outerjoin(History.thread)
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	531 .options(
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	532 contains_eager(History.messages),
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	533 contains_eager(History.subjects),
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	534 contains_eager(History.thread),
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	535 )
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	536 )
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	537
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	538 result = await session.execute(query)
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	539 history = result.scalars().first()
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	540
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	541 return history
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	542
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	543
1ef32316a55e plugin XEP-0444, storage: move `get_history_from_xmpp_id` to storage as it is generally useful. Goffi <goffi@goffi.org> parents: 4270 diff changeset	544 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	545 async def history_get(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	546 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	547 from_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	548 to_jid: Optional[jid.JID],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	549 limit: Optional[int] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	550 between: bool = True,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	551 filters: Optional[Dict[str, str]] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	552 profile: Optional[str] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	553 ) -> List[Tuple[str, int, str, str, Dict[str, str], Dict[str, str], str, str, str]]:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	554 """Retrieve messages in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	555
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	556 @param from_jid: source JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	557 @param to_jid: dest JID (full, or bare for catchall)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	558 @param limit: maximum number of messages to get:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	559 - 0 for no message (returns the empty list)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	560 - None for unlimited
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	561 @param between: confound source and dest (ignore the direction)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	562 @param filters: pattern to filter the history results
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	563 @return: list of messages as in [message_new], minus the profile which is already
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	564 known.
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	565 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	566 # we have to set a default value to profile because it's last argument
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	567 # and thus follow other keyword arguments with default values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	568 # but None should not be used for it
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	569 assert profile is not None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	570 if limit == 0:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	571 return []
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	572 if filters is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	573 filters = {}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	574
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	575 stmt = (
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	576 select(History)
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	577 .filter_by(profile_id=self.profiles[profile])
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	578 .outerjoin(History.messages)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	579 .outerjoin(History.subjects)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	580 .outerjoin(History.thread)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	581 .options(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	582 contains_eager(History.messages),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	583 contains_eager(History.subjects),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	584 contains_eager(History.thread),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	585 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	586 .order_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	587 # timestamp may be identical for 2 close messages (specially when delay is
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	588 # used) that's why we order ties by received_timestamp. We'll reverse the
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	589 # order when returning the result. We use DESC here so LIMIT keep the last
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	590 # messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	591 History.timestamp.desc(),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	592 History.received_timestamp.desc(),
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	593 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	594 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	595
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	596 if not from_jid and not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	597 # no jid specified, we want all one2one communications
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	598 pass
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	599 elif between:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	600 if not from_jid or not to_jid:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	601 # we only have one jid specified, we check all messages
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	602 # from or to this jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	603 jid_ = from_jid or to_jid
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	604 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	605 or_(self._jid_filter(jid_), self._jid_filter(jid_, dest=True))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	606 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	607 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	608 # we have 2 jids specified, we check all communications between
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	609 # those 2 jids
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	610 stmt = stmt.where(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	611 or_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	612 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	613 self._jid_filter(from_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	614 self._jid_filter(to_jid, dest=True),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	615 ),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	616 and_(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	617 self._jid_filter(to_jid),
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	618 self._jid_filter(from_jid, dest=True),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	619 ),
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	620 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	621 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	622 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	623 # we want one communication in specific direction (from somebody or
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	624 # to somebody).
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	625 if from_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	626 stmt = stmt.where(self._jid_filter(from_jid))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	627 if to_jid is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	628 stmt = stmt.where(self._jid_filter(to_jid, dest=True))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	629
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	630 if filters:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	631 if "timestamp_start" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	632 stmt = stmt.where(History.timestamp >= float(filters["timestamp_start"]))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	633 if "before_uid" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	634 # orignially this query was using SQLITE's rowid. This has been changed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	635 # to use coalesce(received_timestamp, timestamp) to be SQL engine independant
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	636 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	637 coalesce(History.received_timestamp, History.timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	638 < (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	639 select(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	640 coalesce(History.received_timestamp, History.timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	641 ).filter_by(uid=filters["before_uid"])
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	642 ).scalar_subquery()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	643 )
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	644 if "body" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	645 # TODO: use REGEXP (function to be defined) instead of GLOB: https://www.sqlite.org/lang_expr.html
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	646 stmt = stmt.where(Message.message.like(f"%{filters['body']}%"))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	647 if "search" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	648 search_term = f"%{filters['search']}%"
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	649 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	650 or_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	651 Message.message.like(search_term),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	652 History.source_res.like(search_term),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	653 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	654 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	655 if "types" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	656 types = filters["types"].split()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	657 stmt = stmt.where(History.type.in_(types))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	658 if "not_types" in filters:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	659 types = filters["not_types"].split()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	660 stmt = stmt.where(History.type.not_in(types))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	661 if "last_stanza_id" in filters:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	662 # this request get the last message with a "stanza_id" that we
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	663 # have in history. This is mainly used to retrieve messages sent
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	664 # while we were offline, using MAM (XEP-0313).
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	665 if filters["last_stanza_id"] is not True or limit != 1:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	666 raise ValueError("Unexpected values for last_stanza_id filter")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	667 stmt = stmt.where(History.stanza_id.is_not(None))
4160 6a0066ea5c97 core (memory/sqla): add `id` filter in `history_get` Goffi <goffi@goffi.org> parents: 4130 diff changeset	668 if "id" in filters:
6a0066ea5c97 core (memory/sqla): add `id` filter in `history_get` Goffi <goffi@goffi.org> parents: 4130 diff changeset	669 stmt = stmt.where(History.uid == filters["id"])
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	670 if "origin_id" in filters:
3797 cc653b2685f0 core (memory/sqla), plugin XEP-0359: always add `origin-id`, and store: Goffi <goffi@goffi.org> parents: 3754 diff changeset	671 stmt = stmt.where(History.origin_id == filters["origin_id"])
4367 b93f95efe329 storage: Add a `thread_id` filter to `history_get`: Goffi <goffi@goffi.org> parents: 4366 diff changeset	672 if "thread_id" in filters:
b93f95efe329 storage: Add a `thread_id` filter to `history_get`: Goffi <goffi@goffi.org> parents: 4366 diff changeset	673 stmt = stmt.where(Thread.thread_id == filters["thread_id"])
4373 5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	674 if "keyword" in filters:
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	675 keyword = filters["keyword"]
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	676 # FIXME: "json_each" works with SQLite, but when PostGreSQL support will
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	677 # be added, "json_array_elements" may have to be used with it.
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	678 subquery = func.json_each(History.extra["keywords"]).table_valued("value")
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	679 stmt = stmt.where(
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	680 exists().where(subquery.c.value == keyword)
5e48ae998976 memory (sqla): Add filter for keyword: Goffi <goffi@goffi.org> parents: 4368 diff changeset	681 )
4376 7ac28a270b7f memory (sqla): Add a filter for `origin` in `history_get`: Goffi <goffi@goffi.org> parents: 4373 diff changeset	682 if "origin" in filters:
7ac28a270b7f memory (sqla): Add a filter for `origin` in `history_get`: Goffi <goffi@goffi.org> parents: 4373 diff changeset	683 stmt = stmt.where(
7ac28a270b7f memory (sqla): Add a filter for `origin` in `history_get`: Goffi <goffi@goffi.org> parents: 4373 diff changeset	684 History.extra["origin"].as_string() == filters["origin"]
7ac28a270b7f memory (sqla): Add a filter for `origin` in `history_get`: Goffi <goffi@goffi.org> parents: 4373 diff changeset	685 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	686
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	687 if limit is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	688 stmt = stmt.limit(limit)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	689
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	690 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	691 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	692
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	693 result = result.scalars().unique().all()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	694 result.reverse()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	695 return [h.as_tuple() for h in result]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	696
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	697 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	698 async def add_to_history(self, data: dict, profile: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	699 """Store a new message in history
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	700
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	701 @param data: message data as build by SatMessageProtocol.onMessage
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	702 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	703 extra = {k: v for k, v in data["extra"].items() if k not in NOT_IN_EXTRA}
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	704 messages = [
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	705 Message(message=mess, language=lang) for lang, mess in data["message"].items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	706 ]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	707 subjects = [
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	708 Subject(subject=mess, language=lang) for lang, mess in data["subject"].items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	709 ]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	710 if "thread" in data["extra"]:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	711 thread = Thread(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	712 thread_id=data["extra"]["thread"],
4367 b93f95efe329 storage: Add a `thread_id` filter to `history_get`: Goffi <goffi@goffi.org> parents: 4366 diff changeset	713 parent_id=data["extra"].get("thread_parent"),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	714 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	715 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	716 thread = None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	717 try:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	718 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	719 async with session.begin():
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	720 session.add(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	721 History(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	722 uid=data["uid"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	723 origin_id=data["extra"].get("origin_id"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	724 stanza_id=data["extra"].get("stanza_id"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	725 update_uid=data["extra"].get("update_uid"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	726 profile_id=self.profiles[profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	727 source_jid=data["from"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	728 dest_jid=data["to"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	729 timestamp=data["timestamp"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	730 received_timestamp=data.get("received_timestamp"),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	731 type=data["type"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	732 extra=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	733 messages=messages,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	734 subjects=subjects,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	735 thread=thread,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	736 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	737 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	738 except IntegrityError as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	739 if "unique" in str(e.orig).lower():
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	740 log.debug(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	741 f"message {data['uid']!r} is already in history, not storing it again"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	742 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	743 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	744 log.error(f"Can't store message {data['uid']!r} in history: {e}")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	745 except Exception as e:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	746 log.critical(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	747 f"Can't store message, unexpected exception (uid: {data['uid']}): {e}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	748 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	749
4368 2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	750 @aio
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	751 async def add_thread(
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	752 self,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	753 message_uid: str,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	754 thread_id: str,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	755 parent_id: str\|None,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	756 is_retroactive: bool,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	757 ) -> None:
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	758 """Add a thread to a message.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	759
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	760 @param message_uid: Internal ID of the message (History.uid) to which the thread
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	761 is added.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	762 @param thread_id: The thread ID to set.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	763 @param parent_id: The parent message ID in the thread.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	764 @param is_retroactive: Whether the thread ID was added after the message was sent.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	765 Is is used when we want to relate a parent message to a new thread, when the
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	766 parent message didn't have a thread ID initially.
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	767 """
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	768 async with self.session() as session:
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	769 thread = Thread(
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	770 thread_id=thread_id,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	771 parent_id=parent_id,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	772 is_retroactive=is_retroactive,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	773 history_uid=message_uid,
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	774 )
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	775 session.add(thread)
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	776 await session.commit()
2bdf0c16d852 storage: Add a `add_thread` method: Goffi <goffi@goffi.org> parents: 4367 diff changeset	777
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	778 ## Private values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	779
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	780 def _get_private_class(self, binary, profile):
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	781 """Get ORM class to use for private values"""
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	782 if profile is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	783 return PrivateGenBin if binary else PrivateGen
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	784 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	785 return PrivateIndBin if binary else PrivateInd
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	786
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	787 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	788 async def get_privates(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	789 self,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	790 namespace: str,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	791 keys: Optional[Iterable[str]] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	792 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	793 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	794 ) -> Dict[str, Any]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	795 """Get private value(s) from databases
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	796
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	797 @param namespace: namespace of the values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	798 @param keys: keys of the values to get None to get all keys/values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	799 @param binary: True to deserialise binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	800 @param profile: profile to use for individual values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	801 None to use general values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	802 @return: gotten keys/values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	803 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	804 if keys is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	805 keys = list(keys)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	806 log.debug(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	807 f"getting {'general' if profile is None else 'individual'}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	808 f"{' binary' if binary else ''} private values from database for namespace "
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	809 f"{namespace}{f' with keys {keys!r}' if keys is not None else ''}"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	810 )
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	811 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	812 stmt = select(cls).filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	813 if keys:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	814 stmt = stmt.where(cls.key.in_(list(keys)))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	815 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	816 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	817 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	818 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	819 return {p.key: p.value for p in result.scalars()}
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	820
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	821 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	822 async def set_private_value(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	823 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	824 namespace: str,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	825 key: str,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	826 value: Any,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	827 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	828 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	829 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	830 """Set a private value in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	831
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	832 @param namespace: namespace of the values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	833 @param key: key of the value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	834 @param value: value to set
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	835 @param binary: True if it's a binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	836 binary values need to be serialised, used for everything but strings
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	837 @param profile: profile to use for individual value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	838 if None, it's a general value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	839 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	840 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	841
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	842 values = {"namespace": namespace, "key": key, "value": value}
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	843 index_elements = [cls.namespace, cls.key]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	844
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	845 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	846 values["profile_id"] = self.profiles[profile]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	847 index_elements.append(cls.profile_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	848
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	849 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	850 await session.execute(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	851 insert(cls)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	852 .values(**values)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	853 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	854 index_elements=index_elements, set_={cls.value: value}
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	855 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	856 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	857 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	858
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	859 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	860 async def del_private_value(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	861 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	862 namespace: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	863 key: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	864 binary: bool = False,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	865 profile: Optional[str] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	866 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	867 """Delete private value from database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	868
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	869 @param category: category of the privateeter
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	870 @param key: key of the private value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	871 @param binary: True if it's a binary values
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	872 @param profile: profile to use for individual value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	873 if None, it's a general value
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	874 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	875 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	876
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	877 stmt = delete(cls).filter_by(namespace=namespace, key=key)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	878
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	879 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	880 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	881
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	882 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	883 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	884 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	885
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	886 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	887 async def del_private_namespace(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	888 self, namespace: str, binary: bool = False, profile: Optional[str] = None
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	889 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	890 """Delete all data from a private namespace
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	891
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	892 Be really cautious when you use this method, as all data with given namespace are
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	893 removed.
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	894 Params are the same as for del_private_value
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	895 """
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	896 cls = self._get_private_class(binary, profile)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	897
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	898 stmt = delete(cls).filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	899
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	900 if profile is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	901 stmt = stmt.filter_by(profile_id=self.profiles[profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	902
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	903 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	904 await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	905 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	906
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	907 ## Files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	908
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	909 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	910 async def get_files(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	911 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	912 client: Optional[SatXMPPEntity],
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	913 file_id: Optional[str] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	914 version: Optional[str] = "",
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	915 parent: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	916 type_: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	917 file_hash: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	918 hash_algo: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	919 name: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	920 namespace: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	921 mime_type: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	922 public_id: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	923 owner: Optional[jid.JID] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	924 access: Optional[dict] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	925 projection: Optional[List[str]] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	926 unique: bool = False,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	927 ) -> List[dict]:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	928 """Retrieve files with with given filters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	929
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	930 @param file_id: id of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	931 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	932 @param version: version of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	933 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	934 empty string to look for current version
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	935 @param parent: id of the directory containing the files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	936 None to ignore
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	937 empty string to look for root files/directories
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	938 @param projection: name of columns to retrieve
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	939 None to retrieve all
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	940 @param unique: if True will remove duplicates
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	941 other params are the same as for [set_file]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	942 @return: files corresponding to filters
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	943 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	944 if projection is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	945 projection = [
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	946 "id",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	947 "version",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	948 "parent",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	949 "type",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	950 "file_hash",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	951 "hash_algo",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	952 "name",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	953 "size",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	954 "namespace",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	955 "media_type",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	956 "media_subtype",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	957 "public_id",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	958 "created",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	959 "modified",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	960 "owner",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	961 "access",
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	962 "extra",
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	963 ]
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	964
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	965 stmt = select(*[getattr(File, f) for f in projection])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	966
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	967 if unique:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	968 stmt = stmt.distinct()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	969
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	970 if client is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	971 stmt = stmt.filter_by(profile_id=self.profiles[client.profile])
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	972 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	973 if public_id is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	974 raise exceptions.InternalError(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	975 "client can only be omitted when public_id is set"
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	976 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	977 if file_id is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	978 stmt = stmt.filter_by(id=file_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	979 if version is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	980 stmt = stmt.filter_by(version=version)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	981 if parent is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	982 stmt = stmt.filter_by(parent=parent)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	983 if type_ is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	984 stmt = stmt.filter_by(type=type_)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	985 if file_hash is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	986 stmt = stmt.filter_by(file_hash=file_hash)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	987 if hash_algo is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	988 stmt = stmt.filter_by(hash_algo=hash_algo)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	989 if name is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	990 stmt = stmt.filter_by(name=name)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	991 if namespace is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	992 stmt = stmt.filter_by(namespace=namespace)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	993 if mime_type is not None:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	994 if "/" in mime_type:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	995 media_type, media_subtype = mime_type.split("/", 1)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	996 stmt = stmt.filter_by(media_type=media_type, media_subtype=media_subtype)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	997 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	998 stmt = stmt.filter_by(media_type=mime_type)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	999 if public_id is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1000 stmt = stmt.filter_by(public_id=public_id)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1001 if owner is not None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1002 stmt = stmt.filter_by(owner=owner)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1003 if access is not None:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1004 raise NotImplementedError("Access check is not implemented yet")
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1005 # a JSON comparison is needed here
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1006
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1007 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1008 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1009
4092 74c66c0d93f3 core (memory/sqla): fix row to dict conversion in `get_files` Goffi <goffi@goffi.org> parents: 4071 diff changeset	1010 return [r._asdict() for r in result]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1011
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1012 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1013 async def set_file(
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1014 self,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1015 client: SatXMPPEntity,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1016 name: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1017 file_id: str,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1018 version: str = "",
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1019 parent: str = "",
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1020 type_: str = C.FILE_TYPE_FILE,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1021 file_hash: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1022 hash_algo: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1023 size: int = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1024 namespace: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1025 mime_type: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1026 public_id: Optional[str] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1027 created: Optional[float] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1028 modified: Optional[float] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1029 owner: Optional[jid.JID] = None,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1030 access: Optional[dict] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1031 extra: Optional[dict] = None,
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1032 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1033 """Set a file metadata
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1034
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1035 @param client: client owning the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1036 @param name: name of the file (must not contain "/")
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1037 @param file_id: unique id of the file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1038 @param version: version of this file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1039 @param parent: id of the directory containing this file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1040 Empty string if it is a root file/directory
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1041 @param type_: one of:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1042 - file
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1043 - directory
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1044 @param file_hash: unique hash of the payload
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1045 @param hash_algo: algorithm used for hashing the file (usually sha-256)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1046 @param size: size in bytes
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1047 @param namespace: identifier (human readable is better) to group files
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1048 for instance, namespace could be used to group files in a specific photo album
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1049 @param mime_type: media type of the file, or None if not known/guessed
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1050 @param public_id: ID used to server the file publicly via HTTP
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1051 @param created: UNIX time of creation
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1052 @param modified: UNIX time of last modification, or None to use created date
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1053 @param owner: jid of the owner of the file (mainly useful for component)
3621 9e1a993ad1bf core (memory/sqla): minor docstring line lenght fix Goffi <goffi@goffi.org> parents: 3596 diff changeset	1054 @param access: serialisable dictionary with access rules. See [memory.memory] for
9e1a993ad1bf core (memory/sqla): minor docstring line lenght fix Goffi <goffi@goffi.org> parents: 3596 diff changeset	1055 details
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1056 @param extra: serialisable dictionary of any extra data
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1057 will be encoded to json in database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1058 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1059 if mime_type is None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1060 media_type = media_subtype = None
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1061 elif "/" in mime_type:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1062 media_type, media_subtype = mime_type.split("/", 1)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1063 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1064 media_type, media_subtype = mime_type, None
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1065
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1066 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1067 async with session.begin():
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1068 session.add(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1069 File(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1070 id=file_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1071 version=version.strip(),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1072 parent=parent,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1073 type=type_,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1074 file_hash=file_hash,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1075 hash_algo=hash_algo,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1076 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1077 size=size,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1078 namespace=namespace,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1079 media_type=media_type,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1080 media_subtype=media_subtype,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1081 public_id=public_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1082 created=time.time() if created is None else created,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1083 modified=modified,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1084 owner=owner,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1085 access=access,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1086 extra=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1087 profile_id=self.profiles[client.profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1088 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1089 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1090
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1091 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1092 async def file_get_used_space(self, client: SatXMPPEntity, owner: jid.JID) -> int:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1093 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1094 result = await session.execute(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1095 select(sum_(File.size)).filter_by(
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1096 owner=owner,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1097 type=C.FILE_TYPE_FILE,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1098 profile_id=self.profiles[client.profile],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1099 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1100 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1101 return result.scalar_one_or_none() or 0
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1102
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1103 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1104 async def file_delete(self, file_id: str) -> None:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1105 """Delete file metadata from the database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1106
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1107 @param file_id: id of the file to delete
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1108 NOTE: file itself must still be removed, this method only handle metadata in
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1109 database
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1110 """
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1111 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1112 await session.execute(delete(File).filter_by(id=file_id))
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1113 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1114
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1115 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1116 async def file_update(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1117 self, file_id: str, column: str, update_cb: Callable[[dict], None]
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1118 ) -> None:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1119 """Update a column value using a method to avoid race conditions
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1120
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1121 the older value will be retrieved from database, then update_cb will be applied to
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1122 update it, and file will be updated checking that older value has not been changed
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1123 meanwhile by another user. If it has changed, it tries again a couple of times
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1124 before failing
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1125 @param column: column name (only "access" or "extra" are allowed)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1126 @param update_cb: method to update the value of the colum
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1127 the method will take older value as argument, and must update it in place
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1128 update_cb must not care about serialization,
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1129 it get the deserialized data (i.e. a Python object) directly
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1130 @raise exceptions.NotFound: there is not file with this id
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1131 """
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1132 if column not in ("access", "extra"):
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1133 raise exceptions.InternalError("bad column name")
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1134 orm_col = getattr(File, column)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1135
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1136 for i in range(5):
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1137 async with self.session() as session:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1138 try:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1139 value = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1140 await session.execute(select(orm_col).filter_by(id=file_id))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1141 ).scalar_one()
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1142 except NoResultFound:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1143 raise exceptions.NotFound
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1144 old_value = copy.deepcopy(value)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1145 update_cb(value)
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1146 stmt = update(File).filter_by(id=file_id).values({column: value})
bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1147 if not old_value:
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1148 # because JsonDefaultDict convert NULL to an empty dict, we have to
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1149 # test both for empty dict and None when we have an empty dict
bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1150 stmt = stmt.where((orm_col == None) \| (orm_col == old_value))
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1151 else:
3673 bd13391ee29e core (memory/sqla): fix `fileUpdate` Goffi <goffi@goffi.org> parents: 3665 diff changeset	1152 stmt = stmt.where(orm_col == old_value)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1153 result = await session.execute(stmt)
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1154 await session.commit()
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1155
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1156 if result.rowcount == 1:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1157 break
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1158
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1159 log.warning(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1160 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1161 "table not updated, probably due to race condition, trying again "
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1162 "({tries})"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1163 ).format(tries=i + 1)
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1164 )
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1165
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1166 else:
f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1167 raise exceptions.DatabaseError(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1168 _("Can't update file {file_id} due to race condition").format(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1169 file_id=file_id
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1170 )
3537 f9a5b810f14d core (memory/storage): backend storage is now based on SQLAlchemy Goffi <goffi@goffi.org> parents: diff changeset	1171 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1172
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1173 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1174 async def get_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1175 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1176 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1177 service: jid.JID,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1178 name: str,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1179 with_items: bool = False,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1180 with_subscriptions: bool = False,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1181 with_affiliations: bool = False,
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1182 create: bool = False,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1183 create_kwargs: dict\|None = None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1184 ) -> PubsubNode\|None:
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1185 """Retrieve a PubsubNode from DB
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1186
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1187 @param service: service hosting the node
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1188 @param name: node's name
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1189 @param with_items: retrieve items in the same query
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1190 @param with_subscriptions: retrieve subscriptions in the same query
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1191 @param with_affiliations: retrieve affiliations in the same query
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1192 @param create: if the node doesn't exist in DB, create it
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1193 @param create_kwargs: keyword arguments to use with ``set_pubsub_node`` if the node
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1194 needs to be created.
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1195 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1196 async with self.session() as session:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1197 stmt = select(PubsubNode).filter_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1198 service=service,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1199 name=name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1200 profile_id=self.profiles[client.profile],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1201 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1202 if with_items:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1203 stmt = stmt.options(joinedload(PubsubNode.items))
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1204 if with_subscriptions:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1205 stmt = stmt.options(joinedload(PubsubNode.subscriptions))
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1206 if with_affiliations:
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1207 stmt = stmt.options(joinedload(PubsubNode.affiliations))
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1208 result = await session.execute(stmt)
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1209 ret = result.unique().scalar_one_or_none()
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1210 if ret is None and create:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1211 # we auto-create the node
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1212 if create_kwargs is None:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1213 create_kwargs = {}
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1214 try:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1215 return await as_future(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1216 self.set_pubsub_node(client, service, name, **create_kwargs)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1217 )
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1218 except IntegrityError as e:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1219 if "unique" in str(e.orig).lower():
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1220 # the node may already exist, if it has been created just after
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1221 # get_pubsub_node above
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1222 log.debug("ignoring UNIQUE constraint error")
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1223 cached_node = await as_future(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1224 self.get_pubsub_node(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1225 client,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1226 service,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1227 name,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1228 with_items=with_items,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1229 with_subscriptions=with_subscriptions,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1230 with_affiliations=with_affiliations,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1231 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1232 )
3862 100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1233 else:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1234 raise e
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1235 else:
100dd30244c6 core (memory/sqla): add argument to `getPubsubNode` to auto-create a node: Goffi <goffi@goffi.org> parents: 3813 diff changeset	1236 return ret
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1237
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1238 @aio
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1239 async def get_pubsub_nodes(
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1240 self,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1241 client: SatXMPPEntity\|None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1242 service: jid.JID\|None
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1243 ) -> list[PubsubNode]:
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1244 """Retrieve pubsub nodes matching arguments.
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1245
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1246 @param client: If set, only return nodes of this client profile.
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1247 @param service: If set, only return nodes from this service.
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1248 @return: List of matching pubsub nodes.
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1249 """
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1250 async with self.session() as session:
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1251 stm = select(PubsubNode)
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1252 if client is not None:
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1253 profile_id = self.profiles[client.profile]
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1254 stm = stm.where(PubsubNode.profile_id == profile_id)
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1255 if service is not None:
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1256 stm = stm.where(PubsubNode.service == service)
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1257 result = await session.execute(stm)
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1258
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1259 return result.scalars().all()
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1260
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1261 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1262 async def set_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1263 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1264 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1265 service: jid.JID,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1266 name: str,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1267 access_model: AccessModel\|None = None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1268 publish_model: PublishModel\|None = None,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1269 analyser: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1270 type_: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1271 subtype: Optional[str] = None,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1272 subscribed: bool = False,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1273 extra: dict\|None = None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1274 items: list[PubsubItem]\|None = None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1275 affiliations: list[PubsubAffiliation]\|None = None,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1276 subscriptions: list[PubsubSub]\|None = None,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1277 ) -> PubsubNode:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1278 node = PubsubNode(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1279 profile_id=self.profiles[client.profile],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1280 service=service,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1281 name=name,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1282 access_model=access_model,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1283 publish_model=publish_model,
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1284 subscribed=subscribed,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1285 analyser=analyser,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1286 type_=type_,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1287 subtype=subtype,
4385 a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1288 extra = extra,
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1289 items = items or [],
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1290 affiliations = affiliations or [],
a1ac33fe6b97 memory (sqla): Add columns and tables to handles permissions: Goffi <goffi@goffi.org> parents: 4376 diff changeset	1291 subscriptions = subscriptions or []
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1292 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1293 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1294 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1295 session.add(node)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1296 return node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1297
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1298 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1299 async def update_pubsub_node_sync_state(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1300 self, node: PubsubNode, state: SyncState
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1301 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1302 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1303 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1304 await session.execute(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1305 update(PubsubNode)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1306 .filter_by(id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1307 .values(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1308 sync_state=state,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1309 sync_state_updated=time.time(),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1310 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1311 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1312
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1313 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1314 async def delete_pubsub_node(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1315 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1316 profiles: Optional[List[str]],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1317 services: Optional[List[jid.JID]],
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1318 names: Optional[List[str]],
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1319 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1320 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1321
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1322 @param profiles: profile names from which nodes must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1323 None to remove nodes from ALL profiles
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1324 @param services: JIDs of pubsub services from which nodes must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1325 None to remove nodes from ALL services
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1326 @param names: names of nodes which must be deleted.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1327 None to remove ALL nodes whatever is their names
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1328 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1329 stmt = delete(PubsubNode)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1330 if profiles is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1331 stmt = stmt.where(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1332 PubsubNode.profile.in_([self.profiles[p] for p in profiles])
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1333 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1334 if services is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1335 stmt = stmt.where(PubsubNode.service.in_(services))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1336 if names is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1337 stmt = stmt.where(PubsubNode.name.in_(names))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1338 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1339 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1340 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1341
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1342 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1343 async def cache_pubsub_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1344 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1345 client: SatXMPPEntity,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1346 node: PubsubNode,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1347 items: List[domish.Element],
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1348 parsed_items: Optional[List[dict]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1349 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1350 """Add items to database, using an upsert taking care of "updated" field"""
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1351 if parsed_items is not None and len(items) != len(parsed_items):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1352 raise exceptions.InternalError(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1353 "parsed_items must have the same lenght as items"
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1354 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1355 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1356 async with session.begin():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1357 for idx, item in enumerate(items):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1358 parsed = parsed_items[idx] if parsed_items else None
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1359 stmt = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1360 insert(PubsubItem)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1361 .values(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1362 node_id=node.id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1363 name=item["id"],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1364 data=item,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1365 parsed=parsed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1366 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1367 .on_conflict_do_update(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1368 index_elements=(PubsubItem.node_id, PubsubItem.name),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1369 set_={
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1370 PubsubItem.data: item,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1371 PubsubItem.parsed: parsed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1372 PubsubItem.updated: now(),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1373 },
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1374 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1375 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1376 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1377 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1378
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1379 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1380 async def delete_pubsub_items(
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1381 self, node: PubsubNode, items_names: Optional[List[str]] = None
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1382 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1383 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1384
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1385 @param node: node from which items must be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1386 @param items_names: names of items to delete
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1387 if None, ALL items will be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1388 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1389 stmt = delete(PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1390 if node is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1391 if isinstance(node, list):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1392 stmt = stmt.where(PubsubItem.node_id.in_([n.id for n in node]))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1393 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1394 stmt = stmt.filter_by(node_id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1395 if items_names is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1396 stmt = stmt.where(PubsubItem.name.in_(items_names))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1397 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1398 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1399 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1400
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1401 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1402 async def purge_pubsub_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1403 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1404 services: Optional[List[jid.JID]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1405 names: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1406 types: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1407 subtypes: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1408 profiles: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1409 created_before: Optional[datetime] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1410 updated_before: Optional[datetime] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1411 ) -> None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1412 """Delete items cached for a node
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1413
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1414 @param node: node from which items must be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1415 @param items_names: names of items to delete
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1416 if None, ALL items will be deleted
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1417 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1418 stmt = delete(PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1419 node_fields = {
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1420 "service": services,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1421 "name": names,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1422 "type_": types,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1423 "subtype": subtypes,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1424 }
3744 658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1425 if profiles is not None:
658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1426 node_fields["profile_id"] = [self.profiles[p] for p in profiles]
658ddbabaf36 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions: Goffi <goffi@goffi.org> parents: 3720 diff changeset	1427
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1428 if any(x is not None for x in node_fields.values()):
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1429 sub_q = select(PubsubNode.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1430 for col, values in node_fields.items():
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1431 if values is None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1432 continue
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1433 sub_q = sub_q.where(getattr(PubsubNode, col).in_(values))
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1434 stmt = stmt.where(PubsubItem.node_id.in_(sub_q)).execution_options(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1435 synchronize_session=False
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1436 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1437
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1438 if created_before is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1439 stmt = stmt.where(PubsubItem.created < created_before)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1440
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1441 if updated_before is not None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1442 stmt = stmt.where(PubsubItem.updated < updated_before)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1443
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1444 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1445 await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1446 await session.commit()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1447
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1448 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1449 async def get_items(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1450 self,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1451 node: PubsubNode,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1452 max_items: Optional[int] = None,
3753 10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1453 item_ids: Optional[list[str]] = None,
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1454 before: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1455 after: Optional[str] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1456 from_index: Optional[int] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1457 order_by: Optional[List[str]] = None,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1458 desc: bool = True,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1459 force_rsm: bool = False,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1460 ) -> Tuple[List[PubsubItem], dict]:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1461 """Get Pubsub Items from cache
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1462
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1463 @param node: retrieve items from this node (must be synchronised)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1464 @param max_items: maximum number of items to retrieve
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1465 @param before: get items which are before the item with this name in given order
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1466 empty string is not managed here, use desc order to reproduce RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1467 behaviour.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1468 @param after: get items which are after the item with this name in given order
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1469 @param from_index: get items with item index (as defined in RSM spec)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1470 starting from this number
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1471 @param order_by: sorting order of items (one of C.ORDER_BY_*)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1472 @param desc: direction or ordering
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1473 @param force_rsm: if True, force the use of RSM worklow.
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1474 RSM workflow is automatically used if any of before, after or
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1475 from_index is used, but if only RSM max_items is used, it won't be
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1476 used by default. This parameter let's use RSM workflow in this
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1477 case. Note that in addition to RSM metadata, the result will not be
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1478 the same (max_items without RSM will returns most recent items,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1479 i.e. last items in modification order, while max_items with RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1480 will return the oldest ones (i.e. first items in modification
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1481 order).
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1482 to be used when max_items is used from RSM
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1483 """
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1484
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1485 metadata = {
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1486 "service": node.service,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1487 "node": node.name,
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1488 "uri": uri.build_xmpp_uri(
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1489 "pubsub",
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1490 path=node.service.full(),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1491 node=node.name,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1492 ),
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1493 }
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1494 if max_items is None:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1495 max_items = 20
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1496
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1497 use_rsm = any((before, after, from_index is not None))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1498 if force_rsm and not use_rsm:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1499 #
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1500 use_rsm = True
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1501 from_index = 0
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1502
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1503 stmt = select(PubsubItem).filter_by(node_id=node.id).limit(max_items)
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1504
3753 10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1505 if item_ids is not None:
10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1506 stmt = stmt.where(PubsubItem.name.in_(item_ids))
10b71e3526bd core (memory/sqla): add attribute to filter on `item_ids` in `getItems` Goffi <goffi@goffi.org> parents: 3744 diff changeset	1507
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1508 if not order_by:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1509 order_by = [C.ORDER_BY_MODIFICATION]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1510
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1511 order = []
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1512 for order_type in order_by:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1513 if order_type == C.ORDER_BY_MODIFICATION:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1514 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1515 order.extend((PubsubItem.updated.desc(), PubsubItem.id.desc()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1516 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1517 order.extend((PubsubItem.updated.asc(), PubsubItem.id.asc()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1518 elif order_type == C.ORDER_BY_CREATION:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1519 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1520 order.append(PubsubItem.id.desc())
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1521 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1522 order.append(PubsubItem.id.asc())
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1523 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1524 raise exceptions.InternalError(f"Unknown order type {order_type!r}")
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1525
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1526 stmt = stmt.order_by(*order)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1527
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1528 if use_rsm:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1529 # CTE to have result row numbers
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1530 row_num_q = select(
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1531 PubsubItem.id,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1532 PubsubItem.name,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1533 # row_number starts from 1, but RSM index must start from 0
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1534 (func.row_number().over(order_by=order) - 1).label("item_index"),
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1535 ).filter_by(node_id=node.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1536
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1537 row_num_cte = row_num_q.cte()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1538
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1539 if max_items > 0:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1540 # as we can't simply use PubsubItem.id when we order by modification,
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1541 # we need to use row number
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1542 item_name = before or after
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1543 row_num_limit_q = (
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1544 select(row_num_cte.c.item_index).where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1545 row_num_cte.c.name == item_name
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1546 )
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1547 ).scalar_subquery()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1548
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1549 stmt = (
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1550 select(row_num_cte.c.item_index, PubsubItem)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1551 .join(row_num_cte, PubsubItem.id == row_num_cte.c.id)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1552 .limit(max_items)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1553 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1554 if before:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1555 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1556 row_num_cte.c.item_index < row_num_limit_q
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1557 ).order_by(row_num_cte.c.item_index.desc())
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1558 elif after:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1559 stmt = stmt.where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1560 row_num_cte.c.item_index > row_num_limit_q
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1561 ).order_by(row_num_cte.c.item_index.asc())
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1562 else:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1563 stmt = stmt.where(row_num_cte.c.item_index >= from_index).order_by(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1564 row_num_cte.c.item_index.asc()
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1565 )
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1566 # from_index is used
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1567
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1568 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1569 if max_items == 0:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1570 items = result = []
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1571 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1572 result = await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1573 result = result.all()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1574 if before:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1575 result.reverse()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1576 items = [row[-1] for row in result]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1577 rows_count = (
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1578 await session.execute(row_num_q.with_only_columns(count()))
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1579 ).scalar_one()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1580
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1581 try:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1582 index = result[0][0]
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1583 except IndexError:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1584 index = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1585
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1586 try:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1587 first = result[0][1].name
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1588 except IndexError:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1589 first = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1590 last = None
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1591 else:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1592 last = result[-1][1].name
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1593
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1594 metadata["rsm"] = {
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1595 k: v
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1596 for k, v in {
3719 cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1597 "index": index,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1598 "count": rows_count,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1599 "first": first,
cf930bb282ac memory (sqla): don't set keys with None as value for RSM in `getItems` Goffi <goffi@goffi.org> parents: 3673 diff changeset	1600 "last": last,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1601 }.items()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1602 if v is not None
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1603 }
3720 40a6374fcd44 memory (sqla): fix `complete` setting when `index` is `None` in `getItems` Goffi <goffi@goffi.org> parents: 3719 diff changeset	1604 metadata["complete"] = (index or 0) + len(result) == rows_count
3595 7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1605
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1606 return items, metadata
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1607
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1608 async with self.session() as session:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1609 result = await session.execute(stmt)
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1610
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1611 result = result.scalars().all()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1612 if desc:
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1613 result.reverse()
7510648e8e3a core (memory/sqla): methods to manipulate pubsub tables Goffi <goffi@goffi.org> parents: 3583 diff changeset	1614 return result, metadata
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1615
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1616 def _get_sqlite_path(self, path: List[Union[str, int]]) -> str:
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1617 """generate path suitable to query JSON element with SQLite"""
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1618 return f"${''.join(f'[{p}]' if isinstance(p, int) else f'.{p}' for p in path)}"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1619
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1620 @aio
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1621 async def search_pubsub_items(
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1622 self,
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1623 query: dict,
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1624 ) -> Tuple[List[PubsubItem]]:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1625 """Search for pubsub items in cache
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1626
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1627 @param query: search terms. Keys can be:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1628 :fts (str):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1629 Full-Text Search query. Currently SQLite FT5 engine is used, its query
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1630 syntax can be used, see `FTS5 Query documentation
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1631 <https://sqlite.org/fts5.html#full_text_query_syntax>`_
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1632 :profiles (list[str]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1633 filter on nodes linked to those profiles
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1634 :nodes (list[str]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1635 filter on nodes with those names
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1636 :services (list[jid.JID]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1637 filter on nodes from those services
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1638 :types (list[str\|None]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1639 filter on nodes with those types. None can be used to filter on nodes with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1640 no type set
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1641 :subtypes (list[str\|None]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1642 filter on nodes with those subtypes. None can be used to filter on nodes with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1643 no subtype set
3754 af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1644 :names (list[str]):
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1645 filter on items with those names
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1646 :parsed (list[dict]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1647 Filter on a parsed data field. The dict must contain 3 keys: ``path``
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1648 which is a list of str or int giving the path to the field of interest
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1649 (str for a dict key, int for a list index), ``operator`` with indicate the
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1650 operator to use to check the condition, and ``value`` which depends of
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1651 field type and operator.
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1652
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1653 See documentation for details on operators (it's currently explained at
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1654 ``doc/libervia-cli/pubsub_cache.rst`` in ``search`` command
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1655 documentation).
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1656
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1657 :order-by (list[dict]):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1658 Indicates how to order results. The dict can contain either a ``order``
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1659 for a well-know order or a ``path`` for a parsed data field path
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1660 (``order`` and ``path`` can't be used at the same time), an an optional
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1661 ``direction`` which can be ``asc`` or ``desc``. See documentation for
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1662 details on well-known orders (it's currently explained at
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1663 ``doc/libervia-cli/pubsub_cache.rst`` in ``search`` command
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1664 documentation).
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1665
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1666 :index (int):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1667 starting index of items to return from the query result. It's translated
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1668 to SQL's OFFSET
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1669
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1670 :limit (int):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1671 maximum number of items to return. It's translated to SQL's LIMIT.
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1672
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1673 @result: found items (the ``node`` attribute will be filled with suitable
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1674 PubsubNode)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1675 """
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1676 # TODO: FTS and parsed data filters use SQLite specific syntax
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1677 # when other DB engines will be used, this will have to be adapted
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1678 stmt = select(PubsubItem)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1679
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1680 # Full-Text Search
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1681 fts = query.get("fts")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1682 if fts:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1683 fts_select = (
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1684 text("SELECT rowid, rank FROM pubsub_items_fts(:fts_query)")
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1685 .bindparams(fts_query=fts)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1686 .columns(rowid=Integer)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1687 .subquery()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1688 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1689 stmt = stmt.select_from(fts_select).outerjoin(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1690 PubsubItem, fts_select.c.rowid == PubsubItem.id
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1691 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1692
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1693 # node related filters
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1694 profiles = query.get("profiles")
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1695 if profiles or any(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1696 query.get(k) for k in ("nodes", "services", "types", "subtypes")
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1697 ):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1698 stmt = stmt.join(PubsubNode).options(contains_eager(PubsubItem.node))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1699 if profiles:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1700 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1701 stmt = stmt.where(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1702 PubsubNode.profile_id.in_(self.profiles[p] for p in profiles)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1703 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1704 except KeyError as e:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1705 raise exceptions.ProfileUnknownError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1706 f"This profile doesn't exist: {e.args[0]!r}"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1707 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1708 for key, attr in (
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1709 ("nodes", "name"),
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1710 ("services", "service"),
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1711 ("types", "type_"),
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1712 ("subtypes", "subtype"),
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1713 ):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1714 value = query.get(key)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1715 if not value:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1716 continue
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1717 if key in ("types", "subtypes") and None in value:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1718 # NULL can't be used with SQL's IN, so we have to add a condition with
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1719 # IS NULL, and use a OR if there are other values to check
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1720 value.remove(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1721 condition = getattr(PubsubNode, attr).is_(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1722 if value:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1723 condition = or_(getattr(PubsubNode, attr).in_(value), condition)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1724 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1725 condition = getattr(PubsubNode, attr).in_(value)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1726 stmt = stmt.where(condition)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1727 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1728 stmt = stmt.options(selectinload(PubsubItem.node))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1729
3754 af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1730 # names
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1731 names = query.get("names")
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1732 if names:
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1733 stmt = stmt.where(PubsubItem.name.in_(names))
af485e8afe03 core (memory/sqla): `searchPubsubItems` can now filter on item names Goffi <goffi@goffi.org> parents: 3753 diff changeset	1734
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1735 # parsed data filters
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1736 parsed = query.get("parsed", [])
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1737 for filter_ in parsed:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1738 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1739 path = filter_["path"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1740 operator = filter_["op"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1741 value = filter_["value"]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1742 except KeyError as e:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1743 raise ValueError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1744 f'missing mandatory key {e.args[0]!r} in "parsed" filter'
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1745 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1746 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1747 op_attr = OP_MAP[operator]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1748 except KeyError:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1749 raise ValueError(f"invalid operator: {operator!r}")
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1750 sqlite_path = self._get_sqlite_path(path)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1751 if operator in ("overlap", "ioverlap", "disjoint", "idisjoint"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1752 col = literal_column("json_each.value")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1753 if operator[0] == "i":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1754 col = func.lower(col)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1755 value = [str(v).lower() for v in value]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1756 condition = (
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1757 select(1)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1758 .select_from(func.json_each(PubsubItem.parsed, sqlite_path))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1759 .where(col.in_(value))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1760 ).scalar_subquery()
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1761 if operator in ("disjoint", "idisjoint"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1762 condition = condition.is_(None)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1763 stmt = stmt.where(condition)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1764 elif operator == "between":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1765 try:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1766 left, right = value
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1767 except (ValueError, TypeError):
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1768 raise ValueError(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1769 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1770 'invalid value for "between" filter, you must use a 2 items '
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1771 "array: {value!r}"
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1772 ).format(value=value)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1773 )
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1774 col = func.json_extract(PubsubItem.parsed, sqlite_path)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1775 stmt = stmt.where(col.between(left, right))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1776 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1777 # we use func.json_extract instead of generic JSON way because SQLAlchemy
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1778 # add a JSON_QUOTE to the value, and we want SQL value
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1779 col = func.json_extract(PubsubItem.parsed, sqlite_path)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1780 stmt = stmt.where(getattr(col, op_attr)(value))
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1781
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1782 # order
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1783 order_by = query.get("order-by") or [{"order": "creation"}]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1784
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1785 for order_data in order_by:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1786 order, path = order_data.get("order"), order_data.get("path")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1787 if order and path:
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1788 raise ValueError(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1789 _(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1790 '"order" and "path" can\'t be used at the same time in '
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1791 '"order-by" data'
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1792 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1793 )
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1794 if order:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1795 if order == "creation":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1796 col = PubsubItem.id
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1797 elif order == "modification":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1798 col = PubsubItem.updated
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1799 elif order == "item_id":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1800 col = PubsubItem.name
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1801 elif order == "rank":
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1802 if not fts:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1803 raise ValueError(
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1804 "'rank' order can only be used with Full-Text Search (fts)"
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1805 )
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1806 col = literal_column("rank")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1807 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1808 raise NotImplementedError(f"Unknown {order!r} order")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1809 else:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1810 # we have a JSON path
4037 524856bd7b19 massive refactoring to switch from camelCase to snake_case: Goffi <goffi@goffi.org> parents: 4003 diff changeset	1811 # sqlite_path = self._get_sqlite_path(path)
3664 9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1812 col = PubsubItem.parsed[path]
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1813 direction = order_data.get("direction", "ASC").lower()
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1814 if not direction in ("asc", "desc"):
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1815 raise ValueError(f"Invalid order-by direction: {direction!r}")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1816 stmt = stmt.order_by(getattr(col, direction)())
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1817
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1818 # offset, limit
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1819 index = query.get("index")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1820 if index:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1821 stmt = stmt.offset(index)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1822 limit = query.get("limit")
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1823 if limit:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1824 stmt = stmt.limit(limit)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1825
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1826 async with self.session() as session:
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1827 result = await session.execute(stmt)
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1828
9ae6ec74face memory (sqla): implement `searchPubsubItems`: Goffi <goffi@goffi.org> parents: 3638 diff changeset	1829 return result.scalars().all()
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1830
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1831 # Notifications
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1832
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1833 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1834 async def add_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1835 self,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1836 client: Optional[SatXMPPEntity],
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1837 type_: NotificationType,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1838 body_plain: str,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1839 body_rich: Optional[str] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1840 title: Optional[str] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1841 requires_action: bool = False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1842 priority: NotificationPriority = NotificationPriority.MEDIUM,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1843 expire_at: Optional[float] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1844 extra: Optional[dict] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1845 ) -> Notification:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1846 """Add a new notification to the DB.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1847
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1848 @param client: client associated with the notification. If None, the notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1849 will be global.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1850 @param type_: type of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1851 @param body_plain: plain text body.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1852 @param body_rich: rich text (XHTML) body.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1853 @param title: optional title.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1854 @param requires_action: True if the notification requires user action (e.g. a
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1855 dialog need to be answered).
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1856 @priority: how urgent the notification is
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1857 @param expire_at: expiration timestamp for the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1858 @param extra: additional data.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1859 @return: created Notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1860 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1861 profile_id = self.profiles[client.profile] if client else None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1862 notification = Notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1863 profile_id=profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1864 type=type_,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1865 body_plain=body_plain,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1866 body_rich=body_rich,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1867 requires_action=requires_action,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1868 priority=priority,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1869 expire_at=expire_at,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1870 title=title,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1871 extra_data=extra,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1872 status=NotificationStatus.new,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1873 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1874 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1875 async with session.begin():
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1876 session.add(notification)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1877 return notification
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1878
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1879 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1880 async def update_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1881 self, client: SatXMPPEntity, notification_id: int, **kwargs
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1882 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1883 """Update an existing notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1884
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1885 @param client: client associated with the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1886 @param notification_id: ID of the notification to update.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1887 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1888 profile_id = self.profiles[client.profile]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1889 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1890 await session.execute(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1891 update(Notification)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1892 .where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1893 and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1894 Notification.profile_id == profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1895 Notification.id == notification_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1896 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1897 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1898 .values(**kwargs)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1899 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1900 await session.commit()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1901
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1902 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1903 async def get_notifications(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1904 self,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1905 client: SatXMPPEntity,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1906 type_: Optional[NotificationType] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1907 status: Optional[NotificationStatus] = None,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1908 requires_action: Optional[bool] = None,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	1909 min_priority: Optional[int] = None,
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1910 ) -> List[Notification]:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1911 """Retrieve all notifications for a given profile with optional filters.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1912
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1913 @param client: client associated with the notifications.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1914 @param type_: filter by type of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1915 @param status: filter by status of the notification.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1916 @param requires_action: filter by notifications that require user action.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1917 @param min_priority: filter by minimum priority value.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1918 @return: list of matching Notification instances.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1919 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1920 profile_id = self.profiles[client.profile]
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	1921 filters = [
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	1922 or_(Notification.profile_id == profile_id, Notification.profile_id.is_(None))
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	1923 ]
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1924
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1925 if type_:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1926 filters.append(Notification.type == type_)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1927 if status:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1928 filters.append(Notification.status == status)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1929 if requires_action is not None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1930 filters.append(Notification.requires_action == requires_action)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1931 if min_priority:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1932 filters.append(Notification.priority >= min_priority)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1933
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1934 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1935 result = await session.execute(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4212 diff changeset	1936 select(Notification).where(and_(*filters)).order_by(Notification.id)
4130 02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1937 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1938 return result.scalars().all()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1939
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1940 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1941 async def delete_notification(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1942 self, client: Optional[SatXMPPEntity], notification_id: str
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1943 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1944 """Delete a notification by its profile and id.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1945
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1946 @param client: client associated with the notification. If None, profile_id will be NULL.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1947 @param notification_id: ID of the notification to delete.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1948 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1949 profile_id = self.profiles[client.profile] if client else None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1950 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1951 await session.execute(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1952 delete(Notification).where(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1953 and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1954 Notification.profile_id == profile_id,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1955 Notification.id == int(notification_id),
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1956 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1957 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1958 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1959 await session.commit()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1960
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1961 @aio
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1962 async def clean_expired_notifications(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1963 self, client: Optional[SatXMPPEntity], limit_timestamp: Optional[float] = None
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1964 ) -> None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1965 """Cleans expired notifications and older profile-specific notifications.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1966
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1967 - Removes all notifications where the expiration timestamp has passed,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1968 irrespective of their profile.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1969 - If a limit_timestamp is provided, removes older notifications with a profile set
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1970 (i.e., not global notifications) that do not require user action. If client is
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1971 provided, only remove notification for this profile.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1972
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1973 @param client: if provided, only expire notification for this client (in addition
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1974 to truly expired notifications for everybody).
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1975 @param limit_timestamp: Timestamp limit for older notifications. If None, only
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1976 truly expired notifications are removed.
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1977 """
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1978
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1979 # Delete truly expired notifications
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1980 expired_condition = Notification.expire_at < time.time()
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1981
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1982 # Delete older profile-specific notifications (created before the limit_timestamp)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1983 if client is None:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1984 profile_condition = Notification.profile_id.isnot(None)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1985 else:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1986 profile_condition = Notification.profile_id == self.profiles[client.profile]
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1987 older_condition = and_(
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1988 profile_condition,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1989 Notification.timestamp < limit_timestamp if limit_timestamp else False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1990 Notification.requires_action == False,
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1991 )
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1992
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1993 # Combine the conditions
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1994 conditions = or_(expired_condition, older_condition)
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1995
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1996 async with self.session() as session:
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1997 await session.execute(delete(Notification).where(conditions))
02f0adc745c6 core: notifications implementation, first draft: Goffi <goffi@goffi.org> parents: 4092 diff changeset	1998 await session.commit()

Mercurial > libervia-backend

annotate libervia/backend/memory/sqla.py @ 4387:a6270030968d default tip