annotate sat/memory/persistent.py @ 2661:661f66d41215

core (xmpp): send initial presence only after all profileConnected have been treated: presence is now sent after profileConnected methods are done, this avoid to have to deal with synchronisation in connection event. For instance, PEP events should not be sent before presence is sent, so profileConnected methods can assume PEP events are not done yet, and do needed initialisation using async method if necessary. This has been done to avoid overcomplicated synchronisation in XEP-0384 plugin (network calls are needed to initialise session, but PEP events need an initialised session to be treated).
author Goffi <goffi@goffi.org>
date Sat, 11 Aug 2018 18:24:55 +0200
parents e107089d6640
children 9702025f6dff
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1766
diff changeset
1 #!/usr/bin/env python2
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
3
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
4 # SAT: a jabber client
2483
0046283a285d dates update
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
5 # Copyright (C) 2009-2018 Jérôme Poisson (goffi@goffi.org)
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
6
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
7 # This program is free software: you can redistribute it and/or modify
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
10 # (at your option) any later version.
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
11
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
12 # This program is distributed in the hope that it will be useful,
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
15 # GNU Affero General Public License for more details.
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
16
609
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
84a6e83157c2 fixed licences in docstrings (they are now in comments)
Goffi <goffi@goffi.org>
parents: 592
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
19
771
bfabeedbf32e core: i18n refactoring:
Goffi <goffi@goffi.org>
parents: 609
diff changeset
20 from sat.core.i18n import _
993
301b342c697a core: use of the new core.log module:
Goffi <goffi@goffi.org>
parents: 979
diff changeset
21 from sat.core.log import getLogger
301b342c697a core: use of the new core.log module:
Goffi <goffi@goffi.org>
parents: 979
diff changeset
22 log = getLogger(__name__)
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
23 from twisted.python import failure
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
24
592
e5a875a3311b Fix pep8 support in src/memory.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 588
diff changeset
25
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
26 class MemoryNotInitializedError(Exception):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
27 pass
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
28
592
e5a875a3311b Fix pep8 support in src/memory.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 588
diff changeset
29
588
beaf6bec2fcd Remove every old-style class.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 587
diff changeset
30 class PersistentDict(object):
592
e5a875a3311b Fix pep8 support in src/memory.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 588
diff changeset
31 r"""A dictionary which save persistently each value assigned
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
32
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
33 /!\ be careful, each assignment means a database write
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
34 /!\ Memory must be initialised before loading/setting value with instances of this class"""
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
35 storage = None
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
36 binary = False
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
37
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
38 def __init__(self, namespace, profile=None):
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
39 """
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
40
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
41 @param namespace: unique namespace for this dictionary
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
42 @param profile(unicode, None): profile which *MUST* exists, or None for general values
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
43 """
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
44 if not self.storage:
993
301b342c697a core: use of the new core.log module:
Goffi <goffi@goffi.org>
parents: 979
diff changeset
45 log.error(_("PersistentDict can't be used before memory initialisation"))
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
46 raise MemoryNotInitializedError
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
47 self._cache = None
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
48 self.namespace = namespace
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
49 self.profile = profile
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
50
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
51 def _setCache(self, data):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
52 self._cache = data
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
53
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
54 def load(self):
1029
f6182f6418ea memory: add class ProfileSessions based on Sessions
souliane <souliane@mailoo.org>
parents: 993
diff changeset
55 """Load persistent data from storage.
f6182f6418ea memory: add class ProfileSessions based on Sessions
souliane <souliane@mailoo.org>
parents: 993
diff changeset
56
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
57 need to be called before any other operation
1029
f6182f6418ea memory: add class ProfileSessions based on Sessions
souliane <souliane@mailoo.org>
parents: 993
diff changeset
58 @return: defers the PersistentDict instance itself
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
59 """
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
60 d = self.storage.getPrivates(self.namespace, binary=self.binary, profile=self.profile)
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
61 d.addCallback(self._setCache)
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
62 d.addCallback(lambda dummy: self)
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
63 return d
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
64
1292
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
65 def iteritems(self):
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
66 return self._cache.iteritems()
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
67
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
68 def items(self):
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
69 return self._cache.items()
b29a065a66f0 core: added items() and iteritems() methods to PersistentDict
Goffi <goffi@goffi.org>
parents: 1029
diff changeset
70
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
71 def __repr__(self):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
72 return self._cache.__repr__()
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
73
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
74 def __str__(self):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
75 return self._cache.__str__()
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
76
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
77 def __lt__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
78 return self._cache.__lt__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
79
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
80 def __le__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
81 return self._cache.__le__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
82
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
83 def __eq__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
84 return self._cache.__eq__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
85
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
86 def __ne__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
87 return self._cache.__ne__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
88
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
89 def __gt__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
90 return self._cache.__gt__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
91
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
92 def __ge__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
93 return self._cache.__ge__(other)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
94
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
95 def __cmp__(self, other):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
96 return self._cache.__cmp__(other)
587
952322b1d490 Remove trailing whitespaces.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 585
diff changeset
97
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
98 def __hash__(self):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
99 return self._cache.__hash__()
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
100
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
101 def __nonzero__(self):
439
866dbb0d7d87 plugin maildir: maildir now use PersistentBinaryDictionary to store profile specific data
Goffi <goffi@goffi.org>
parents: 432
diff changeset
102 return self._cache.__len__()
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
103
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
104 def __contains__(self, key):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
105 return self._cache.__contains__(key)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
106
979
8f8bdd31375a core (memory): added __iter__ to PersistentDict
Goffi <goffi@goffi.org>
parents: 811
diff changeset
107 def __iter__(self):
8f8bdd31375a core (memory): added __iter__ to PersistentDict
Goffi <goffi@goffi.org>
parents: 811
diff changeset
108 return self._cache.__iter__()
8f8bdd31375a core (memory): added __iter__ to PersistentDict
Goffi <goffi@goffi.org>
parents: 811
diff changeset
109
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
110 def __getitem__(self, key):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
111 return self._cache.__getitem__(key)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
112
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
113 def __setitem__(self, key, value):
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
114 self.storage.setPrivateValue(self.namespace, key, value, self.binary, self.profile)
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
115 return self._cache.__setitem__(key, value)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
116
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
117 def __delitem__(self, key):
2302
f824b6026b73 core (memory/persistent): fixed calls to delPrivateValue
Goffi <goffi@goffi.org>
parents: 2209
diff changeset
118 self.storage.delPrivateValue(self.namespace, key, self.binary, self.profile)
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
119 return self._cache.__delitem__(key)
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
120
442
ab9ddf6c7eaf core: added get method to PersistentDictionary
Goffi <goffi@goffi.org>
parents: 439
diff changeset
121 def get(self, key, default=None):
ab9ddf6c7eaf core: added get method to PersistentDictionary
Goffi <goffi@goffi.org>
parents: 439
diff changeset
122 return self._cache.get(key, default)
ab9ddf6c7eaf core: added get method to PersistentDictionary
Goffi <goffi@goffi.org>
parents: 439
diff changeset
123
1340
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
124 def setdefault(self, key, default):
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
125 try:
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
126 return self._cache[key]
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
127 except:
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
128 self.__setitem__(key, default)
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
129 return default
91e72da1d093 core (persistent dict): added setdefault method
Goffi <goffi@goffi.org>
parents: 1292
diff changeset
130
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
131 def force(self, name):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
132 """Force saving of an attribute to storage
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
133
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
134 @return: deferred fired when data is actually saved
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
135 """
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
136 return self.storage.setPrivateValue(self.namespace, name, self._cache[name], self.binary, self.profile)
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
137
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
138
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
139 class PersistentBinaryDict(PersistentDict):
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
140 """Persistent dict where value can be any python data (instead of string only)"""
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
141 binary = True
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
142
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
143
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
144 class LazyPersistentBinaryDict(PersistentBinaryDict):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
145 ur"""PersistentBinaryDict which get key/value when needed
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
146
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
147 This Persistent need more database access, it is suitable for largest data,
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
148 to save memory.
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
149 /!\ most of methods return a Deferred
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
150 """
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
151 # TODO: missing methods should be implemented using database access
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
152 # TODO: a cache would be useful (which is deleted after a timeout)
587
952322b1d490 Remove trailing whitespaces.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 585
diff changeset
153
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
154 def load(self):
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
155 # we show a warning as calling load on LazyPersistentBinaryDict sounds like a code mistake
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
156 log.warning(_(u"Calling load on LazyPersistentBinaryDict while it's not needed"))
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
157
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
158 def iteritems(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
159 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
160
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
161 def items(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
162 return self.storage.getPrivates(self.namespace, binary=self.binary, profile=self.profile)
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
163
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
164 def __repr__(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
165 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
166
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
167 def __str__(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
168 return "LazyPersistentBinaryDict (namespace: {})".format(self.namespace)
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
169
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
170 def __lt__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
171 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
172
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
173 def __le__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
174 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
175
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
176 def __eq__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
177 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
178
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
179 def __ne__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
180 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
181
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
182 def __gt__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
183 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
184
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
185 def __ge__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
186 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
187
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
188 def __cmp__(self, other):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
189 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
190
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
191 def __hash__(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
192 return hash(unicode(self.__class__) + self.namespace + (self.profile or u''))
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
193
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
194 def __nonzero__(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
195 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
196
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
197 def __contains__(self, key):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
198 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
199
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
200 def __iter__(self):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
201 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
202
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
203 def _data2value(self, data, key):
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
204 try:
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
205 return data[key]
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
206 except KeyError as e:
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
207 # we return a Failure here to avoid the jump
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
208 # into debugger in debug mode.
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
209 raise failure.Failure(e)
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
210
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
211 def __getitem__(self, key):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
212 """get the value as a Deferred"""
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
213 d = self.storage.getPrivates(self.namespace, keys=[key], binary=self.binary,
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
214 profile=self.profile)
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
215 d.addCallback(self._data2value, key)
2209
ea41cf1e6d29 memory (persistent, sqlite): fixed getPrivates bug with keys arguments and its use in LazyPersistentBinaryDict
Goffi <goffi@goffi.org>
parents: 2182
diff changeset
216 return d
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
217
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
218 def __setitem__(self, key, value):
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
219 self.storage.setPrivateValue(self.namespace, key, value, self.binary,
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
220 self.profile)
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
221
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
222 def __delitem__(self, key):
2302
f824b6026b73 core (memory/persistent): fixed calls to delPrivateValue
Goffi <goffi@goffi.org>
parents: 2209
diff changeset
223 self.storage.delPrivateValue(self.namespace, key, self.binary, self.profile)
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
224
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
225 def _defaultOrException(self, failure_, default):
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
226 failure_.trap(KeyError)
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
227 return default
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
228
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
229 def get(self, key, default=None):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
230 d = self.__getitem__(key)
2644
e107089d6640 core (memory/persistent): better default value handling in LazyPersistentBinaryDict (don't jump to debugger in debug mode if value is not found)
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
231 d.addErrback(self._defaultOrException, default=default)
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
232 return d
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
233
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
234 def setdefault(self, key, default):
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
235 raise NotImplementedError
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
236
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
237 def force(self, name, value):
432
31e8c48b5f5d core: - memory refactoring (moved memory.py and sqlite.py from tools to memory)
Goffi <goffi@goffi.org>
parents:
diff changeset
238 """Force saving of an attribute to storage
2182
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
239
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
240 @param value(object): value is needed for LazyPersistentBinaryDict
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
241 @return: deferred fired when data is actually saved
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
242 """
087eec4c6c07 memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
243 return self.storage.setPrivateValue(self.namespace, name, value, self.binary, self.profile)