Mercurial > libervia-backend

annotate src/tools/common/data_format.py @ 2115:3435e8d2e8e4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
quick frontends (chat): send actual value instead of None on status and state updates
author Goffi <goffi@goffi.org>
date Mon, 09 Jan 2017 23:10:16 +0100
parents c8e561a5b2b6
children f67434fd88d2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1919
diff changeset
1 #!/usr/bin/env python2
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
3
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # SAT: a jabber client
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1660
diff changeset
5 # Copyright (C) 2009-2016 Jérôme Poisson (goffi@goffi.org)
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
6
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
10 # (at your option) any later version.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
11
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
16
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
19
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
20 """ tools common to backend and frontends """
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
21
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from sat.core import exceptions
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
23
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
24
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
25 def dict2iter(name, dict_, pop=False):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
26 """iterate into a list serialised in a dict
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
27
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
28 name is the name of the key.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
29 Serialisation is done with [name] [name#1] [name#2] and so on
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
30 e.g.: if name is 'group', keys are group, group#1, group#2, ...
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
31 iteration stop at first missing increment
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
32 Empty values are possible
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
33 @param name(unicode): name of the key
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
34 @param dict_(dict): dictionary with the serialised list
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
35 @param pop(bool): if True, remove the value from dict
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
36 @return iter: iterate through the deserialised list
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
37 """
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
38 if pop:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
39 get=lambda d,k: d.pop(k)
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
40 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
41 get=lambda d,k: d[k]
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
42
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
43 try:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
44 yield get(dict_,name)
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
45 except KeyError:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
46 return
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
47 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
48 idx = 1
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
49 while True:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
50 try:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
51 yield get(dict_,u'{}#{}'.format(name, idx))
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
52 except KeyError:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
53 return
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
54 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
55 idx += 1
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
56
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
57 def iter2dict(name, iter_, dict_=None, check_conflict=True):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
58 """Fill a dict with values from an iterable
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
59
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
60 name is used to serialise iter_, in the same way as in [dict2iter]
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
61 Build from the tags a dict using the microblog data format.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
62
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
63 @param name(unicode): key to use for serialisation
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
64 e.g. "group" to have keys "group", "group#1", "group#2", ...
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
65 @param iter_(iterable): values to store
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
66 @param dict_(None, dict): dictionary to fill, or None to create one
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
67 @param check_conflict(bool): if True, raise an exception in case of existing key
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
68 @return (dict): filled dict, or newly created one
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
69 @raise exceptions.ConflictError: a needed key already exists
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
70 """
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
71 if dict_ is None:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
72 dict_ = {}
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
73 for idx, value in enumerate(iter_):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
74 if idx == 0:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
75 key = name
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
76 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
77 key = u'{}#{}'.format(name, idx)
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
78 if check_conflict and key in dict_:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
79 raise exceptions.ConflictError
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
80 dict_[key] = value
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
81 return dict
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
82
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
83 def getSubDict(name, dict_, sep=u'_'):
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
84 """get a sub dictionary from a serialised dictionary
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
85
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
86 look for keys starting with name, and create a dict with it
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
87 eg.: if "key" is looked for, {'html': 1, 'key_toto': 2, 'key_titi': 3} will return:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
88 {None: 1, toto: 2, titi: 3}
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
89 @param name(unicode): name of the key
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
90 @param dict_(dict): dictionary with the serialised list
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
91 @param sep(unicode): separator used between name and subkey
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
92 @return iter: iterate through the deserialised items
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
93 """
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
94 for k,v in dict_.iteritems():
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
95 if k.startswith(name):
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
96 if k == name:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
97 yield None, v
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
98 else:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
99 if k[len(name)] != sep:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
100 continue
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
101 else:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
102 yield k[len(name)+1:], v