Mercurial > libervia-backend

annotate sat/tools/common/data_format.py @ 2721:4aaa47f62d8d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
core (memory/sqlite): fixed v7 update performance issue: Performance issue was due to bad ordering in table dropping, this has been fixed. "infos" message are also deleted by this update as they are containing only presence data and take a lot of space for a barely useful data. A config option may be available in the future to store presence data in logs.
author Goffi <goffi@goffi.org>
date Tue, 11 Dec 2018 23:53:27 +0100
parents 46f2733a2a9b
children 003b8b4b56a7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1919
diff changeset
1 #!/usr/bin/env python2
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
3
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # SAT: a jabber client
2483
0046283a285d dates update
Goffi <goffi@goffi.org>
parents: 2414
diff changeset
5 # Copyright (C) 2009-2018 Jérôme Poisson (goffi@goffi.org)
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
6
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
10 # (at your option) any later version.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
11
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
16
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
19
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
20 """ tools common to backend and frontends """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
21 #  FIXME: json may be more appropriate than manual serialising like done here
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
22
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from sat.core import exceptions
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
24 import json
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
25
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
26
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
27 def dict2iter(name, dict_, pop=False):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
28 """iterate into a list serialised in a dict
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
29
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
30 name is the name of the key.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
31 Serialisation is done with [name] [name#1] [name#2] and so on
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
32 e.g.: if name is 'group', keys are group, group#1, group#2, ...
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
33 iteration stop at first missing increment
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
34 Empty values are possible
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
35 @param name(unicode): name of the key
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
36 @param dict_(dict): dictionary with the serialised list
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
37 @param pop(bool): if True, remove the value from dict
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
38 @return iter: iterate through the deserialised list
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
39 """
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
40 if pop:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
41 get = lambda d, k: d.pop(k)
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
42 else:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
43 get = lambda d, k: d[k]
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
44
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
45 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
46 yield get(dict_, name)
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
47 except KeyError:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
48 return
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
49 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
50 idx = 1
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
51 while True:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
52 try:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
53 yield get(dict_, u"{}#{}".format(name, idx))
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
54 except KeyError:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
55 return
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
56 else:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
57 idx += 1
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
58
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
59
2153
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
60 def dict2iterdict(name, dict_, extra_keys, pop=False):
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
61 """like dict2iter but yield dictionaries
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
62
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
63 params are like in [dict2iter], extra_keys is used for extra dict keys.
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
64 e.g. dict2iterdict(comments, mb_data, ('node', 'service')) will yield dicts like:
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
65 {u'comments': u'value1', u'node': u'value2', u'service': u'value3'}
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
66 """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
67 #  FIXME: this format seem overcomplicated, it may be more appropriate to use json here
2153
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
68 if pop:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
69 get = lambda d, k: d.pop(k)
2153
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
70 else:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
71 get = lambda d, k: d[k]
2153
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
72 for idx, main_value in enumerate(dict2iter(name, dict_, pop=pop)):
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
73 ret = {name: main_value}
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
74 for k in extra_keys:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
75 ret[k] = get(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
76 dict_, u"{}{}_{}".format(name, (u"#" + unicode(idx)) if idx else u"", k)
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
77 )
2153
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
78 yield ret
f67434fd88d2 tools (common/data_format): added dict2iterdict to yield dictionary from serialised complex values
Goffi <goffi@goffi.org>
parents: 2079
diff changeset
79
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
80
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
81 def iter2dict(name, iter_, dict_=None, check_conflict=True):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
82 """Fill a dict with values from an iterable
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
83
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
84 name is used to serialise iter_, in the same way as in [dict2iter]
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
85 Build from the tags a dict using the microblog data format.
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
86
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
87 @param name(unicode): key to use for serialisation
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
88 e.g. "group" to have keys "group", "group#1", "group#2", ...
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
89 @param iter_(iterable): values to store
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
90 @param dict_(None, dict): dictionary to fill, or None to create one
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
91 @param check_conflict(bool): if True, raise an exception in case of existing key
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
92 @return (dict): filled dict, or newly created one
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
93 @raise exceptions.ConflictError: a needed key already exists
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
94 """
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
95 if dict_ is None:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
96 dict_ = {}
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
97 for idx, value in enumerate(iter_):
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
98 if idx == 0:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
99 key = name
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
100 else:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
101 key = u"{}#{}".format(name, idx)
1660
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
102 if check_conflict and key in dict_:
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
103 raise exceptions.ConflictError
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
104 dict_[key] = value
204c2a7fe68b core (tools): new tools.common module with tools used by backend and frontends. dict2iter and iter2dict method to serialise iterable in a dict (for bridge)
Goffi <goffi@goffi.org>
parents:
diff changeset
105 return dict
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
106
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
107
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
108 def getSubDict(name, dict_, sep=u"_"):
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
109 """get a sub dictionary from a serialised dictionary
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
110
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
111 look for keys starting with name, and create a dict with it
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
112 eg.: if "key" is looked for, {'html': 1, 'key_toto': 2, 'key_titi': 3} will return:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
113 {None: 1, toto: 2, titi: 3}
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
114 @param name(unicode): name of the key
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
115 @param dict_(dict): dictionary with the serialised list
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
116 @param sep(unicode): separator used between name and subkey
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
117 @return iter: iterate through the deserialised items
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
118 """
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
119 for k, v in dict_.iteritems():
2079
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
120 if k.startswith(name):
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
121 if k == name:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
122 yield None, v
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
123 else:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
124 if k[len(name)] != sep:
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
125 continue
c8e561a5b2b6 core (tools/common): added getSubDict method in data_format to get sub dictionary from serialised one
Goffi <goffi@goffi.org>
parents: 1934
diff changeset
126 else:
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
127 yield k[len(name) + 1 :], v
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
128
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
129 def serialise(data):
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
130 """Serialise data so it can be sent to bridge
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
131
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
132 @return(unicode): serialised data, can be transmitted as string to the bridge
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
133 """
2709
46f2733a2a9b tools (common/data_format): use "default" in json.dumps in serialise (useful notably to serialise jid.JID)
Goffi <goffi@goffi.org>
parents: 2697
diff changeset
134 return json.dumps(data, ensure_ascii=False, default=unicode)
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
135
2697
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
136 def deserialise(serialised_data, default=None, type_check=dict):
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
137 """Deserialize data from bridge
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
138
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
139 @param serialised_data(unicode): data to deserialise
2697
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
140 @default (object): value to use when serialised data is empty string
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
141 @param type_check(type): if not None, the deserialised data must be of this type
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
142 @return(object): deserialised data
2697
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
143 @raise ValueError: serialised_data is of wrong type
2656
8cacf7929f3c tools (common/data_format): added serialise and deserialise methods (using json for now)
Goffi <goffi@goffi.org>
parents: 2624
diff changeset
144 """
2697
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
145 if serialised_data == u"":
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
146 return default
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
147 ret = json.loads(serialised_data)
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
148 if type_check is not None and not isinstance(ret, type_check):
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
149 raise ValueError(u"Bad data type, was expecting {type_check}, got {real_type}"
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
150 .format(type_check=type_check, real_type=type(ret)))
fcc945537d5f tools (common/data_format): serialise now check types and return a default value when empty string is parsed.
Goffi <goffi@goffi.org>
parents: 2656
diff changeset
151 return ret