Mercurial > libervia-backend
annotate frontends/src/tools/strings.py @ 2316:7b448ac50a69
jp (pubsub): new search command:
search is a kind of "grep for Pubsub". It's a powerful command which allows to look for specific data in a pubsub node, recurse sub nodes if requested, and execute an action on the result.
search allows to look for items with following filter:
- simple text search
- regex
- xpath
- python code
filters are read an applied in the order in which they appear on the command line.
Then flags can be used to modify behaviour, currently there are:
- ignore-case to specify if search must be case sensitive or not
- invert to invert result of the search (i.e. don't match instead of match)
- dot-all which is specific for regex, cf. re module
- only-matching which return the matching part instead of the full item
Once a item match filters, an action is applied to it, currenlty there are:
- print, which do a simple output of the full item (default)
- exec, which run a jp command, specifying the service, node and item corresponding to the match
- exteral, which run a external command, sending the full item on stdin
By default search is only done on requested node, but if max-depth is more than 0, sub nodes will be searched too.
author | Goffi <goffi@goffi.org> |
---|---|
date | Sat, 08 Jul 2017 21:54:24 +0200 |
parents | 2c31ddf633e5 |
children |
rev | line source |
---|---|
1934
2daf7b4c6756
use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents:
1879
diff
changeset
|
1 #!/usr/bin/env python2 |
690
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
2 # -*- coding: utf-8 -*- |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
3 |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
4 # SAT helpers methods for plugins |
1766 | 5 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org) |
690
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
6 |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
7 # This program is free software: you can redistribute it and/or modify |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
8 # it under the terms of the GNU Affero General Public License as published by |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
9 # the Free Software Foundation, either version 3 of the License, or |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
10 # (at your option) any later version. |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
11 |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
12 # This program is distributed in the hope that it will be useful, |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
15 # GNU Affero General Public License for more details. |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
16 |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
17 # You should have received a copy of the GNU Affero General Public License |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
18 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
19 |
769
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
20 import re |
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
21 |
2107
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
22 # Regexp from http://daringfireball.net/2010/07/improved_regex_for_matching_urls |
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
23 RE_URL = re.compile(r"""(?i)\b((?:[a-z]{3,}://|(www|ftp)\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/|mailto:|xmpp:)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?]))""") |
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
24 |
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
25 |
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
26 # TODO: merge this class with an other module or at least rename it (strings is not a good name) |
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
27 |
690
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
28 |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
29 def getURLParams(url): |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
30 """This comes from pyjamas.Location.makeUrlDict with a small change |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
31 to also parse full URLs, and parameters with no value specified |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
32 (in that case the default value "" is used). |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
33 @param url: any URL with or without parameters |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
34 @return: a dictionary of the parameters, if any was given, or {} |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
35 """ |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
36 dict_ = {} |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
37 if "/" in url: |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
38 # keep the part after the last "/" |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
39 url = url[url.rindex("/") + 1:] |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
40 if url.startswith("?"): |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
41 # remove the first "?" |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
42 url = url[1:] |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
43 pairs = url.split("&") |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
44 for pair in pairs: |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
45 if len(pair) < 3: |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
46 continue |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
47 kv = pair.split("=", 1) |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
48 dict_[kv[0]] = kv[1] if len(kv) > 1 else "" |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
49 return dict_ |
d8e7a58eaa00
misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff
changeset
|
50 |
769
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
51 |
1790
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
52 def addURLToText(string, new_target=True): |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
53 """Check a text for what looks like an URL and make it clickable. |
769
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
54 |
1790
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
55 @param string (unicode): text to process |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
56 @param new_target (bool): if True, make the link open in a new window |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
57 """ |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
58 # XXX: report any change to libervia.browser.strings.addURLToText |
769
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
59 def repl(match): |
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
60 url = match.group(0) |
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
61 if not re.match(r"""[a-z]{3,}://|mailto:|xmpp:""", url): |
8b6137f7b4e1
tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents:
719
diff
changeset
|
62 url = "http://" + url |
1790
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
63 target = ' target="_blank"' if new_target else '' |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
64 return '<a href="%s"%s class="url">%s</a>' % (url, target, match.group(0)) |
2107
2c31ddf633e5
frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents:
1935
diff
changeset
|
65 return RE_URL.sub(repl, string) |
842
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
66 |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
67 |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
68 def addURLToImage(string): |
1790
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
69 """Check a XHTML text for what looks like an imageURL and make it clickable. |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
70 |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
71 @param string (unicode): text to process |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
72 """ |
bf7e468fe440
tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents:
1766
diff
changeset
|
73 # XXX: report any change to libervia.browser.strings.addURLToImage |
842
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
74 def repl(match): |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
75 url = match.group(1) |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
76 return '<a href="%s" target="_blank">%s</a>' % (url, match.group(0)) |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
77 pattern = r"""<img[^>]* src="([^"]+)"[^>]*>""" |
429c6a0ef73d
frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents:
811
diff
changeset
|
78 return re.sub(pattern, repl, string) |
1801
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
79 |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
80 |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
81 def fixXHTMLLinks(xhtml): |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
82 """Add http:// if the scheme is missing and force opening in a new window. |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
83 |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
84 @param string (unicode): XHTML Content |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
85 """ |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
86 subs = [] |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
87 for match in re.finditer(r'<a( \w+="[^"]*")* ?/?>', xhtml): |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
88 tag = match.group(0) |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
89 url = re.search(r'href="([^"]*)"', tag) |
1879
b43ee22eac98
frontends (tools/strings): fixXHTMLLinks skips internal anchors (links starting with "#")
souliane <souliane@mailoo.org>
parents:
1801
diff
changeset
|
90 if url and not url.group(1).startswith("#"): # skip internal anchor |
1801
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
91 if not re.search(r'target="([^"]*)"', tag): # no target |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
92 subs.append((tag, '<a target="_blank"%s' % tag[2:])) |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
93 if not re.match(r"^\w+://", url.group(1)): # no scheme |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
94 subs.append((url.group(0), 'href="http://%s"' % url.group(1))) |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
95 |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
96 for url, new_url in subs: |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
97 xhtml = xhtml.replace(url, new_url) |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
98 return xhtml |
07390a9d1c09
tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents:
1790
diff
changeset
|
99 |