Mercurial > libervia-backend

annotate sat_frontends/tools/strings.py @ 2766:93a421de0e3d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
tools (common/data_objects): metadata parsing in BlogItems: metadata are parsed to deserialise some well known values like rsm index and count, and some properties have been added to have easier access. A "complete" property (or item in metadata) is set to True if we are on the last page, False if we are not, or None if we don't have enough data to know if we are on the last page or not.
author Goffi <goffi@goffi.org>
date Fri, 11 Jan 2019 16:35:13 +0100
parents 56f94936df1e
children a754523fc0f6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1934
2daf7b4c6756 use of /usr/bin/env instead of /usr/bin/python in shebang
Goffi <goffi@goffi.org>
parents: 1879
diff changeset
1 #!/usr/bin/env python2
690
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
3
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
4 # SAT helpers methods for plugins
1766
d17772b0fe22 copyright update
Goffi <goffi@goffi.org>
parents: 1396
diff changeset
5 # Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
690
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
6
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
7 # This program is free software: you can redistribute it and/or modify
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
8 # it under the terms of the GNU Affero General Public License as published by
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
9 # the Free Software Foundation, either version 3 of the License, or
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
10 # (at your option) any later version.
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
11
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
12 # This program is distributed in the hope that it will be useful,
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
15 # GNU Affero General Public License for more details.
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
16
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
17 # You should have received a copy of the GNU Affero General Public License
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
19
769
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
20 import re
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
21
2107
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
22 # Regexp from http://daringfireball.net/2010/07/improved_regex_for_matching_urls
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
23 RE_URL = re.compile(
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
24 r"""(?i)\b((?:[a-z]{3,}://|(www|ftp)\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/|mailto:|xmpp:)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'".,<>?]))"""
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
25 )
2107
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
26
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
27
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
28 # TODO: merge this class with an other module or at least rename it (strings is not a good name)
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
29
690
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
30
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
31 def getURLParams(url):
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
32 """This comes from pyjamas.Location.makeUrlDict with a small change
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
33 to also parse full URLs, and parameters with no value specified
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
34 (in that case the default value "" is used).
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
35 @param url: any URL with or without parameters
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
36 @return: a dictionary of the parameters, if any was given, or {}
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
37 """
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
38 dict_ = {}
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
39 if "/" in url:
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
40 # keep the part after the last "/"
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
41 url = url[url.rindex("/") + 1 :]
690
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
42 if url.startswith("?"):
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
43 # remove the first "?"
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
44 url = url[1:]
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
45 pairs = url.split("&")
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
46 for pair in pairs:
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
47 if len(pair) < 3:
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
48 continue
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
49 kv = pair.split("=", 1)
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
50 dict_[kv[0]] = kv[1] if len(kv) > 1 else ""
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
51 return dict_
d8e7a58eaa00 misc: added a file for frontend's string operations:
souliane <souliane@mailoo.org>
parents:
diff changeset
52
769
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
53
1790
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
54 def addURLToText(string, new_target=True):
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
55 """Check a text for what looks like an URL and make it clickable.
769
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
56
1790
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
57 @param string (unicode): text to process
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
58 @param new_target (bool): if True, make the link open in a new window
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
59 """
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
60 # XXX: report any change to libervia.browser.strings.addURLToText
769
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
61 def repl(match):
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
62 url = match.group(0)
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
63 if not re.match(r"""[a-z]{3,}://|mailto:|xmpp:""", url):
8b6137f7b4e1 tools: addURLToText moved from libervia to sat_frontends/tools/strings
souliane <souliane@mailoo.org>
parents: 719
diff changeset
64 url = "http://" + url
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
65 target = ' target="_blank"' if new_target else ""
1790
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
66 return '<a href="%s"%s class="url">%s</a>' % (url, target, match.group(0))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
67
2107
2c31ddf633e5 frontends(tools/strings): put URL regex outside of getURLParams and precompile it
Goffi <goffi@goffi.org>
parents: 1935
diff changeset
68 return RE_URL.sub(repl, string)
842
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
69
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
70
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
71 def addURLToImage(string):
1790
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
72 """Check a XHTML text for what looks like an imageURL and make it clickable.
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
73
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
74 @param string (unicode): text to process
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
75 """
bf7e468fe440 tools (strings): add a parameter "new_target" to addURLToText
souliane <souliane@mailoo.org>
parents: 1766
diff changeset
76 # XXX: report any change to libervia.browser.strings.addURLToImage
842
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
77 def repl(match):
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
78 url = match.group(1)
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
79 return '<a href="%s" target="_blank">%s</a>' % (url, match.group(0))
2624
56f94936df1e code style reformatting using black
Goffi <goffi@goffi.org>
parents: 2562
diff changeset
80
842
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
81 pattern = r"""<img[^>]* src="([^"]+)"[^>]*>"""
429c6a0ef73d frontends (tools): addURLToImage makes an image clickable
souliane <souliane@mailoo.org>
parents: 811
diff changeset
82 return re.sub(pattern, repl, string)
1801
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
83
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
84
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
85 def fixXHTMLLinks(xhtml):
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
86 """Add http:// if the scheme is missing and force opening in a new window.
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
87
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
88 @param string (unicode): XHTML Content
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
89 """
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
90 subs = []
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
91 for match in re.finditer(r'<a( \w+="[^"]*")* ?/?>', xhtml):
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
92 tag = match.group(0)
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
93 url = re.search(r'href="([^"]*)"', tag)
1879
b43ee22eac98 frontends (tools/strings): fixXHTMLLinks skips internal anchors (links starting with "#")
souliane <souliane@mailoo.org>
parents: 1801
diff changeset
94 if url and not url.group(1).startswith("#"): # skip internal anchor
1801
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
95 if not re.search(r'target="([^"]*)"', tag): # no target
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
96 subs.append((tag, '<a target="_blank"%s' % tag[2:]))
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
97 if not re.match(r"^\w+://", url.group(1)): # no scheme
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
98 subs.append((url.group(0), 'href="http://%s"' % url.group(1)))
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
99
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
100 for url, new_url in subs:
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
101 xhtml = xhtml.replace(url, new_url)
07390a9d1c09 tools (strings): add method fixXHTMLLinks to add a scheme if missing, and force opening in new tab
souliane <souliane@mailoo.org>
parents: 1790
diff changeset
102 return xhtml