libervia-backend: libervia/backend/plugins/plugin_misc_url

annotate libervia/backend/plugins/plugin_misc_url_preview.py @ 4314:6a70fcd93a7a

plugin XEP-0131: Stanza Headers and Internet Metadata implementation: - SHIM is now supported and put in `msg_data["extra"]["headers"]`. - `Keywords` are converted from and to list of string in `msg_data["extra"]["keywords"]` field (if present in headers on message sending, values are merged). - Python minimal version upgraded to 3.11 due to use of `StrEnum`. rel 451

author	Goffi <goffi@goffi.org>
date	Sat, 28 Sep 2024 15:56:04 +0200
parents	0d7bb4df2343
children

rev	line source
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	2
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	3
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	4 # Libervia plugin to handle events
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	5 # Copyright (C) 2009-2022 Jérôme Poisson (goffi@goffi.org)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	6
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	10 # (at your option) any later version.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	11
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	16
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	19
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	20 from dataclasses import dataclass
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	21 import json
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	22 from textwrap import dedent
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	23 from typing import Callable, Dict, List, Optional, Union
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	24 from urllib import parse
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	25 import fnmatch
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	26
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	27 from lxml import etree
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	28 import treq
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	29 from twisted.internet import defer
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	30
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	31 from libervia.backend.core.constants import Const as C
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	32 from libervia.backend.core.core_types import SatXMPPEntity
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	33 from libervia.backend.core.exceptions import ConflictError
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	34 from libervia.backend.core.i18n import _
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	35 from libervia.backend.core.log import getLogger
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	36 from libervia.backend.tools.common import data_format
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	37 from libervia.backend.tools.common.async_utils import async_lru
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	38
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	39 log = getLogger(__name__)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	40
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	41 PLUGIN_INFO = {
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	42 C.PI_NAME: "Preview",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	43 C.PI_IMPORT_NAME: "Preview",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	44 C.PI_TYPE: C.PLUG_TYPE_MISC,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	45 C.PI_PROTOCOLS: ["Open Graph", "oEmbed"],
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	46 C.PI_DEPENDENCIES: ["TEXT_SYNTAXES"],
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	47 C.PI_MAIN: "Preview",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	48 C.PI_HANDLER: "no",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	49 C.PI_DESCRIPTION: dedent(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	50 _(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	51 """\
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	52 Retrieves and provides a preview of URLs using various protocols. Initially, it
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	53 uses the Open Graph protocol for most web pages. Specialized handlers are
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	54 implemented for YouTube using the oEmbed protocol.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	55 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	56 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	57 ),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	58 }
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	59
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	60 OG_TAGS = [
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	61 "title",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	62 "type",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	63 "image",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	64 "url",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	65 "audio",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	66 "description",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	67 "determiner",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	68 "locale",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	69 "locale:alternate",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	70 "site_name",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	71 "video",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	72 ]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	73
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	74
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	75 class PreviewFetchError(Exception):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	76 pass
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	77
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	78
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	79 @dataclass
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	80 class Protocol:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	81 name: str
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	82 callback: Callable
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	83 priority: int
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	84
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	85
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	86 class Preview:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	87 protocols: Dict[str, Protocol] = {}
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	88 domain_protocols: Dict[str, str] = {}
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	89
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	90 def __init__(self, host):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	91 log.info(_("Preview plugin initialization"))
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	92 self.host = host
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	93
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	94 # generic protocols
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	95
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	96 self.register("open_graph", self.fetch_open_graph_data, priority=100)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	97 self.register("oembed", self.fetch_generic_oembed_data, priority=50)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	98 self.register("generic", self.fetch_generic_data, priority=0)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	99
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	100 # domain specific protocols
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	101
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	102 self.register("oembed-youtube", self.fetch_youtube_oembed_data, priority=-100)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	103 self.register_domain_protocol(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	104 ["www.youtube.com", "youtu.be", "m.youtube.com"], "oembed-youtube"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	105 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	106
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	107 self.register("wikipedia", self.fetch_wikipedia_data, priority=-80)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	108 self.register_domain_protocol(["*.wikipedia.org"], "wikipedia")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	109
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	110 self.register("invidious", self.fetch_invidious_data, priority=-90)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	111 self.register_domain_protocol(
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	112 ["yewtu.be", "www.yewtu.be", "invidious.fdn.fr"], "invidious"
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	113 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	114
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	115 # bridge methods
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	116
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	117 host.bridge.add_method(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	118 "url_preview_get",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	119 ".plugin",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	120 in_sign="sss",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	121 out_sign="s",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	122 method=self._url_preview_get,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	123 async_=True,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	124 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	125
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	126 # API
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	127
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	128 def _url_preview_get(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	129 self, url: str, options: str, profile_key: str
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	130 ) -> defer.Deferred:
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	131 client = self.host.get_client(profile_key)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	132 d = defer.ensureDeferred(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	133 self.get_preview_data(client, url, data_format.deserialise(options))
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	134 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	135 d.addCallback(data_format.serialise)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	136 return d
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	137
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	138 @async_lru()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	139 async def get_preview_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	140 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	141 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	142 """Fetch preview data from a url using registered protocols
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	143
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	144 @param url: The url to fetch the preview data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	145 @param options: Additional options that may be used while fetching preview data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	146 @return: A dictionary containing the preview data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	147 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	148 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	149 parsed_url = parse.urlparse(url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	150 domain = parsed_url.netloc
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	151
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	152 preview_data: Optional[dict] = None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	153 matched_protocol = None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	154 for registered_domain, registered_protocol in self.domain_protocols.items():
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	155 if fnmatch.fnmatch(domain, registered_domain):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	156 matched_protocol = registered_protocol
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	157 break
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	158
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	159 if matched_protocol is not None:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	160 callback = self.protocols[matched_protocol].callback
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	161 preview_data = await callback(client, url, options)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	162 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	163 for name, protocol in sorted(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	164 self.protocols.items(), key=lambda item: item[1].priority, reverse=True
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	165 ):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	166 try:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	167 preview_data = await protocol.callback(client, url, options)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	168 except Exception as e:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	169 log.warning(f"Can't run protocol {name} for {url}: {e}")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	170 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	171 if preview_data is not None:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	172 matched_protocol = protocol.name
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	173 break
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	174
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	175 if preview_data is not None:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	176 preview_data["protocol"] = matched_protocol
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	177 # we don't clean html for youtube as we need Javascript to make it work, and
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	178 # for invidious as we generate it ourself
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	179 if "html" in preview_data:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	180 if matched_protocol in ("oembed-youtube", "invidious"):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	181 # this flag indicate that we know the source of HTML and we should be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	182 # able to trust it. This will add `allow-scripts` and
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	183 # `allow-same-origin` in the preview <iframe> "sandbox" attribute
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	184 preview_data["html_known"] = True
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	185 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	186 preview_data["html_known"] = False
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	187 clean_xhtml = self.host.plugins["TEXT_SYNTAXES"].clean_xhtml
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	188 try:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	189 preview_data["html"] = clean_xhtml(preview_data["html"])
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	190 except Exception as e:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	191 log.warning(f"Can't clean html data: {e}\n{preview_data}")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	192 del preview_data["html"]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	193
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	194 return preview_data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	195
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	196 @classmethod
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	197 def register(cls, name: str, callback: Callable, priority: int = 0):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	198 """Register a protocol to retrieve preview data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	199
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	200 The registered callback should return a dictionary of preview data if available,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	201 or None otherwise.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	202
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	203 @param name: Unique name of the protocol
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	204 @param callback: Async callback function to fetch preview data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	205 @param priority: Priority of the protocol, with higher numbers indicating higher
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	206 priority
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	207 @return: None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	208 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	209 if name in cls.protocols:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	210 raise ConflictError(f"Protocol with the name {name} is already registered.")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	211
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	212 cls.protocols[name] = Protocol(name=name, callback=callback, priority=priority)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	213
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	214 @classmethod
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	215 def register_domain_protocol(cls, domains: Union[str, List[str]], protocol_name: str):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	216 """Register a protocol for a specific domain or list of domains
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	217
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	218 @param domains: The domain name or list of domain names
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	219 @param protocol_name: The name of the protocol to be associated with the domain(s)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	220 @return: None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	221 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	222 protocol_name = protocol_name.replace(" ", "").lower()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	223 if protocol_name not in cls.protocols:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	224 raise ConflictError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	225 f"Protocol with the name {protocol_name} is not registered."
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	226 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	227
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	228 if isinstance(domains, str):
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	229 domains = [domains]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	230
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	231 for domain in domains:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	232 domain = domain.strip()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	233 if not domain:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	234 log.warning("empty string used as domain, ignoring")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	235 continue
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	236 cls.domain_protocols[domain] = protocol_name
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	237
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	238 # Open Graph
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	239
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	240 async def fetch_open_graph_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	241 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	242 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	243 """Fetch Open Graph data from a url
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	244
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	245 This method implements the Open Graph protocol, details of which can be found at:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	246 http://ogp.me/
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	247
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	248 @param url: The url to fetch the Open Graph data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	249 @param options: Additional options that may be used while fetching data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	250 @return: A dictionary containing the Open Graph data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	251 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	252 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	253 resp = await treq.get(url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	254
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	255 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	256 html = await resp.text()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	257 parser = etree.HTMLParser()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	258 tree = etree.fromstring(html, parser)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	259
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	260 # Extract Open Graph data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	261 metadata = {}
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	262 for tag in OG_TAGS:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	263 og_el = tree.find('.//meta[@property="og:{tag}"]'.format(tag=tag))
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	264 if og_el is not None:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	265 metadata[tag] = og_el.get("content")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	266
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	267 if metadata:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	268 if "site_name" in metadata and not "provider_name" in metadata:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	269 metadata["provider_name"] = metadata["site_name"]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	270 return metadata
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	271
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	272 return None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	273 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	274 raise PreviewFetchError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	275 f"Failed to fetch preview for {url}, status code: {resp.code}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	276 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	277
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	278 # oEmbed
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	279
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	280 async def _fetch_oembed_data(self, oembed_url: str) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	281 """Fetch oEmbed data from a given oEmbed URL
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	282
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	283 @param oembed_url: The url to fetch the oEmbed data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	284 @return: A dictionary containing the oEmbed data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	285 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	286 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	287 resp = await treq.get(oembed_url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	288 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	289 return json.loads(await resp.text())
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	290 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	291 raise PreviewFetchError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	292 f"Failed to fetch oEmbed preview for {oembed_url}, status code: "
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	293 f"{resp.code}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	294 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	295
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	296 async def fetch_youtube_oembed_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	297 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	298 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	299 """Fetch YouTube oEmbed data from a url
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	300
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	301 @param url: The url to fetch the YouTube oEmbed data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	302 @param options: Additional options that may be used while fetching data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	303 @return: A dictionary containing the YouTube oEmbed data or None if no data could
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	304 be fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	305 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	306 oembed_url = f"https://www.youtube.com/oembed?url={parse.quote(url)}&format=json"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	307 data = await self._fetch_oembed_data(oembed_url)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	308 if data is not None and "html" in data:
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	309 html = data["html"]
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	310 root = etree.HTML(html)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	311 iframe_elt = root.xpath("//iframe")
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	312 if iframe_elt:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	313 iframe_elt[0].attrib[
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	314 "style"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	315 ] = "position: absolute; top: 0; left: 0; width: 100%; height: 100%;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	316 data["html"] = etree.tostring(root, method="html", encoding="unicode")
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	317 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	318 log.warning("No <iframe> found in the YouTube oEmbed response")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	319
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	320 return data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	321
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	322 async def fetch_generic_oembed_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	323 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	324 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	325 """Fetch generic oEmbed data from a url
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	326
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	327 @param url: The url to fetch the oEmbed data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	328 @param options: Additional options that may be used while fetching data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	329 @return: A dictionary containing the oEmbed data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	330 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	331 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	332 resp = await treq.get(url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	333 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	334 html = await resp.text()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	335 parser = etree.HTMLParser()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	336 tree = etree.fromstring(html, parser)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	337
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	338 # Find oEmbed URL
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	339 oembed_link = tree.find('.//link[@type="application/json+oembed"]')
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	340 if oembed_link is not None:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	341 oembed_url = oembed_link.get("href")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	342 return await self._fetch_oembed_data(oembed_url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	343 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	344 return None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	345 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	346 raise PreviewFetchError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	347 f"Failed to fetch preview for {url}, status code: {resp.code}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	348 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	349
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	350 async def fetch_generic_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	351 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	352 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	353 """Fetch generic data from a url
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	354
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	355 This method attempts to extract the title, description, and author metadata from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	356 the HTML of the page. If these data cannot be found, the method will return None.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	357
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	358 @param url: The url to fetch the generic data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	359 @param options: Additional options that may be used while fetching data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	360 @return: A dictionary containing the generic data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	361 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	362 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	363 resp = await treq.get(url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	364 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	365 html = await resp.text()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	366 parser = etree.HTMLParser()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	367 tree = etree.fromstring(html, parser)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	368
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	369 # Find title, description, and author metadata
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	370 title_el = tree.find(".//title")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	371 desc_el = tree.find('.//meta[@name="description"]')
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	372 author_el = tree.find('.//meta[@name="author"]')
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	373
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	374 metadata = {
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	375 "title": title_el.text if title_el is not None else "",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	376 "description": desc_el.get("content") if desc_el is not None else "",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	377 "author_name": author_el.get("content") if author_el is not None else "",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	378 "url": url,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	379 "provider_name": parse.urlparse(url).netloc,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	380 "provider_url": f"{parse.urlparse(url).scheme}://{parse.urlparse(url).netloc}",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	381 }
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	382
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	383 return metadata if any(metadata.values()) else None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	384 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	385 raise PreviewFetchError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	386 f"Failed to fetch generic preview for {url}, status code: {resp.code}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	387 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	388
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	389 # Wikipedia
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	390
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	391 async def fetch_wikipedia_data(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	392 self, client: SatXMPPEntity, url: str, options: dict
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	393 ) -> Optional[dict]:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	394 """Fetch Wikipedia data from a url
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	395
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	396 This method implements the Wikipedia API, details of which can be found at:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	397 https://www.mediawiki.org/wiki/API:Main_page
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	398
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	399 @param url: The url to fetch the Wikipedia data from
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	400 @param options: Additional options that may be used while fetching data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	401 @return: A dictionary containing the Wikipedia data or None if no data could be
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	402 fetched
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	403 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	404 parsed_url = parse.urlparse(url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	405 page_name = parsed_url.path.split("/")[-1]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	406
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	407 # Use the Wikipedia API to get a summary of the page and a preview image
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	408 api_url = (
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	409 f"https://{parsed_url.netloc}/w/api.php?format=json&action=query&"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	410 f"prop=extracts\|pageimages&exintro&explaintext&redirects=1&piprop=thumbnail"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	411 f"&pithumbsize=300&titles={page_name}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	412 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	413
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	414 resp = await treq.get(api_url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	415 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	416 data = json.loads(await resp.text())
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	417 # The page ID is the first key in the "pages" dictionary
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	418 page_id = next(iter(data["query"]["pages"].keys()))
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	419 page = data["query"]["pages"][page_id]
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	420
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	421 # The API may return a page with a missing title or extract if the page does
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	422 # not exist
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	423 if "missing" in page:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	424 return None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	425
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	426 return {
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	427 "provider_name": "Wikipedia",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	428 "provider_url": "https://www.wikipedia.org",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	429 "title": page.get("title"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	430 "description": page.get("extract"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	431 "url": url,
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	432 "image": (
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	433 page.get("thumbnail", {}).get("source")
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	434 if "thumbnail" in page
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	435 else None
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	436 ),
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	437 }
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	438 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	439 raise PreviewFetchError(
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	440 f"Failed to fetch Wikipedia preview for {url}, status code: {resp.code}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	441 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	442
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	443 # Invidious
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	444
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	445 async def fetch_invidious_data(
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	446 self, client: SatXMPPEntity, url: str, options: dict
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	447 ) -> Optional[dict]:
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	448 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	449 Fetch Invidious data from a url and generate HTML iframe.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	450
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	451 @param url: The url to fetch the Invidious data from.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	452 @param options: Additional options that may be used while fetching data.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	453 @return: A dictionary containing the Invidious data or None if no data could be fetched.
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	454 """
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	455 parsed_url = parse.urlparse(url)
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	456 if "watch" in parsed_url.path:
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	457 video_id = parse.parse_qs(parsed_url.query).get("v", [None])[0]
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	458 else:
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	459 video_id = parsed_url.path.strip("/")
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	460 if not video_id:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	461 log.warning(f"Can't extract video ID from {url}")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	462 return None
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	463
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	464 invidious_api_url = f"https://{parsed_url.netloc}/api/v1/videos/{video_id}"
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	465
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	466 resp = await treq.get(invidious_api_url)
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	467 if resp.code == 200:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	468 video_data = await resp.json()
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	469 # construct the iframe html code
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	470 html = (
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	471 f"<iframe"
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	472 f' width="100%"'
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	473 f' height="auto"'
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	474 f' src="https://{parsed_url.netloc}/embed/{video_id}"'
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	475 f' frameborder="0" '
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	476 f' allow="'
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	477 f" accelerometer;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	478 f" autoplay;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	479 f" clipboard-write;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	480 f" encrypted-media;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	481 f" gyroscope;"
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	482 f' picture-in-picture"'
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	483 f' style="'
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	484 f" position: absolute;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	485 f" top: 0;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	486 f" left: 0;"
0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	487 f" width: 100%;"
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	488 f' height: 100%;"'
4270 0d7bb4df2343 Reformatted code base using black. Goffi <goffi@goffi.org> parents: 4103 diff changeset	489 f" allowfullscreen></iframe>"
4103 eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	490 )
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	491 # structure the data to be returned
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	492 data = {
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	493 "title": video_data.get("title"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	494 "description": video_data.get("description"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	495 "url": url,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	496 "image": video_data.get("videoThumbnails", [{}])[0].get("url"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	497 "provider_name": "Invidious",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	498 "provider_url": f"https://{parsed_url.netloc}",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	499 "html": html,
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	500 "author_name": video_data.get("author"),
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	501 "author_url": f"https://{parsed_url.netloc}/channel/{video_data.get('authorId')}",
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	502 }
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	503 return data
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	504 else:
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	505 log.warning(f"Unable to fetch video data from Invidious API for {video_id}")
eaa0daa7f834 plugin URL preview: URL preview first draft Goffi <goffi@goffi.org> parents: diff changeset	506 return None

Mercurial > libervia-backend

annotate libervia/backend/plugins/plugin_misc_url_preview.py @ 4314:6a70fcd93a7a