libervia-backend: libervia/frontends/tools/webrtc.py annotate

annotate libervia/frontends/tools/webrtc.py @ 4242:8acf46ed7f36

frontends: remote control implementation: This is the frontends common part of remote control implementation. It handle the creation of WebRTC session, and management of inputs. For now the reception use freedesktop.org Desktop portal, and works mostly with Wayland based Desktop Environments. rel 436

author	Goffi <goffi@goffi.org>
date	Sat, 11 May 2024 13:52:43 +0200
parents	79c8a70e1813
children	0d7bb4df2343

rev	line source
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	2
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	5
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	10
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	15
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	18
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	19 from collections.abc import Awaitable
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	20 import gi
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	21
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	22 gi.require_versions({"Gst": "1.0", "GstWebRTC": "1.0"})
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	23 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	24
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	25 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	26
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	27 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	28 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	29 except ImportError:
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	30 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	31 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	32 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	33 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	34 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	35 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	36 import logging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	37 import re
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	38 from typing import Callable, Final
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	39 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	40
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	41 from libervia.backend.tools.common import data_format
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	42 from libervia.frontends.tools import aio, display_servers, jid
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	43 from .webrtc_models import (
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	44 CallData,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	45 SinksApp,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	46 SinksAuto,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	47 SinksData,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	48 SinksDataChannel,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	49 SinksNone,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	50 SourcesAuto,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	51 SourcesData,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	52 SourcesDataChannel,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	53 SourcesNone,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	54 SourcesPipeline,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	55 SourcesTest,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	56 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	57
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	58 current_server = display_servers.detect()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	59 if current_server == display_servers.X11:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	60 # GSTreamer's ximagesrc documentation asks to run this function
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	61 import ctypes
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	62
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	63 ctypes.CDLL("libX11.so.6").XInitThreads()
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	64
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	65
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	66 log = logging.getLogger(__name__)
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	67 VIDEO_SOURCE_AUTO: Final = "v4l2src"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	68 AUDIO_SOURCE_AUTO: Final = "pulsesrc"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	69 NONE_NOT_IMPLEMENTED_MSG: Final = "None value is not handled yet."
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	70
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	71 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	72
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	73
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	74 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	75 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	76
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	77 This class encapsulates the WebRTC functionalities required for initiating and
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	78 handling audio and video calls, and data channels.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	79 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	80
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	81 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	82 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	83 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	84 profile: str,
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	85 sources_data: SourcesData\|None = None,
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	86 sinks_data: SinksData\|None = None,
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	87 reset_cb: Callable \| None = None,
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	88 merge_pip: bool \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	89 target_size: tuple[int, int] \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	90 call_start_cb: Callable[[str, dict, str], Awaitable[str]] \| None = None,
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	91 dc_data_list: list[SourcesDataChannel\|SinksDataChannel]\|None = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	92 ) -> None:
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	93 """Initializes a new WebRTC instance.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	94
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	95 @param bridge: An instance of backend bridge.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	96 @param profile: Libervia profile.
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	97 @param sources_data: Data of the sources.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	98 The model used will determine which sources to use.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	99 SourcesDataChannel can be used here as a convenience. It will then be moved
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	100 to ``data_channels`` and ``SourcesNone`` will be used instead for
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	101 ``sources_data``.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	102 If None, SourcesAuto will be used.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	103 @param sinks_data: Data of the sinks.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	104 The model used will determine which sinks to use.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	105 SinksDataChannel can be used here as a convenience. It will then be moved
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	106 to ``data_channels`` and ``SinksNone`` will be used instead for
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	107 ``sinks_data``.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	108 If None, SinksAuto will be used.
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	109 @param reset_cb: An optional Callable that is triggered on reset events. Can be
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	110 used to reset UI data on new calls.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	111 @param merge_pip: A boolean flag indicating whether Picture-in-Picture mode is
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	112 enabled. When PiP is used, local feedback is merged to remote video stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	113 Only one video stream is then produced (the local one).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	114 If None, PiP mode is selected automatically according to selected sink (it's
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	115 used for SinksAuto only for now).
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	116 @param target_size: Expected size of the final sink stream. Mainly use by composer
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	117 when ``merge_pip`` is set.
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	118 None to autodetect (no real autodetection implemeted yet, default to
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	119 (1280,720)).
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	120 @param call_start_cb: Called when call is started.
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	121 @param dc_data_list: Data Channels to create.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	122 If a SourcesDataChannel is used as ``sources_data``, or a SinksDataChannel is
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	123 used as ``sinks_data``, they will be automatically added to this list.
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	124 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	125 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	126 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	127 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	128 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	129 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	130 self._video_muted = False
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	131 self._desktop_sharing = False
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	132 self.desktop_sharing_data = None
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	133 if dc_data_list is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	134 dc_data_list = []
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	135 self.dc_data_list = dc_data_list
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	136 if sources_data is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	137 sources_data = SourcesAuto()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	138 elif isinstance(sources_data, SourcesDataChannel):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	139 dc_data_list.append(sources_data)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	140 sources_data = SourcesNone()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	141 self.sources_data = sources_data
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	142 if sinks_data is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	143 sinks_data = SinksAuto()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	144 elif isinstance(sinks_data, SinksDataChannel):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	145 dc_data_list.append(sinks_data)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	146 sinks_data = SinksNone()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	147 self.sinks_data = sinks_data
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	148 if target_size is None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	149 target_size = (1280, 720)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	150 self.target_width, self.target_height = target_size
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	151 if merge_pip is None:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	152 merge_pip = isinstance(sinks_data, SinksAuto)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	153 self.merge_pip = merge_pip
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	154 if call_start_cb is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	155 call_start_cb = self._call_start
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	156 self.call_start_cb = call_start_cb
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	157 if isinstance(sinks_data, SinksApp):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	158 if merge_pip and sinks_data.remote_video_cb is not None:
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	159 raise ValueError("Remote_video_cb can't be used when merge_pip is used!")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	160 self.reset_cb = reset_cb
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	161 if current_server == display_servers.WAYLAND:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	162 from .portal_desktop import DesktopPortal
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	163
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	164 self.desktop_portal = DesktopPortal(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	165 on_session_closed_cb=self.on_portal_session_closed
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	166 )
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	167 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	168 self.desktop_portal = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	169 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	170
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	171 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	172 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	173 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	174
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	175 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	176 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	177 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	178 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	179 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	180
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	181 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	182 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	183 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	184
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	185 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	186 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	187 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	188 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	189 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	190
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	191 @property
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	192 def desktop_sharing(self):
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	193 return self._desktop_sharing
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	194
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	195 @desktop_sharing.setter
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	196 def desktop_sharing(self, active: bool) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	197 if active != self._desktop_sharing:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	198 self._desktop_sharing = active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	199 self.on_desktop_switch(active)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	200 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	201 cb = self.bindings["desktop_sharing"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	202 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	203 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	204 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	205 cb(active)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	206
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	207 @property
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	208 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	209 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	210
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	211 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	212 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	213 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	214 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	215 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	216 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	217 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	218
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	219 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	220 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	221 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	222 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	223 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	224
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	225 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	226 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	227 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	228 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	229
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	230 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	231 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	232 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	233 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	234 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	235
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	236 def bind(self, **kwargs: Callable) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	237 self.bindings.clear()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	238 for key, cb in kwargs.items():
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	239 if key not in ("desktop_sharing",):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	240 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	241 'Only "desktop_sharing" is currently allowed for binding'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	242 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	243 self.bindings[key] = cb
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	244
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	245 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	246 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	247 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	248 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	249 with_timestamp: bool = True,
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	250 bin_: Gst.Bin \| None = None,
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	251 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	252 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	253
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	254 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	255 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	256 See
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	257 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	258 for details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	259
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	260 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	261 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	262 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	263 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	264 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	265 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	266 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	267 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	268 if with_timestamp:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	269 timestamp = datetime.now().isoformat(timespec="milliseconds")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	270 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	271
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	272 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	273
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	274 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	275 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	276
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	277 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	278 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	279 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	280 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	281 return index
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	282 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	283
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	284 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	285 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	286
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	287 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	288 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	289 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	290 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	291 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	292
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	293 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	294 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	295 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	296 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	297
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	298 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	299
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	300 def _a_call(self, method, args, *kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	301 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	302 aio.run_from_thread(method, args, *kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	303
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	304 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	305 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	306 ) -> dict[str, int \| None]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	307 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	308
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	309 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	310 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	311 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	312 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	313 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	314 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	315 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	316 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	317 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	318 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	319 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	320 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	321 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	322 continue
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	323 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	324 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	325 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	326 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	327 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	328 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	329 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	330 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	331 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	332
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	333 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	334 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	335
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	336 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	337 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	338 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	339 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	340 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	341 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	342 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	343 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	344 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	345 if match:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	346 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	347
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	348 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	349 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	350 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	351 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	352
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	353 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	354 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	355
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	356 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	357 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	358
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	359 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	360 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	361 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	362 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	363 return None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	364
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	365 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	366 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	367
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	368 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	369 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	370 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	371 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	372 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	373 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	374
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	375 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	376 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	377
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	378 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	379 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	380
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	381 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	382
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	383 def extract_ufrag_pwd(self, sdp: str) -> None:
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	384 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	385
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	386 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	387 """
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	388 lines = sdp.splitlines()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	389 media = ''
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	390 mid_media_map = {}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	391 bundle_media = set()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	392 bundle_ufrag = ''
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	393 bundle_pwd = ''
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	394 in_bundle = False
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	395
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	396 for line in lines:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	397 if line.startswith('m='):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	398 media = line.split('=')[1].split()[0]
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	399 elif line.startswith('a=mid:'):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	400 mid = line.split(':')[1].strip()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	401 mid_media_map[mid] = media
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	402 elif line.startswith('a=group:BUNDLE'):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	403 in_bundle = True
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	404 bundle_media = set(line.split(':')[1].strip().split())
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	405 elif line.startswith('a=ice-ufrag:'):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	406 if in_bundle:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	407 bundle_ufrag = line.split(':')[1].strip()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	408 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	409 self.ufrag[media] = line.split(':')[1].strip()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	410 elif line.startswith('a=ice-pwd:'):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	411 if in_bundle:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	412 bundle_pwd = line.split(':')[1].strip()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	413 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	414 self.pwd[media] = line.split(':')[1].strip()
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	415 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	416 in_bundle = False
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	417
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	418 if bundle_ufrag and bundle_pwd:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	419 for mid in bundle_media:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	420 media = mid_media_map[mid]
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	421 self.ufrag[media] = bundle_ufrag
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	422 self.pwd[media] = bundle_pwd
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	423
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	424 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	425 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	426 self.role: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	427 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	428 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	429 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	430 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	431 self.sid: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	432 self.offer: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	433 self.local_candidates_buffer = {}
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	434 self.ufrag: dict[str, str] = {}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	435 self.pwd: dict[str, str] = {}
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	436 self.callee: jid.JID \| None = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	437 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	438 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	439 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	440 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	441 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	442 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	443 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	444 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	445 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	446 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	447 self.video_valve = None
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	448 self.video_selector = None
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	449 if self.desktop_portal is not None:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	450 self.desktop_portal.end_session()
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	451 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	452 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	453 self.bindings = {}
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	454 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	455 self.reset_cb()
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	456 self.data_channels: dict[str, GstWebRTC.WebRTCDataChannel] = {}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	457
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	458 @property
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	459 def data_channel(self) -> GstWebRTC.WebRTCDataChannel:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	460 """Convenience method to get WebRTCDataChannel instance when there is only one."""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	461 if len(self.data_channels) != 1:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	462 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	463 "self.data_channel can only be used in a single Data Channel scenario. "
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	464 "Use self.data_channels dict instead."
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	465 )
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	466 return next(iter(self.data_channels.values()))
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	467
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	468 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	469 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	470 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	471 audio_pt: int \| None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	472 video_pt: int \| None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	473 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	474 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	475
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	476 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	477 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	478 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	479
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	480 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	481 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	482 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	483
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	484 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	485 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	486 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	487 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	488 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	489
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	490
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	491 if isinstance(self.sources_data, SourcesPipeline):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	492 if self.sources_data.video_pipeline!= "" and video_pt is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	493 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	494 if self.sources_data.audio_pipeline!= "" and audio_pt is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	495 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	496 elif isinstance(self.sources_data, SourcesNone):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	497 pass
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	498 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	499 if audio_pt is None or video_pt is None:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	500 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	501
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	502 match self.sources_data:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	503 case SourcesAuto():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	504 video_source_elt = VIDEO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	505 audio_source_elt = AUDIO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	506 case SourcesNone():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	507 video_source_elt = ""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	508 audio_source_elt = ""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	509 case SourcesPipeline() as source:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	510 if source.video_pipeline is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	511 video_source_elt = VIDEO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	512 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	513 video_source_elt = source.video_pipeline
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	514 if source.audio_pipeline is None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	515 audio_source_elt = AUDIO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	516 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	517 audio_source_elt = source.audio_pipeline
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	518 case SourcesTest():
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	519 video_source_elt = "videotestsrc is-live=true pattern=ball"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	520 audio_source_elt = "audiotestsrc"
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	521 case _:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	522 raise exceptions.InternalError(
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	523 f'Unexpected "sources_data" value: {self.sources_data!r}'
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	524 )
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	525
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	526 match self.sinks_data:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	527 case SinksApp():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	528 local_video_sink_elt = (
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	529 "appsink name=local_video_sink emit-signals=true drop=true "
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	530 "max-buffers=1 sync=True"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	531 )
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	532 case SinksAuto():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	533 if isinstance(self.sources_data, SourcesNone):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	534 local_video_sink_elt = ""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	535 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	536 local_video_sink_elt = "autovideosink"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	537 case SinksNone():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	538 local_video_sink_elt = ""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	539 case _:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	540 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	541 f'Unexpected "sinks_data" value {self.sinks_data!r}'
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	542 )
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	543
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	544 gst_pipe_elements = [
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	545 "webrtcbin latency=30 name=sendrecv bundle-policy=max-bundle"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	546 ]
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	547
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	548 if self.merge_pip and local_video_sink_elt:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	549 # Compositor is used to merge local video feedback in video sink, useful when
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	550 # we have only a single video sink.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	551 gst_pipe_elements.append(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	552 "compositor name=compositor background=black "
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	553 f"! video/x-raw,width={self.target_width},"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	554 f"height={self.target_height},framerate=30/1 "
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	555 f"! {local_video_sink_elt}"
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	556 )
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	557 local_video_sink_elt = "compositor.sink_1"
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	558
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	559 if video_source_elt:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	560 # Video source with an input-selector to switch between normal and video mute
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	561 # (or desktop sharing).
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	562 gst_pipe_elements.append(f"""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	563 input-selector name=video_selector
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	564 ! videorate drop-only=1 max-rate=30
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	565 ! video/x-raw,framerate=30/1
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	566 ! tee name=t
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	567
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	568 {video_source_elt} name=video_src
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	569 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	570 ! video_selector.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	571
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	572 videotestsrc name=muted_src is-live=true pattern=black
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	573 ! queue leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	574 ! video_selector.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	575
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	576 t.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	577 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	578 ! videoscale
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	579 ! videoconvert
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	580 ! vp8enc deadline=1 keyframe-max-dist=30
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	581 ! rtpvp8pay picture-id-mode=15-bit
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	582 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	583 ! sendrecv.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	584 """)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	585
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	586 if local_video_sink_elt:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	587 # Local video feedback.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	588 gst_pipe_elements.append(f"""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	589 t.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	590 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	591 ! videoconvert
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	592 ! {local_video_sink_elt}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	593 """)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	594
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	595 if audio_source_elt:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	596 # Audio with a valve for muting.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	597 gst_pipe_elements.append(r"""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	598 {audio_source_elt} name=audio_src
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	599 ! valve
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	600 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	601 ! audioconvert
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	602 ! audioresample
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	603 ! opusenc audio-type=voice
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	604 ! rtpopuspay
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	605 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	606 ! sendrecv.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	607 """)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	608
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	609 self.gst_pipe_desc = "\n\n".join(gst_pipe_elements)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	610
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	611 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	612
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	613 # Create the pipeline
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	614 try:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	615 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	616 except Exception:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	617 log.exception("Can't parse pipeline")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	618 self.pipeline = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	619 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	620 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	621
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	622 if not isinstance(self.pipeline, Gst.Pipeline):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	623 # in the case of Data Channel there is a single element, and Gst.parse_launch
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	624 # doesn't create a Pipeline in this case, so we do it manually.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	625 pipeline = Gst.Pipeline()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	626 pipeline.add(self.pipeline)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	627 self.pipeline = pipeline
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	628
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	629 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	630 if self.webrtcbin is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	631 raise exceptions.InternalError("Can't get the pipeline.")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	632
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	633 # If video or audio sources are not created, ``get_by_name`` will return None.
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	634 self.video_src = self.pipeline.get_by_name("video_src")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	635 self.muted_src = self.pipeline.get_by_name("muted_src")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	636 self.video_selector = self.pipeline.get_by_name("video_selector")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	637 if self.video_src and isinstance(self.sources_data, SourcesPipeline):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	638 for name, value in self.sources_data.video_properties.items():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	639 self.video_src.set_property(name, value)
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	640
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	641 self.audio_src = self.pipeline.get_by_name("audio_src")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	642 if self.audio_src and isinstance(self.sources_data, SourcesPipeline):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	643 for name, value in self.sources_data.audio_properties.items():
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	644 self.audio_src.set_property(name, value)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	645
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	646 self.audio_valve = self.pipeline.get_by_name("audio_valve")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	647
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	648 if self.video_muted:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	649 self.on_video_mute(True)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	650 if self.audio_muted:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	651 self.on_audio_mute(True)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	652
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	653 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	654 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	655 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	656 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	657
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	658 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	659 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	660 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	661 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	662 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	663 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	664 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	665 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	666 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	667 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	668 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	669 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	670 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	671 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	672 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	673 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	674 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	675 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	676
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	677 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	678 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	679
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	680 # local video feedback
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	681 if isinstance(self.sinks_data, SinksApp):
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	682 local_video_sink = self.pipeline.get_by_name("local_video_sink")
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	683 if local_video_sink is not None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	684 local_video_sink.set_property("emit-signals", True)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	685 local_video_sink.connect("new-sample", self.sinks_data.local_video_cb)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	686 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	687 local_video_sink.set_property("caps", local_video_sink_caps)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	688
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	689 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	690 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	691 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	692 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	693 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	694
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	695 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	696 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	697 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	698 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	699 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	700 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	701 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	702 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	703 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	704 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	705 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	706 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	707
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	708 for dc_data in self.dc_data_list:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	709 self.create_data_channel(dc_data)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	710
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	711 def create_data_channel(self, dc_data: SourcesDataChannel\|SinksDataChannel) -> None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	712 """Create a Data Channel and connect relevant callbacks."""
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	713 assert self.pipeline is not None
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	714 if isinstance(dc_data, SourcesDataChannel):
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	715 # Data channel configuration for compatibility with browser defaults
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	716 data_channel_options = Gst.Structure.new_empty("data-channel-options")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	717 data_channel_options.set_value("ordered", True)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	718 data_channel_options.set_value("protocol", "")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	719
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	720 # Create the data channel
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	721 self.pipeline.set_state(Gst.State.READY)
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	722 self.data_channels[dc_data.name] = data_channel = self.webrtcbin.emit(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	723 "create-data-channel", dc_data.name, data_channel_options
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	724 )
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	725 if data_channel is None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	726 log.error("Failed to create data channel")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	727 return
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	728 data_channel.connect("on-open", dc_data.dc_open_cb)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	729 elif isinstance(dc_data, SinksDataChannel):
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	730 self.webrtcbin.connect("on-data-channel", dc_data.dc_on_data_channel)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	731 else:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	732 raise ValueError(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	733 "Only SourcesDataChannel or SinksDataChannel are allowed."
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	734 )
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	735
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	736 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	737 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	738 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	739 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	740
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	741 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	742 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	743 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	744 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	745 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	746 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	747 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	748
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	749 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	750 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	751 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	752 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	753 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	754 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	755 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	756 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	757 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	758 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	759 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	760 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	761 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	762 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	763 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	764 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	765
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	766 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	767 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	768
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	769 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	770 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	771 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	772 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	773 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	774 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	775 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	776 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	777 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	778 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	779 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	780 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	781
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	782 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	783 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	784 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	785 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	786
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	787 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	788 self, source: Gst.Element, dest: Gst.Element \| Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	789 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	790 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	791 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	792
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	793 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	794 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	795 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	796 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	797 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	798 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	799 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	800 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	801 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	802 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	803 log.error(f"Unexpected type for dest: {type(dest)}")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	804 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	805
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	806 return True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	807
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	808 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	809 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	810 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	811 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	812
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	813 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	814 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	815 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	816 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	817 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	818 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	819 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	820 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	821
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	822 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	823 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	824
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	825 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	826 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	827
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	828 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	829 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	830
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	831 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	832 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	833 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	834
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	835 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	836 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	837 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	838 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	839 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	840 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	841
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	842 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	843 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	844 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	845 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	846 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	847
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	848 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	849
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	850 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	851 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	852
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	853 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	854
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	855 # Check and log the original size of the video
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	856 width = self.target_width
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	857 height = self.target_height
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	858 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	859
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	860 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	861 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	862 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	863 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	864 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	865 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	866 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	867 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	868 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	869
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	870 conv = Gst.ElementFactory.make("videoconvert")
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	871 if self.merge_pip:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	872 # with ``merge_pip`` set, we plug the remote stream to the composer
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	873 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	874
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	875 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	876
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	877 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	878 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	879 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	880 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	881 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	882 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	883
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	884 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	885 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	886 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	887 sink1_pad.set_property("height", local_height)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	888 sink1_pad.set_property("sizing-policy", 1)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	889 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	890
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	891 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	892 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	893 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	894 remote_video_sink.set_property("zorder", 0)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	895 remote_video_sink.set_property("width", width)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	896 remote_video_sink.set_property("height", height)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	897 remote_video_sink.set_property("sizing-policy", 1)
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	898 elif isinstance(self.sinks_data, SinksApp):
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	899 # ``app`` sink without ``self.merge_pip`` set, be create the sink and
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	900 # connect it to the ``remote_video_cb``.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	901 remote_video_sink = Gst.ElementFactory.make("appsink")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	902
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	903 remote_video_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	904 remote_video_sink.set_property("caps", remote_video_caps)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	905
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	906 remote_video_sink.set_property("emit-signals", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	907 remote_video_sink.set_property("drop", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	908 remote_video_sink.set_property("max-buffers", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	909 remote_video_sink.set_property("sync", True)
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	910 remote_video_sink.connect("new-sample", self.sinks_data.remote_video_cb)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	911 self.pipeline.add(remote_video_sink)
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	912 elif isinstance(self.sinks_data, SinksAuto):
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	913 # if ``self.merge_pip`` is not set, we create a dedicated
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	914 # ``autovideosink`` for remote stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	915 remote_video_sink = Gst.ElementFactory.make("autovideosink")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	916 self.pipeline.add(remote_video_sink)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	917 else:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	918 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	919 f'Unhandled "sinks_data" value: {self.sinks_data!r}'
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	920 )
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	921
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	922 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	923 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	924 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	925 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	926 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	927 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	928 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	929
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	930 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	931
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	932 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	933 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	934 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	935 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	936 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	937 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	938 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	939 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	940
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	941 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	942 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	943
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	944 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	945 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	946 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	947 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	948 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	949 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	950
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	951 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	952 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	953 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	954 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	955 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	956 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	957 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	958 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	959 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	960 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	961 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	962 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	963 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	964
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	965 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	966 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	967
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	968 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	969 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	970
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	971 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	972 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	973
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	974 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	975 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	976 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	977 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	978 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	979 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	980
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	981 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	982 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	983 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	984 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	985 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	986
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	987 async def _call_start(self, callee: jid.JID, call_data: dict, profile: str) -> str:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	988 return await self.bridge.call_start(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	989 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	990 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	991
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	992 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	993 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	994
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	995 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	996 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	997 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	998 assert self.callee
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	999 assert self.call_start_cb is not None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1000 self.sid = await self.call_start_cb(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1001 self.callee, {"sdp": self.offer}, self.profile
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1002 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1003 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1004 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1005 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1006 )
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1007 if not self.pwd:
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1008 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1009 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1010 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1011 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1012 ice_data[media_type] = {
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1013 "ufrag": self.ufrag[media_type],
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1014 "pwd": self.pwd[media_type],
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1015 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1016 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1017 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1018 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1019 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1020 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1021
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1022 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1023 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1024 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1025
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1026 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1027 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1028
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1029 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1030 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1031 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1032 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1033
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1034 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1035 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1036 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1037 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1038 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1039 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1040
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1041 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1042 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1043
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1044 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1045 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1046
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1047 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1048 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1049 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1050 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1051 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1052 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1053 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1054 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1055 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1056 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1057 assert "OPUS" in payload_types
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1058 try:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1059 await self.setup_call(
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1060 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1061 )
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1062 except Exception:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1063 log.exception("Can't setup call")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1064 raise
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1065 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1066 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1067 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1068 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1069 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1070 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1071
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1072 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1073 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1074
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1075 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1076 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1077 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1078 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1079 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1080 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1081 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1082 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1083 if parsed_candidate is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1084 log.warning(f"Can't parse candidate: {candidate_sdp}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1085 return
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1086 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1087 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1088 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1089 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1090
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1091 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1092 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1093 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1094 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1095 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1096 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1097 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1098 assert sdp is not None
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1099 self.extract_ufrag_pwd(sdp)
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1100 ice_data = {
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1101 "ufrag": self.ufrag[media_type],
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1102 "pwd": self.pwd[media_type],
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1103 "candidates": [parsed_candidate]
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1104 }
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1105 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1106 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1107 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1108 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1109 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1110 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1111
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1112 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1113 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1114
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1115 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1116 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1117
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1118 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1119 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1120 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1121 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1122 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1123 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1124 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1125 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1126 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1127 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1128 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1129 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1130 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1131 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1132 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1133 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1134 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1135 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1136 log.warning(
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1137 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1138 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1139 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1140
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1141 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1142 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1143 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1144
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1145 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1146 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1147 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1148 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1149 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1150
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1151 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1152 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1153
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1154 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1155 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1156 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1157 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1158 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1159 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1160
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1161 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1162 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1163
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1164 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1165 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1166 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1167 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1168
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1169 def on_audio_mute(self, muted: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1170 """Handles (un)muting of audio.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1171
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1172 @param muted: True if audio is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1173 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1174 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1175 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1176 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1177 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1178
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1179 def on_video_mute(self, muted: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1180 """Handles (un)muting of video.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1181
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1182 @param muted: True if video is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1183 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1184 if self.video_selector is not None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1185 current_source = (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1186 None if muted else "desktop" if self.desktop_sharing else "video"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1187 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1188 self.switch_video_source(current_source)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1189 state = "muted" if muted else "unmuted"
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1190 log.info(f"Video is now {state}")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1191
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1192 def on_desktop_switch(self, desktop_active: bool) -> None:
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1193 """Switches the video source between desktop and video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1194
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1195 @param desktop_active: True if desktop must be active. False for video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1196 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1197 if desktop_active and self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1198 aio.run_async(self.on_desktop_switch_portal(desktop_active))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1199 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1200 self.do_desktop_switch(desktop_active)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1201
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1202 async def on_desktop_switch_portal(self, desktop_active: bool) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1203 """Call freedesktop screenshare portal and the activate the shared stream"""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1204 assert self.desktop_portal is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1205 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1206 screenshare_data = await self.desktop_portal.request_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1207 except exceptions.CancelError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1208 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1209 return
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1210 self.desktop_sharing_data = {"path": str(screenshare_data["node_id"])}
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1211 self.do_desktop_switch(desktop_active)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1212
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1213 def on_portal_session_closed(self) -> None:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1214 self.desktop_sharing = False
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1215
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1216 def do_desktop_switch(self, desktop_active: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1217 if self.video_muted:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1218 # Update the active source state but do not switch
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1219 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1220 return
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1221
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1222 source = "desktop" if desktop_active else "video"
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1223 self.switch_video_source(source)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1224 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1225
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1226 def switch_video_source(self, source: str \| None) -> None:
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1227 """Activates the specified source while deactivating the others.
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1228
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1229 @param source: 'desktop', 'video', 'muted' or None for muted source.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1230 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1231 if source is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1232 source = "muted"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1233 if source not in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1234 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1235 f"Invalid source: {source!r}, use one of {'video', 'muted', 'desktop'}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1236 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1237
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1238 self.pipeline.set_state(Gst.State.PAUSED)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1239
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1240 # Create a new desktop source if necessary
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1241 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1242 self._setup_desktop_source(self.desktop_sharing_data)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1243
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1244 # Activate the chosen source and deactivate the others
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1245 for src_name in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1246 src_element = self.pipeline.get_by_name(f"{src_name}_src")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1247 if src_name == source:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1248 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1249 src_element.set_state(Gst.State.PLAYING)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1250 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1251 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1252 if src_name == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1253 self._remove_desktop_source(src_element)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1254 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1255 src_element.set_state(Gst.State.NULL)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1256
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1257 # Set the video_selector active pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1258 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1259 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1260 pad = self.desktop_sink_pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1261 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1262 log.error(f"No desktop pad available")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1263 pad = None
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1264 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1265 pad_name = f"sink_{['video', 'muted'].index(source)}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1266 pad = self.video_selector.get_static_pad(pad_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1267
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1268 if pad is not None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1269 self.video_selector.props.active_pad = pad
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1270
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1271 self.pipeline.set_state(Gst.State.PLAYING)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1272
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1273 def _setup_desktop_source(self, properties: dict[str, object] \| None) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1274 """Set up a new desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1275
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1276 @param properties: The properties to set on the desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1277 """
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1278 source_elt = "ximagesrc" if self.desktop_portal is None else "pipewiresrc"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1279 desktop_src = Gst.ElementFactory.make(source_elt, "desktop_src")
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1280 if properties is None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1281 properties = {}
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1282 for key, value in properties.items():
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1283 log.debug(f"setting {source_elt} property: {key!r}={value!r}")
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1284 desktop_src.set_property(key, value)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1285 video_convert = Gst.ElementFactory.make("videoconvert", "desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1286 queue = Gst.ElementFactory.make("queue", "desktop_queue")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1287 queue.set_property("leaky", "downstream")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1288
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1289 self.pipeline.add(desktop_src)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1290 self.pipeline.add(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1291 self.pipeline.add(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1292
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1293 desktop_src.link(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1294 video_convert.link(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1295
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1296 sink_pad_template = self.video_selector.get_pad_template("sink_%u")
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1297 self.desktop_sink_pad = self.video_selector.request_pad(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1298 sink_pad_template, None, None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1299 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1300 queue_src_pad = queue.get_static_pad("src")
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1301 queue_src_pad.link(self.desktop_sink_pad)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1302
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1303 desktop_src.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1304 video_convert.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1305 queue.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1306
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1307 def _remove_desktop_source(self, desktop_src: Gst.Element) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1308 """Remove the desktop source from the pipeline.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1309
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1310 @param desktop_src: The desktop source to remove.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1311 """
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1312 # Remove elements for the desktop source
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1313 video_convert = self.pipeline.get_by_name("desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1314 queue = self.pipeline.get_by_name("desktop_queue")
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1315
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1316 if video_convert:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1317 video_convert.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1318 desktop_src.unlink(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1319 self.pipeline.remove(video_convert)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1320
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1321 if queue:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1322 queue.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1323 self.pipeline.remove(queue)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1324
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1325 desktop_src.set_state(Gst.State.NULL)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1326 self.pipeline.remove(desktop_src)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1327
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1328 # Release the pad associated with the desktop source
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1329 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1330 self.video_selector.release_request_pad(self.desktop_sink_pad)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1331 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1332
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1333 if self.desktop_portal is not None:
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1334 self.desktop_portal.end_session()
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1335
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1336 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1337 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1338 self.reset_instance()
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1339
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1340
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1341 class WebRTCCall:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1342 """Helper class to create and handle WebRTC.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1343
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1344 This class handles signals and communication of connection data with backend.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1345
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1346 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1347
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1348 def __init__(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1349 self,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1350 bridge,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1351 profile: str,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1352 callee: jid.JID,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1353 on_call_setup_cb: Callable \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1354 on_call_ended_cb: Callable \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1355 **kwargs,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1356 ):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1357 """Create and setup a webRTC instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1358
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1359 @param bridge: async Bridge.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1360 @param profile: profile making or receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1361 @param callee: peer jid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1362 @param kwargs: extra kw args to use when instantiating WebRTC
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1363 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1364 self.profile = profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1365 self.webrtc = WebRTC(bridge, profile, **kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1366 self.webrtc.callee = callee
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1367 self.on_call_setup_cb = on_call_setup_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1368 self.on_call_ended_cb = on_call_ended_cb
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1369 bridge.register_signal("ice_candidates_new", self.on_ice_candidates_new, "plugin")
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1370 bridge.register_signal("call_setup", self.on_call_setup, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1371 bridge.register_signal("call_ended", self.on_call_ended, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1372
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1373 @classmethod
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1374 async def make_webrtc_call(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1375 cls, bridge, profile: str, call_data: CallData, **kwargs
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1376 ) -> "WebRTCCall":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1377 """Create the webrtc_call instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1378
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1379 @param call_data: Call data of the command
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1380 @param kwargs: extra args used to instanciate WebRTCCall
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1381
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1382 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1383 webrtc_call = cls(bridge, profile, call_data.callee, call_data.kwargs, kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1384 if call_data.sid is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1385 # we are making the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1386 await webrtc_call.start()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1387 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1388 # we are receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1389 webrtc_call.sid = call_data.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1390 if call_data.action_id is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1391 await bridge.action_launch(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1392 call_data.action_id,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1393 data_format.serialise({"cancelled": False}),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1394 profile,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1395 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1396 return webrtc_call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1397
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1398 @property
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1399 def sid(self) -> str \| None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1400 return self.webrtc.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1401
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1402 @sid.setter
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1403 def sid(self, new_sid: str \| None) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1404 self.webrtc.sid = new_sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1405
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1406 async def on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1407 self, sid: str, candidates_s: str, profile: str
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1408 ) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1409 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1410 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1411 self.webrtc.on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1412 data_format.deserialise(candidates_s),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1413 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1414
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1415 async def on_call_setup(self, sid: str, setup_data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1416 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1417 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1418 setup_data = data_format.deserialise(setup_data_s)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1419 try:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1420 role = setup_data["role"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1421 sdp = setup_data["sdp"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1422 except KeyError:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1423 log.error(f"Invalid setup data received: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1424 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1425 if role == "initiator":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1426 self.webrtc.on_accepted_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1427 elif role == "responder":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1428 await self.webrtc.answer_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1429 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1430 log.error(f"Invalid role received during setup: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1431 if self.on_call_setup_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1432 await aio.maybe_async(self.on_call_setup_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1433
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1434 async def on_call_ended(self, sid: str, data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1435 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1436 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1437 await self.webrtc.end_call()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1438 if self.on_call_ended_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1439 await aio.maybe_async(self.on_call_ended_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1440
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1441 async def start(self):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1442 """Start a call.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1443
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1444 To be used only if we are initiator
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1445 """
4240 79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1446 try:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1447 await self.webrtc.setup_call("initiator")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1448 except Exception:
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1449 log.exception("Can't setup call")
79c8a70e1813 backend, frontend: prepare remote control: Goffi <goffi@goffi.org> parents: 4233 diff changeset	1450 raise
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1451 self.webrtc.start_pipeline()

Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4242:8acf46ed7f36