libervia-backend: libervia/frontends/tools/webrtc.py annotate

annotate libervia/frontends/tools/webrtc.py @ 4233:d01b8d002619

cli (call, file), frontends: implement webRTC data channel transfer: - file send/receive commands now supports webRTC transfer. In `send` command, the `--webrtc` flags is currenty used to activate it. - WebRTC related code have been factorized and moved to `libervia.frontends.tools.webrtc*` modules. rel 442

author	Goffi <goffi@goffi.org>
date	Sat, 06 Apr 2024 13:43:09 +0200
parents	fe29fbdabce6
children	79c8a70e1813

rev	line source
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	2
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	5
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	10
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	15
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	18
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	19 from collections.abc import Awaitable
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	20 import gi
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	21
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	22 gi.require_versions({"Gst": "1.0", "GstWebRTC": "1.0"})
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	23 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	24
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	25 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	26
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	27 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	28 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	29 except ImportError:
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	30 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	31 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	32 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	33 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	34 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	35 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	36 import logging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	37 import re
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	38 from typing import Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	39 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	40
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	41 from libervia.backend.tools.common import data_format
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	42 from libervia.frontends.tools import aio, display_servers, jid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	43 from .webrtc_models import AppSinkData, CallData
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	44 from .webrtc_screenshare import DesktopPortal
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	45
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	46 current_server = display_servers.detect()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	47 if current_server == display_servers.X11:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	48 # GSTreamer's ximagesrc documentation asks to run this function
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	49 import ctypes
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	50
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	51 ctypes.CDLL("libX11.so.6").XInitThreads()
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	52
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	53
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	54 log = logging.getLogger(__name__)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	55
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	56 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	57
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	58 SOURCES_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	59 SOURCES_TEST = "test"
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	60 SOURCES_DATACHANNEL = "datachannel"
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	61 SINKS_APP = "app"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	62 SINKS_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	63 SINKS_TEST = "test"
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	64 SINKS_DATACHANNEL = "datachannel"
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	65
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	66
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	67 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	68 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	69
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	70 This class encapsulates the WebRTC functionalities required for initiating and
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	71 handling audio and video calls, and data channels.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	72 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	73
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	74 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	75 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	76 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	77 profile: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	78 sources: str = SOURCES_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	79 sinks: str = SINKS_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	80 appsink_data: AppSinkData \| None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	81 reset_cb: Callable \| None = None,
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	82 merge_pip: bool \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	83 target_size: tuple[int, int] \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	84 call_start_cb: Callable[[str, dict, str], Awaitable[str]] \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	85 dc_open_cb: (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	86 Callable[[GstWebRTC.WebRTCDataChannel], Awaitable[None]] \| None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	87 ) = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	88 dc_on_data_channel: (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	89 Callable[[GstWebRTC.WebRTCDataChannel], Awaitable[None]] \| None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	90 ) = None,
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	91 ) -> None:
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	92 """Initializes a new WebRTC instance.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	93
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	94 @param bridge: An instance of backend bridge.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	95 @param profile: Libervia profile.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	96 @param sources: Which kind of source to use.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	97 @param sinks: Which kind of sinks to use.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	98 @param appsink_data: configuration data for appsink (when SINKS_APP is used). Must
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	99 not be used for other sinks.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	100 @param reset_cb: An optional Callable that is triggered on reset events. Can be
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	101 used to reset UI data on new calls.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	102 @param merge_pip: A boolean flag indicating whether Picture-in-Picture mode is
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	103 enabled. When PiP is used, local feedback is merged to remote video stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	104 Only one video stream is then produced (the local one).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	105 If None, PiP mode is selected automatically according to selected sink (it's
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	106 used for SINKS_AUTO only for now).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	107 @param target_size: Expected size of the final sink stream. Mainly use by composer
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	108 when ``merge_pip`` is set.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	109 None to autodetect (not real autodetection implemeted yet, default to
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	110 (1280,720)).
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	111 @param call_start_cb: Called when call is started.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	112 @param dc_open_cb: Called when Data Channel is open (for SOURCES_DATACHANNEL).
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	113 This callback will be run in a GStreamer thread.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	114 @param dc_open_cb: Called when Data Channel is created (for SINKS_DATACHANNEL).
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	115 This callback will be run in a GStreamer thread.
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	116 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	117 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	118 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	119 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	120 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	121 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	122 self._video_muted = False
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	123 self._desktop_sharing = False
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	124 self.desktop_sharing_data = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	125 self.sources = sources
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	126 self.sinks = sinks
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	127 if target_size is None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	128 target_size = (1280, 720)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	129 self.target_width, self.target_height = target_size
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	130 if merge_pip is None:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	131 merge_pip = sinks == SINKS_AUTO
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	132 self.merge_pip = merge_pip
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	133 if call_start_cb is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	134 call_start_cb = self._call_start
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	135 self.call_start_cb = call_start_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	136 if sources == SOURCES_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	137 assert dc_open_cb is not None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	138 self.dc_open_cb = dc_open_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	139 if sinks == SINKS_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	140 assert dc_on_data_channel is not None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	141 self.dc_on_data_channel = dc_on_data_channel
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	142 if sinks == SINKS_APP:
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	143 if (
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	144 merge_pip
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	145 and appsink_data is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	146 and appsink_data.remote_video_cb is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	147 ):
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	148 raise ValueError("Remote_video_cb can't be used when merge_pip is used!")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	149 self.appsink_data = appsink_data
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	150 elif appsink_data is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	151 raise exceptions.InternalError(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	152 "appsink_data can only be used for SINKS_APP sinks"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	153 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	154 self.reset_cb = reset_cb
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	155 if current_server == display_servers.WAYLAND:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	156 self.desktop_portal = DesktopPortal(self)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	157 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	158 self.desktop_portal = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	159 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	160
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	161 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	162 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	163 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	164
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	165 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	166 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	167 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	168 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	169 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	170
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	171 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	172 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	173 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	174
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	175 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	176 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	177 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	178 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	179 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	180
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	181 @property
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	182 def desktop_sharing(self):
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	183 return self._desktop_sharing
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	184
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	185 @desktop_sharing.setter
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	186 def desktop_sharing(self, active: bool) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	187 if active != self._desktop_sharing:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	188 self._desktop_sharing = active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	189 self.on_desktop_switch(active)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	190 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	191 cb = self.bindings["desktop_sharing"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	192 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	193 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	194 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	195 cb(active)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	196
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	197 @property
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	198 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	199 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	200
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	201 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	202 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	203 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	204 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	205 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	206 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	207 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	208
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	209 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	210 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	211 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	212 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	213 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	214
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	215 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	216 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	217 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	218 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	219
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	220 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	221 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	222 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	223 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	224 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	225
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	226 def bind(self, **kwargs: Callable) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	227 self.bindings.clear()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	228 for key, cb in kwargs.items():
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	229 if key not in ("desktop_sharing",):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	230 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	231 'Only "desktop_sharing" is currently allowed for binding'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	232 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	233 self.bindings[key] = cb
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	234
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	235 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	236 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	237 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	238 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	239 with_timestamp: bool = True,
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	240 bin_: Gst.Bin \| None = None,
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	241 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	242 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	243
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	244 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	245 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	246 See
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	247 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	248 for details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	249
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	250 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	251 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	252 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	253 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	254 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	255 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	256 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	257 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	258 if with_timestamp:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	259 timestamp = datetime.now().isoformat(timespec="milliseconds")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	260 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	261
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	262 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	263
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	264 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	265 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	266
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	267 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	268 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	269 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	270 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	271 return index
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	272 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	273
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	274 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	275 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	276
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	277 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	278 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	279 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	280 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	281 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	282
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	283 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	284 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	285 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	286 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	287
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	288 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	289
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	290 def _a_call(self, method, args, *kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	291 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	292 aio.run_from_thread(method, args, *kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	293
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	294 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	295 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	296 ) -> dict[str, int \| None]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	297 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	298
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	299 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	300 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	301 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	302 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	303 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	304 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	305 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	306 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	307 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	308 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	309 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	310 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	311 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	312 continue
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	313 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	314 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	315 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	316 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	317 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	318 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	319 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	320 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	321 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	322
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	323 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	324 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	325
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	326 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	327 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	328 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	329 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	330 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	331 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	332 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	333 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	334 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	335 if match:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	336 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	337
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	338 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	339 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	340 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	341 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	342
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	343 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	344 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	345
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	346 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	347 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	348
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	349 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	350 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	351 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	352 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	353 return None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	354
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	355 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	356 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	357
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	358 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	359 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	360 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	361 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	362 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	363 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	364
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	365 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	366 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	367
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	368 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	369 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	370
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	371 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	372
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	373 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	374 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	375
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	376 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	377 @return: ufrag and pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	378 @raise ValueError: Can't extract ufrag and password
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	379 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	380 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	381 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	382
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	383 if ufrag_line and pwd_line:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	384 ufrag = self.ufrag = ufrag_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	385 pwd = self.pwd = pwd_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	386 return ufrag, pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	387 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	388 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	389 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	390
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	391 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	392 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	393 self.role: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	394 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	395 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	396 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	397 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	398 self.sid: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	399 self.offer: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	400 self.local_candidates_buffer = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	401 self.ufrag: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	402 self.pwd: str \| None = None
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	403 self.callee: jid.JID \| None = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	404 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	405 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	406 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	407 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	408 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	409 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	410 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	411 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	412 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	413 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	414 self.video_valve = None
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	415 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	416 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	417 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	418 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	419 self.bindings = {}
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	420 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	421 self.reset_cb()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	422
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	423 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	424 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	425 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	426 audio_pt: int \| None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	427 video_pt: int \| None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	428 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	429 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	430
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	431 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	432 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	433 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	434
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	435 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	436 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	437 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	438
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	439 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	440 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	441 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	442 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	443 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	444
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	445 if self.sources == SOURCES_DATACHANNEL or self.sinks == SINKS_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	446 # Setup pipeline for datachannel only, no media streams.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	447 self.gst_pipe_desc = f"""
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	448 webrtcbin name=sendrecv bundle-policy=max-bundle
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	449 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	450 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	451 if audio_pt is None or video_pt is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	452 raise NotImplementedError("None value is not handled yet")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	453
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	454 if self.sources == SOURCES_AUTO:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	455 video_source_elt = "v4l2src"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	456 audio_source_elt = "pulsesrc"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	457 elif self.sources == SOURCES_TEST:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	458 video_source_elt = "videotestsrc is-live=true pattern=ball"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	459 audio_source_elt = "audiotestsrc"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	460 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	461 raise exceptions.InternalError(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	462 f'Unknown "sources" value: {self.sources!r}'
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	463 )
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	464
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	465 if self.sinks == SINKS_APP:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	466 local_video_sink_elt = (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	467 "appsink name=local_video_sink emit-signals=true drop=true max-buffers=1 "
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	468 "sync=True"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	469 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	470 elif self.sinks == SINKS_AUTO:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	471 local_video_sink_elt = "autovideosink"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	472 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	473 raise exceptions.InternalError(f"Unknown sinks value {self.sinks!r}")
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	474
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	475 if self.merge_pip:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	476 extra_elt = (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	477 "compositor name=compositor background=black "
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	478 f"! video/x-raw,width={self.target_width},height={self.target_height},"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	479 "framerate=30/1 "
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	480 f"! {local_video_sink_elt}"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	481 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	482 local_video_sink_elt = "compositor.sink_1"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	483 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	484 extra_elt = ""
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	485
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	486 self.gst_pipe_desc = f"""
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	487 webrtcbin latency=100 name=sendrecv bundle-policy=max-bundle
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	488
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	489 input-selector name=video_selector
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	490 ! videorate
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	491 ! video/x-raw,framerate=30/1
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	492 ! tee name=t
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	493
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	494 {extra_elt}
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	495
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	496 {video_source_elt} name=video_src ! queue leaky=downstream ! video_selector.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	497 videotestsrc name=muted_src is-live=true pattern=black ! queue leaky=downstream ! video_selector.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	498
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	499 t.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	500 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	501 ! videoconvert
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	502 ! vp8enc deadline=1 keyframe-max-dist=60
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	503 ! rtpvp8pay picture-id-mode=15-bit
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	504 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	505 ! sendrecv.
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	506
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	507 t.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	508 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	509 ! videoconvert
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	510 ! {local_video_sink_elt}
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	511
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	512 {audio_source_elt} name=audio_src
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	513 ! valve
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	514 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	515 ! audioconvert
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	516 ! audioresample
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	517 ! opusenc audio-type=voice
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	518 ! rtpopuspay
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	519 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	520 ! sendrecv.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	521 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	522
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	523 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	524
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	525 # Create the pipeline
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	526 try:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	527 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	528 except Exception:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	529 log.exception("Can't parse pipeline")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	530 self.pipeline = None
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	531 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	532 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	533
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	534 if not isinstance(self.pipeline, Gst.Pipeline):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	535 # in the case of Data Channel there is a single element, and Gst.parse_launch
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	536 # doesn't create a Pipeline in this case, so we do it manually.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	537 pipeline = Gst.Pipeline()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	538 pipeline.add(self.pipeline)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	539 self.pipeline = pipeline
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	540
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	541 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	542 if self.webrtcbin is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	543 raise exceptions.InternalError("Can't get the pipeline.")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	544
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	545 # For datachannel setups, media source, selector, and sink elements are not
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	546 # created
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	547 if self.sources != SOURCES_DATACHANNEL and self.sinks != SINKS_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	548 self.video_src = self.pipeline.get_by_name("video_src")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	549 self.muted_src = self.pipeline.get_by_name("muted_src")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	550 self.video_selector = self.pipeline.get_by_name("video_selector")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	551 self.audio_valve = self.pipeline.get_by_name("audio_valve")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	552
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	553 if self.video_muted:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	554 self.on_video_mute(True)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	555 if self.audio_muted:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	556 self.on_audio_mute(True)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	557
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	558 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	559 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	560 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	561 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	562
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	563 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	564 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	565 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	566 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	567 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	568 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	569 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	570 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	571 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	572 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	573 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	574 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	575 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	576 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	577 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	578 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	579 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	580 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	581
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	582 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	583 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	584
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	585 # local video feedback
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	586 if self.sinks == SINKS_APP and self.sources != SOURCES_DATACHANNEL:
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	587 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	588 local_video_sink = self.pipeline.get_by_name("local_video_sink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	589 local_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	590 local_video_sink.connect("new-sample", self.appsink_data.local_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	591 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	592 local_video_sink.set_property("caps", local_video_sink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	593
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	594 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	595 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	596 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	597 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	598 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	599
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	600 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	601 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	602 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	603 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	604 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	605 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	606 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	607 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	608 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	609 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	610 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	611 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	612
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	613 if self.sources == SOURCES_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	614 # Data channel configuration for compatibility with browser defaults
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	615 data_channel_options = Gst.Structure.new_empty("data-channel-options")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	616 data_channel_options.set_value("ordered", True)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	617 data_channel_options.set_value("protocol", "")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	618
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	619 # Create the data channel
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	620 self.pipeline.set_state(Gst.State.READY)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	621 self.data_channel = self.webrtcbin.emit(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	622 "create-data-channel", "file", data_channel_options
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	623 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	624 if self.data_channel is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	625 log.error("Failed to create data channel")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	626 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	627 self.data_channel.connect("on-open", self.dc_open_cb)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	628 if self.sinks == SINKS_DATACHANNEL:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	629 self.webrtcbin.connect("on-data-channel", self.dc_on_data_channel)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	630
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	631 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	632 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	633 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	634 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	635
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	636 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	637 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	638 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	639 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	640 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	641 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	642 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	643
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	644 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	645 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	646 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	647 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	648 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	649 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	650 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	651 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	652 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	653 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	654 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	655 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	656 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	657 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	658 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	659 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	660
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	661 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	662 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	663
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	664 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	665 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	666 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	667 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	668 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	669 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	670 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	671 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	672 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	673 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	674 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	675 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	676
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	677 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	678 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	679 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	680 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	681
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	682 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	683 self, source: Gst.Element, dest: Gst.Element \| Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	684 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	685 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	686 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	687
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	688 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	689 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	690 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	691 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	692 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	693 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	694 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	695 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	696 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	697 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	698 log.error(f"Unexpected type for dest: {type(dest)}")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	699 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	700
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	701 return True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	702
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	703 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	704 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	705 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	706 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	707
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	708 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	709 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	710 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	711 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	712 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	713 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	714 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	715 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	716
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	717 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	718 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	719
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	720 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	721 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	722
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	723 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	724 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	725
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	726 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	727 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	728 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	729
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	730 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	731 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	732 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	733 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	734 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	735 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	736
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	737 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	738 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	739 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	740 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	741 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	742
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	743 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	744
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	745 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	746 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	747
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	748 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	749
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	750 # Check and log the original size of the video
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	751 width = self.target_width
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	752 height = self.target_height
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	753 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	754
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	755 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	756 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	757 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	758 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	759 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	760 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	761 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	762 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	763 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	764
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	765 conv = Gst.ElementFactory.make("videoconvert")
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	766 if self.merge_pip:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	767 # with ``merge_pip`` set, we plug the remote stream to the composer
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	768 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	769
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	770 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	771
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	772 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	773 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	774 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	775 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	776 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	777 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	778
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	779 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	780 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	781 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	782 sink1_pad.set_property("height", local_height)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	783 sink1_pad.set_property("sizing-policy", 1)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	784 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	785
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	786 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	787 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	788 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	789 remote_video_sink.set_property("zorder", 0)
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	790 remote_video_sink.set_property("width", width)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	791 remote_video_sink.set_property("height", height)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	792 remote_video_sink.set_property("sizing-policy", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	793 elif self.sinks == SINKS_APP:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	794 # ``app`` sink without ``self.merge_pip`` set, be create the sink and
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	795 # connect it to the ``remote_video_cb``.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	796 assert self.appsink_data is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	797 remote_video_sink = Gst.ElementFactory.make("appsink")
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	798
4209 fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	799 remote_video_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	800 remote_video_sink.set_property("caps", remote_video_caps)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	801
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	802 remote_video_sink.set_property("emit-signals", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	803 remote_video_sink.set_property("drop", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	804 remote_video_sink.set_property("max-buffers", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	805 remote_video_sink.set_property("sync", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	806 remote_video_sink.connect("new-sample", self.appsink_data.remote_video_cb)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	807 self.pipeline.add(remote_video_sink)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	808 elif self.sinks == SINKS_AUTO:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	809 # if ``self.merge_pip`` is not set, we create a dedicated
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	810 # ``autovideosink`` for remote stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	811 remote_video_sink = Gst.ElementFactory.make("autovideosink")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size: Goffi <goffi@goffi.org> parents: 4205 diff changeset	812 self.pipeline.add(remote_video_sink)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	813 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	814 raise exceptions.InternalError(f'Unhandled "sinks" value: {self.sinks!r}')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	815
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	816 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	817 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	818 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	819 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	820 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	821 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	822 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	823
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	824 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	825
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	826 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	827 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	828 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	829 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	830 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	831 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	832 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	833 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	834
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	835 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	836 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	837
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	838 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	839 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	840 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	841 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	842 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	843 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	844
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	845 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	846 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	847 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	848 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	849 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	850 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	851 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	852 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	853 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	854 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	855 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	856 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	857 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	858
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	859 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	860 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	861
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	862 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	863 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	864
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	865 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	866 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	867
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	868 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	869 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	870 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	871 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	872 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	873 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	874
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	875 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	876 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	877 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	878 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	879 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	880
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	881 async def _call_start(self, callee: jid.JID, call_data: dict, profile: str) -> str:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	882 return await self.bridge.call_start(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	883 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	884 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	885
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	886 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	887 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	888
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	889 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	890 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	891 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	892 assert self.callee
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	893 assert self.call_start_cb is not None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	894 self.sid = await self.call_start_cb(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	895 self.callee, {"sdp": self.offer}, self.profile
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	896 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	897 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	898 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	899 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	900 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	901 if self.pwd is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	902 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	903 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	904 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	905 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	906 ice_data[media_type] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	907 "ufrag": self.ufrag,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	908 "pwd": self.pwd,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	909 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	910 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	911 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	912 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	913 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	914 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	915
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	916 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	917 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	918 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	919
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	920 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	921 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	922
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	923 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	924 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	925 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	926 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	927
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	928 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	929 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	930 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	931 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	932 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	933 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	934
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	935 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	936 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	937
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	938 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	939 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	940
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	941 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	942 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	943 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	944 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	945 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	946 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	947 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	948 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	949 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	950 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	951 assert "OPUS" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	952 await self.setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	953 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	954 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	955 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	956 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	957 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	958 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	959 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	960 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	961
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	962 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	963 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	964
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	965 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	966 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	967 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	968 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	969 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	970 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	971 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	972 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	973 if parsed_candidate is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	974 log.warning(f"Can't parse candidate: {candidate_sdp}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	975 return
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	976 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	977 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	978 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	979 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	980
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	981 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	982 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	983 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	984 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	985 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	986 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	987 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	988 assert sdp is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	989 ufrag, pwd = self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	990 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	991 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	992 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	993 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	994 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	995 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	996 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	997
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	998 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	999 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1000
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1001 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1002 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1003
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1004 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1005 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1006 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1007 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1008 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1009 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1010 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1011 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1012 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1013 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1014 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1015 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1016 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1017 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1018 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1019 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1020 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1021 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1022 log.warning(
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1023 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1024 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1025 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1026
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1027 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1028 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1029 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1030
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1031 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1032 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1033 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1034 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1035 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1036
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1037 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1038 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1039
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1040 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1041 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1042 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1043 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1044 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1045 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1046
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1047 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1048 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1049
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1050 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1051 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1052 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1053 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1054
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1055 def on_audio_mute(self, muted: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1056 """Handles (un)muting of audio.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1057
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1058 @param muted: True if audio is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1059 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1060 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1061 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1062 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1063 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1064
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1065 def on_video_mute(self, muted: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1066 """Handles (un)muting of video.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1067
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1068 @param muted: True if video is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1069 """
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1070 if self.video_selector is not None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1071 current_source = (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1072 None if muted else "desktop" if self.desktop_sharing else "video"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1073 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1074 self.switch_video_source(current_source)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1075 state = "muted" if muted else "unmuted"
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1076 log.info(f"Video is now {state}")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1077
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1078 def on_desktop_switch(self, desktop_active: bool) -> None:
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1079 """Switches the video source between desktop and video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1080
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1081 @param desktop_active: True if desktop must be active. False for video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1082 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1083 if desktop_active and self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1084 aio.run_async(self.on_desktop_switch_portal(desktop_active))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1085 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1086 self.do_desktop_switch(desktop_active)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1087
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1088 async def on_desktop_switch_portal(self, desktop_active: bool) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1089 """Call freedesktop screenshare portal and the activate the shared stream"""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1090 assert self.desktop_portal is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1091 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1092 screenshare_data = await self.desktop_portal.request_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1093 except exceptions.CancelError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1094 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1095 return
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1096 self.desktop_sharing_data = {"path": str(screenshare_data["node_id"])}
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1097 self.do_desktop_switch(desktop_active)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1098
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1099 def do_desktop_switch(self, desktop_active: bool) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1100 if self.video_muted:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1101 # Update the active source state but do not switch
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1102 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1103 return
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1104
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1105 source = "desktop" if desktop_active else "video"
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1106 self.switch_video_source(source)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1107 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1108
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1109 def switch_video_source(self, source: str \| None) -> None:
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1110 """Activates the specified source while deactivating the others.
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1111
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1112 @param source: 'desktop', 'video', 'muted' or None for muted source.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1113 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1114 if source is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1115 source = "muted"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1116 if source not in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1117 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1118 f"Invalid source: {source!r}, use one of {'video', 'muted', 'desktop'}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1119 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1120
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1121 self.pipeline.set_state(Gst.State.PAUSED)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1122
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1123 # Create a new desktop source if necessary
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1124 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1125 self._setup_desktop_source(self.desktop_sharing_data)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1126
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1127 # Activate the chosen source and deactivate the others
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1128 for src_name in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1129 src_element = self.pipeline.get_by_name(f"{src_name}_src")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1130 if src_name == source:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1131 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1132 src_element.set_state(Gst.State.PLAYING)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1133 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1134 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1135 if src_name == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1136 self._remove_desktop_source(src_element)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1137 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1138 src_element.set_state(Gst.State.NULL)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1139
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1140 # Set the video_selector active pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1141 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1142 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1143 pad = self.desktop_sink_pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1144 else:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1145 log.error(f"No desktop pad available")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1146 pad = None
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1147 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1148 pad_name = f"sink_{['video', 'muted'].index(source)}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1149 pad = self.video_selector.get_static_pad(pad_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1150
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1151 if pad is not None:
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1152 self.video_selector.props.active_pad = pad
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1153
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1154 self.pipeline.set_state(Gst.State.PLAYING)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1155
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1156 def _setup_desktop_source(self, properties: dict[str, object] \| None) -> None:
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1157 """Set up a new desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1158
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1159 @param properties: The properties to set on the desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1160 """
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1161 source_elt = "ximagesrc" if self.desktop_portal is None else "pipewiresrc"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1162 desktop_src = Gst.ElementFactory.make(source_elt, "desktop_src")
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1163 if properties is None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1164 properties = {}
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1165 for key, value in properties.items():
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1166 log.debug(f"setting {source_elt} property: {key!r}={value!r}")
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1167 desktop_src.set_property(key, value)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1168 video_convert = Gst.ElementFactory.make("videoconvert", "desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1169 queue = Gst.ElementFactory.make("queue", "desktop_queue")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1170 queue.set_property("leaky", "downstream")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1171
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1172 self.pipeline.add(desktop_src)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1173 self.pipeline.add(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1174 self.pipeline.add(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1175
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1176 desktop_src.link(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1177 video_convert.link(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1178
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1179 sink_pad_template = self.video_selector.get_pad_template("sink_%u")
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1180 self.desktop_sink_pad = self.video_selector.request_pad(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1181 sink_pad_template, None, None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1182 )
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1183 queue_src_pad = queue.get_static_pad("src")
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1184 queue_src_pad.link(self.desktop_sink_pad)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1185
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1186 desktop_src.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1187 video_convert.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1188 queue.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1189
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1190 def _remove_desktop_source(self, desktop_src: Gst.Element) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1191 """Remove the desktop source from the pipeline.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1192
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1193 @param desktop_src: The desktop source to remove.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1194 """
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1195 # Remove elements for the desktop source
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1196 video_convert = self.pipeline.get_by_name("desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1197 queue = self.pipeline.get_by_name("desktop_queue")
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1198
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1199 if video_convert:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1200 video_convert.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1201 desktop_src.unlink(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1202 self.pipeline.remove(video_convert)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1203
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1204 if queue:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1205 queue.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1206 self.pipeline.remove(queue)
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1207
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1208 desktop_src.set_state(Gst.State.NULL)
4204 879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing: Goffi <goffi@goffi.org> parents: 4146 diff changeset	1209 self.pipeline.remove(desktop_src)
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1210
4205 17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1211 # Release the pad associated with the desktop source
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1212 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1213 self.video_selector.release_request_pad(self.desktop_sink_pad)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1214 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1215
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1216 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1217 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: Goffi <goffi@goffi.org> parents: 4204 diff changeset	1218
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1219 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1220 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1221 self.reset_instance()
4233 d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1222
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1223
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1224 class WebRTCCall:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1225 """Helper class to create and handle WebRTC.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1226
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1227 This class handles signals and communication of connection data with backend.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1228
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1229 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1230
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1231 def __init__(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1232 self,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1233 bridge,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1234 profile: str,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1235 callee: jid.JID,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1236 on_call_setup_cb: Callable \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1237 on_call_ended_cb: Callable \| None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1238 **kwargs,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1239 ):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1240 """Create and setup a webRTC instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1241
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1242 @param bridge: async Bridge.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1243 @param profile: profile making or receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1244 @param callee: peer jid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1245 @param kwargs: extra kw args to use when instantiating WebRTC
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1246 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1247 self.profile = profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1248 self.webrtc = WebRTC(bridge, profile, **kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1249 self.webrtc.callee = callee
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1250 self.on_call_setup_cb = on_call_setup_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1251 self.on_call_ended_cb = on_call_ended_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1252 bridge.register_signal(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1253 "ice_candidates_new", self.on_ice_candidates_new, "plugin"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1254 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1255 bridge.register_signal("call_setup", self.on_call_setup, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1256 bridge.register_signal("call_ended", self.on_call_ended, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1257
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1258 @classmethod
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1259 async def make_webrtc_call(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1260 cls, bridge, profile: str, call_data: CallData, **kwargs
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1261 ) -> "WebRTCCall":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1262 """Create the webrtc_call instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1263
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1264 @param call_data: Call data of the command
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1265 @param kwargs: extra args used to instanciate WebRTCCall
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1266
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1267 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1268 webrtc_call = cls(bridge, profile, call_data.callee, call_data.kwargs, kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1269 if call_data.sid is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1270 # we are making the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1271 await webrtc_call.start()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1272 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1273 # we are receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1274 webrtc_call.sid = call_data.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1275 if call_data.action_id is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1276 await bridge.action_launch(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1277 call_data.action_id,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1278 data_format.serialise({"cancelled": False}),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1279 profile,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1280 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1281 return webrtc_call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1282
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1283 @property
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1284 def sid(self) -> str \| None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1285 return self.webrtc.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1286
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1287 @sid.setter
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1288 def sid(self, new_sid: str \| None) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1289 self.webrtc.sid = new_sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1290
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1291 async def on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1292 self, sid: str, candidates_s: str, profile: str
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1293 ) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1294 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1295 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1296 self.webrtc.on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1297 data_format.deserialise(candidates_s),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1298 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1299
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1300 async def on_call_setup(self, sid: str, setup_data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1301 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1302 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1303 setup_data = data_format.deserialise(setup_data_s)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1304 try:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1305 role = setup_data["role"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1306 sdp = setup_data["sdp"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1307 except KeyError:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1308 log.error(f"Invalid setup data received: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1309 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1310 if role == "initiator":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1311 self.webrtc.on_accepted_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1312 elif role == "responder":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1313 await self.webrtc.answer_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1314 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1315 log.error(f"Invalid role received during setup: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1316 if self.on_call_setup_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1317 await aio.maybe_async(self.on_call_setup_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1318
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1319 async def on_call_ended(self, sid: str, data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1320 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1321 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1322 await self.webrtc.end_call()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1323 if self.on_call_ended_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1324 await aio.maybe_async(self.on_call_ended_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1325
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1326 async def start(self):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1327 """Start a call.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1328
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1329 To be used only if we are initiator
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1330 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1331 await self.webrtc.setup_call("initiator")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer: Goffi <goffi@goffi.org> parents: 4209 diff changeset	1332 self.webrtc.start_pipeline()

Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4233:d01b8d002619