libervia-backend: libervia/frontends/tools/webrtc.py annotate

annotate libervia/frontends/tools/webrtc.py @ 4180:b86912d3fd33

plugin IP: fix use of legacy URL + coroutine use: An https:/salut-a-toi.org URL was used to retrieve external IP, but it's not valid anymore, resulting in an exception. This feature is currently disabled. Also moved several methods from legacy inline callbacks to coroutines.

author	Goffi <goffi@goffi.org>
date	Sat, 09 Dec 2023 14:30:54 +0100
parents	60d107f2178a
children	879bad48cc2d

rev	line source
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	2
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	5
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	10
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	15
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	18
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	19 import gi
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	20 gi.require_versions({
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	21 "Gst": "1.0",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	22 "GstWebRTC": "1.0"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	23 })
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	24 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	25
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	26 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	27
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	28 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	29 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	30 except ImportError:
4145 970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	31 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	32 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing: Goffi <goffi@goffi.org> parents: 4139 diff changeset	33 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	34 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	35 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	36 from dataclasses import dataclass
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	37 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	38 import logging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	39 import re
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	40 from typing import Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	41 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	42
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	43 from libervia.backend.tools.common import data_format
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	44 from libervia.frontends.tools import aio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	45
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	46
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	47 log = logging.getLogger(__name__)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	48
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	49 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	50
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	51 SOURCES_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	52 SOURCES_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	53 SINKS_APP = "app"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	54 SINKS_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	55 SINKS_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	56
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	57
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	58 @dataclass
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	59 class AppSinkData:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	60 local_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	61 remote_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	62
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	63
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	64 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	65 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	66
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	67 This class encapsulates the WebRTC functionalities required for initiating and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	68 handling audio and video calls.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	69 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	70
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	71 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	72 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	73 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	74 profile: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	75 sources: str = SOURCES_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	76 sinks: str = SINKS_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	77 appsink_data: AppSinkData \| None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	78 reset_cb: Callable \| None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	79 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	80 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	81 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	82 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	83 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	84 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	85 self._video_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	86 self.sources = sources
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	87 self.sinks = sinks
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	88 if sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	89 self.appsink_data = appsink_data
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	90 elif appsink_data is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	91 raise exceptions.InternalError(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	92 "appsink_data can only be used for SINKS_APP sinks"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	93 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	94 self.reset_cb = reset_cb
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	95 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	96
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	97 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	98 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	99 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	100
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	101 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	102 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	103 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	104 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	105 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	106
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	107 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	108 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	109 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	110
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	111 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	112 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	113 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	114 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	115 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	116
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	117 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	118 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	119 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	120
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	121 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	122 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	123 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	124 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	125 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	126 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	127 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	128
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	129 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	130 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	131 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	132 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	133 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	134
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	135 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	136 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	137 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	138 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	139
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	140 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	141 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	142 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	143 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	144 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	145
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	146 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	147 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	148 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	149 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	150 with_timestamp: bool = True,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	151 bin_: Gst.Bin\|None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	152 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	153 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	154
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	155 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	156 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	157 See
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	158 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	159 for details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	160
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	161 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	162 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	163 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	164 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	165 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	166 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	167 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	168 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	169 if with_timestamp:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	170 timestamp = datetime.now().isoformat(timespec='milliseconds')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	171 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	172
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	173 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	174
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	175 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	176 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	177
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	178 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	179 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	180 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	181 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	182 return index
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	183 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	184
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	185 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	186 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	187
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	188 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	189 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	190 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	191 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	192 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	193
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	194 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	195 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	196 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	197 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	198
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	199 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	200
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	201 def _a_call(self, method, args, *kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	202 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	203 aio.run_from_thread(method, args, *kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	204
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	205 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	206 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	207 ) -> dict[str, int \| None]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	208 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	209
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	210 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	211 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	212 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	213 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	214 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	215 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	216 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	217 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	218 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	219 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	220 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	221 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	222 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	223 continue
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	224 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	225 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	226 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	227 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	228 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	229 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	230 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	231 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	232 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	233
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	234 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	235 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	236
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	237 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	238 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	239 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	240 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	241 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	242 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	243 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	244 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	245 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	246 if match:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	247 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	248
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	249 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	250 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	251 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	252 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	253
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	254 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	255 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	256
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	257 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	258 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	259
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	260 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	261 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	262 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	263 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	264 return None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	265
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	266 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	267 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	268
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	269 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	270 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	271 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	272 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	273 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	274 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	275
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	276 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	277 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	278
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	279 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	280 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	281
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	282 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	283
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	284 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	285 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	286
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	287 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	288 @return: ufrag and pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	289 @raise ValueError: Can't extract ufrag and password
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	290 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	291 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	292 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	293
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	294 if ufrag_line and pwd_line:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	295 ufrag = self.ufrag = ufrag_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	296 pwd = self.pwd = pwd_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	297 return ufrag, pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	298 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	299 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	300 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	301
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	302 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	303 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	304 self.role: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	305 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	306 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	307 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	308 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	309 self.sid: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	310 self.offer: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	311 self.local_candidates_buffer = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	312 self.ufrag: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	313 self.pwd: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	314 self.callee: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	315 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	316 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	317 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	318 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	319 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	320 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	321 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	322 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	323 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	324 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	325 self.video_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	326 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	327 self.reset_cb()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	328
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	329
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	330 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	331 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	332 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	333 audio_pt: int \| None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	334 video_pt: int \| None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	335 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	336 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	337
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	338 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	339 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	340 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	341
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	342 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	343 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	344 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	345
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	346 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	347 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	348 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	349 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	350 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	351 if audio_pt is None or video_pt is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	352 raise NotImplementedError("None value is not handled yet")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	353
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	354 if self.sources == SOURCES_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	355 video_source_elt = "v4l2src"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	356 audio_source_elt = "pulsesrc"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	357 elif self.sources == SOURCES_TEST:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	358 video_source_elt = "videotestsrc is-live=true pattern=ball"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	359 audio_source_elt = "audiotestsrc"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	360 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	361 raise exceptions.InternalError(f'Unknown "sources" value: {self.sources!r}')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	362
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	363 extra_elt = ""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	364
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	365 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	366 local_video_sink_elt = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	367 "appsink name=local_video_sink emit-signals=true drop=true max-buffers=1 "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	368 "sync=True"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	369 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	370 elif self.sinks == SINKS_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	371 extra_elt = "compositor name=compositor ! autovideosink"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	372 local_video_sink_elt = """compositor.sink_1"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	373 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	374 raise exceptions.InternalError(f"Unknown sinks value {self.sinks!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	375
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	376 self.gst_pipe_desc = f"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	377 webrtcbin latency=100 name=sendrecv bundle-policy=max-bundle
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	378
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	379 input-selector name=video_selector
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	380 ! videorate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	381 ! video/x-raw,framerate=30/1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	382 ! tee name=t
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	383
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	384 {extra_elt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	385
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	386 {video_source_elt} name=video_src ! queue leaky=downstream ! video_selector.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	387 videotestsrc is-live=true pattern=black ! queue leaky=downstream ! video_selector.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	388
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	389 t.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	390 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	391 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	392 ! vp8enc deadline=1 keyframe-max-dist=60
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	393 ! rtpvp8pay picture-id-mode=15-bit
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	394 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	395 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	396
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	397 t.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	398 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	399 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	400 ! {local_video_sink_elt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	401
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	402 {audio_source_elt} name=audio_src
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	403 ! valve
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	404 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	405 ! audioconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	406 ! audioresample
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	407 ! opusenc audio-type=voice
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	408 ! rtpopuspay
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	409 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	410 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	411 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	412
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	413 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	414
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	415 # Create the pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	416 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	417 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	418 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	419
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	420 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	421 self.video_src = self.pipeline.get_by_name("video_src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	422 self.video_selector = self.pipeline.get_by_name("video_selector")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	423 self.audio_valve = self.pipeline.get_by_name("audio_valve")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	424
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	425 if self.video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	426 self.on_video_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	427 if self.audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	428 self.on_audio_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	429
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	430 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	431 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	432 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	433 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	434
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	435 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	436 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	437 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	438 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	439 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	440 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	441 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	442 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	443 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	444 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	445 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	446 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	447 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	448 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	449 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	450 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	451 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	452 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	453
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	454 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	455 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	456
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	457 # local video feedback
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	458 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	459 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	460 local_video_sink = self.pipeline.get_by_name("local_video_sink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	461 local_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	462 local_video_sink.connect("new-sample", self.appsink_data.local_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	463 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	464 local_video_sink.set_property("caps", local_video_sink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	465
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	466 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	467 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	468 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	469 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	470 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	471
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	472 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	473 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	474 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	475 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	476 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	477 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	478 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	479 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	480 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	481 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	482 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	483 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	484
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	485 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	486 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	487 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	488 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	489
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	490 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	491 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	492 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	493 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	494 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	495 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	496 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	497
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	498 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	499 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	500 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	501 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	502 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	503 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	504 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	505 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	506 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	507 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	508 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	509 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	510 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	511 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	512 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	513 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	514
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	515 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	516 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	517
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	518 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	519 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	520 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	521 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	522 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	523 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	524 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	525 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	526 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	527 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	528 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	529 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	530
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	531 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	532 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	533 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	534 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	535
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	536 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	537 self, source: Gst.Element, dest: Gst.Element \| Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	538 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	539 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	540 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	541
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	542 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	543 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	544 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	545 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	546 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	547 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	548 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	549 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	550 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	551 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	552 log.error(f"Unexpected type for dest: {type(sink)}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	553 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	554
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	555 return True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	556
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	557 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	558 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	559 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	560 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	561
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	562 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	563 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	564 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	565 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	566 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	567 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	568 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	569 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	570
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	571 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	572 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	573
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	574 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	575 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	576
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	577 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	578 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	579
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	580 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	581 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	582 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	583
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	584 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	585 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	586 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	587 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	588 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	589 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	590
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	591 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	592 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	593 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	594 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	595 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	596
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	597 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	598
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	599 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	600 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	601
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	602 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	603
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	604 # Check and log the original size of the video
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	605 width = s.get_int("width").value
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	606 height = s.get_int("height").value
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	607 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	608
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	609 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	610 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	611 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	612 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	613 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	614 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	615 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	616 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	617 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	618
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	619 conv = Gst.ElementFactory.make("videoconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	620 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	621 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	622 remote_video_sink = Gst.ElementFactory.make("appsink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	623
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	624 appsink_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	625 remote_video_sink.set_property("caps", appsink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	626
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	627 remote_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	628 remote_video_sink.set_property("drop", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	629 remote_video_sink.set_property("max-buffers", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	630 remote_video_sink.set_property("sync", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	631 remote_video_sink.connect("new-sample", self.appsink_data.remote_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	632 self.pipeline.add(remote_video_sink)
4146 60d107f2178a frontends (tools/webrtc): fix `SINKS_APP` condition check. Goffi <goffi@goffi.org> parents: 4145 diff changeset	633 elif self.sinks == SINKS_AUTO:
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	634 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	635
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	636 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	637
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	638 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	639 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	640 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	641 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	642 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	643 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	644
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	645 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	646 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	647 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	648 sink1_pad.set_property("height", local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	649 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	650
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	651 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	652 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	653 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	654 remote_video_sink.set_property("zorder", 0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	655
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	656 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	657 raise exceptions.InternalError(f'Unhandled "sinks" value: {self.sinks!r}')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	658
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	659 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	660 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	661 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	662 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	663 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	664 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	665 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	666
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	667 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	668
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	669
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	670 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	671 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	672 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	673 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	674 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	675 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	676 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	677 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	678
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	679 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	680 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	681
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	682 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	683 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	684 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	685 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	686 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	687 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	688
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	689 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	690 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	691 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	692 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	693 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	694 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	695 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	696 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	697 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	698 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	699 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	700 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	701 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	702
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	703 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	704 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	705
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	706 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	707 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	708
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	709 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	710 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	711
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	712 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	713 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	714 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	715 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	716 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	717 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	718
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	719 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	720 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	721 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	722 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	723 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	724
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	725 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	726 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	727
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	728 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	729 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	730 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	731 assert self.callee
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	732 self.sid = await self.bridge.call_start(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	733 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	734 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	735 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	736 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	737 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	738 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	739 if self.pwd is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	740 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	741 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	742 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	743 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	744 ice_data[media_type] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	745 "ufrag": self.ufrag,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	746 "pwd": self.pwd,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	747 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	748 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	749 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	750 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	751 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	752 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	753
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	754 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	755 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	756 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	757
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	758 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	759 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	760
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	761 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	762 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	763 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	764 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	765
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	766 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	767 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	768 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	769 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	770 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	771 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	772
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	773 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	774 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	775
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	776 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	777 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	778
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	779 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	780 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	781 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	782 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	783 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	784 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	785 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	786 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	787 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	788 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	789 assert "OPUS" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	790 await self.setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	791 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	792 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	793 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	794 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	795 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	796 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	797 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	798 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	799
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	800 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	801 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	802
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	803 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	804 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	805 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	806 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	807 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	808 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	809 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	810 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	811 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	812 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	813 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	814 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	815
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	816 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	817 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	818 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	819 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	820 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	821 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	822 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	823 assert sdp is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	824 ufrag, pwd = self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	825 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	826 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	827 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	828 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	829 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	830 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	831 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	832
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	833 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	834 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	835
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	836 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	837 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	838
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	839 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	840 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	841 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	842 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	843 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	844 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	845 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	846 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	847 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	848 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	849 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	850 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	851 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	852 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	853 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	854 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	855 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	856 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	857 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	858 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	859 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	860 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	861
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	862 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	863 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	864 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	865
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	866 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	867 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	868 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	869 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	870 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	871
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	872 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	873 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	874
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	875 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	876 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	877 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	878 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	879 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	880 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	881
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	882 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	883 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	884
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	885 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	886 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	887 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	888 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	889
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	890 def on_audio_mute(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	891 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	892 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	893 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	894 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	895
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	896 def on_video_mute(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	897 if self.video_selector is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	898 # when muted, we switch to a black image and deactivate the camera
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	899 if not muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	900 self.video_src.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	901 pad = self.video_selector.get_static_pad("sink_1" if muted else "sink_0")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	902 self.video_selector.props.active_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	903 if muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	904 self.video_src.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	905 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	906 log.info(f"video is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	907
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	908 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	909 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	910 self.reset_instance()

Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4180:b86912d3fd33