libervia-backend: libervia/frontends/tools/webrtc.py annotate

annotate libervia/frontends/tools/webrtc.py @ 4142:783bbdbf8567

cli (base): new `a_quit` method to run async cleaning methods: it may be necessary to run some async cleaning method (e.g. if bridge is needed), this new method handles async cleaning method, and will be called when `SystemExit` is used, or if `a_quit` is called explicitely. rel 426

author	Goffi <goffi@goffi.org>
date	Wed, 01 Nov 2023 14:08:00 +0100
parents	6745c6bd4c7a
children	970b6209526a

rev	line source
4139 6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	2
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	5
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	10
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	15
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	18
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	19 import gi
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	20 gi.require_versions({
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	21 "Gst": "1.0",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	22 "GstWebRTC": "1.0"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	23 })
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	24 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	25
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	26 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	27 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	28 except ImportError:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	29 print(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	30 "no GStreamer python overrides available, please install relevant pacakges on "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	31 "your system."
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	32 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	33 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	34 from dataclasses import dataclass
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	35 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	36 import logging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	37 import re
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	38 from typing import Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	39 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	40
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	41 from libervia.backend.core import exceptions
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	42 from libervia.backend.tools.common import data_format
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	43 from libervia.frontends.tools import aio
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	44
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	45
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	46 log = logging.getLogger(__name__)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	47
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	48 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	49
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	50 SOURCES_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	51 SOURCES_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	52 SINKS_APP = "app"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	53 SINKS_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	54 SINKS_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	55
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	56
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	57 @dataclass
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	58 class AppSinkData:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	59 local_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	60 remote_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	61
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	62
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	63 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	64 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	65
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	66 This class encapsulates the WebRTC functionalities required for initiating and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	67 handling audio and video calls.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	68 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	69
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	70 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	71 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	72 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	73 profile: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	74 sources: str = SOURCES_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	75 sinks: str = SINKS_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	76 appsink_data: AppSinkData \| None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	77 reset_cb: Callable \| None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	78 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	79 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	80 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	81 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	82 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	83 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	84 self._video_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	85 self.sources = sources
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	86 self.sinks = sinks
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	87 if sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	88 self.appsink_data = appsink_data
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	89 elif appsink_data is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	90 raise exceptions.InternalError(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	91 "appsink_data can only be used for SINKS_APP sinks"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	92 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	93 self.reset_cb = reset_cb
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	94 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	95
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	96 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	97 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	98 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	99
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	100 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	101 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	102 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	103 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	104 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	105
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	106 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	107 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	108 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	109
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	110 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	111 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	112 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	113 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	114 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	115
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	116 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	117 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	118 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	119
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	120 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	121 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	122 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	123 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	124 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	125 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	126 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	127
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	128 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	129 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	130 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	131 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	132 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	133
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	134 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	135 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	136 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	137 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	138
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	139 @property
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	140 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	141 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	142 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	143 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	144
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	145 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	146 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	147 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	148 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	149 with_timestamp: bool = True,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	150 bin_: Gst.Bin\|None = None,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	151 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	152 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	153
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	154 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	155 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	156 See
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	157 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	158 for details.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	159
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	160 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	161 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	162 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	163 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	164 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	165 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	166 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	167 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	168 if with_timestamp:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	169 timestamp = datetime.now().isoformat(timespec='milliseconds')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	170 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	171
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	172 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	173
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	174 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	175 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	176
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	177 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	178 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	179 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	180 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	181 return index
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	182 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	183
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	184 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	185 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	186
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	187 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	188 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	189 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	190 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	191 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	192
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	193 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	194 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	195 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	196 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	197
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	198 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	199
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	200 def _a_call(self, method, args, *kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	201 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	202 aio.run_from_thread(method, args, *kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	203
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	204 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	205 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	206 ) -> dict[str, int \| None]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	207 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	208
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	209 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	210 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	211 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	212 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	213 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	214 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	215 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	216 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	217 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	218 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	219 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	220 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	221 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	222 continue
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	223 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	224 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	225 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	226 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	227 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	228 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	229 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	230 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	231 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	232
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	233 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	234 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	235
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	236 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	237 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	238 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	239 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	240 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	241 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	242 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	243 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	244 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	245 if match:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	246 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	247
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	248 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	249 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	250 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	251 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	252
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	253 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	254 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	255
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	256 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	257 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	258
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	259 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	260 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	261 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	262 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	263 return None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	264
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	265 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	266 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	267
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	268 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	269 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	270 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	271 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	272 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	273 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	274
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	275 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	276 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	277
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	278 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	279 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	280
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	281 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	282
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	283 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	284 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	285
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	286 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	287 @return: ufrag and pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	288 @raise ValueError: Can't extract ufrag and password
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	289 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	290 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	291 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	292
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	293 if ufrag_line and pwd_line:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	294 ufrag = self.ufrag = ufrag_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	295 pwd = self.pwd = pwd_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	296 return ufrag, pwd
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	297 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	298 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	299 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	300
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	301 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	302 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	303 self.role: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	304 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	305 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	306 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	307 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	308 self.sid: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	309 self.offer: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	310 self.local_candidates_buffer = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	311 self.ufrag: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	312 self.pwd: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	313 self.callee: str \| None = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	314 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	315 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	316 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	317 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	318 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	319 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	320 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	321 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	322 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	323 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	324 self.video_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	325 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	326 self.reset_cb()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	327
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	328
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	329 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	330 self,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	331 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	332 audio_pt: int \| None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	333 video_pt: int \| None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	334 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	335 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	336
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	337 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	338 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	339 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	340
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	341 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	342 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	343 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	344
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	345 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	346 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	347 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	348 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	349 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	350 if audio_pt is None or video_pt is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	351 raise NotImplementedError("None value is not handled yet")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	352
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	353 if self.sources == SOURCES_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	354 video_source_elt = "v4l2src"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	355 audio_source_elt = "pulsesrc"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	356 elif self.sources == SOURCES_TEST:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	357 video_source_elt = "videotestsrc is-live=true pattern=ball"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	358 audio_source_elt = "audiotestsrc"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	359 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	360 raise exceptions.InternalError(f'Unknown "sources" value: {self.sources!r}')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	361
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	362 extra_elt = ""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	363
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	364 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	365 local_video_sink_elt = (
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	366 "appsink name=local_video_sink emit-signals=true drop=true max-buffers=1 "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	367 "sync=True"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	368 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	369 elif self.sinks == SINKS_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	370 extra_elt = "compositor name=compositor ! autovideosink"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	371 local_video_sink_elt = """compositor.sink_1"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	372 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	373 raise exceptions.InternalError(f"Unknown sinks value {self.sinks!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	374
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	375 self.gst_pipe_desc = f"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	376 webrtcbin latency=100 name=sendrecv bundle-policy=max-bundle
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	377
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	378 input-selector name=video_selector
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	379 ! videorate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	380 ! video/x-raw,framerate=30/1
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	381 ! tee name=t
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	382
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	383 {extra_elt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	384
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	385 {video_source_elt} name=video_src ! queue leaky=downstream ! video_selector.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	386 videotestsrc is-live=true pattern=black ! queue leaky=downstream ! video_selector.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	387
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	388 t.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	389 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	390 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	391 ! vp8enc deadline=1 keyframe-max-dist=60
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	392 ! rtpvp8pay picture-id-mode=15-bit
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	393 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	394 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	395
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	396 t.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	397 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	398 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	399 ! {local_video_sink_elt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	400
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	401 {audio_source_elt} name=audio_src
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	402 ! valve
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	403 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	404 ! audioconvert
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	405 ! audioresample
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	406 ! opusenc audio-type=voice
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	407 ! rtpopuspay
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	408 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	409 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	410 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	411
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	412 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	413
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	414 # Create the pipeline
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	415 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	416 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	417 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	418
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	419 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	420 self.video_src = self.pipeline.get_by_name("video_src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	421 self.video_selector = self.pipeline.get_by_name("video_selector")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	422 self.audio_valve = self.pipeline.get_by_name("audio_valve")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	423
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	424 if self.video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	425 self.on_video_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	426 if self.audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	427 self.on_audio_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	428
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	429 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	430 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	431 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	432 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	433
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	434 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	435 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	436 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	437 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	438 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	439 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	440 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	441 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	442 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	443 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	444 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	445 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	446 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	447 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	448 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	449 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	450 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	451 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	452
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	453 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	454 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	455
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	456 # local video feedback
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	457 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	458 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	459 local_video_sink = self.pipeline.get_by_name("local_video_sink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	460 local_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	461 local_video_sink.connect("new-sample", self.appsink_data.local_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	462 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	463 local_video_sink.set_property("caps", local_video_sink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	464
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	465 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	466 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	467 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	468 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	469 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	470
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	471 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	472 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	473 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	474 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	475 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	476 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	477 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	478 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	479 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	480 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	481 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	482 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	483
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	484 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	485 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	486 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	487 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	488
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	489 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	490 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	491 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	492 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	493 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	494 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	495 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	496
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	497 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	498 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	499 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	500 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	501 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	502 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	503 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	504 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	505 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	506 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	507 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	508 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	509 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	510 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	511 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	512 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	513
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	514 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	515 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	516
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	517 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	518 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	519 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	520 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	521 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	522 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	523 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	524 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	525 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	526 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	527 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	528 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	529
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	530 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	531 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	532 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	533 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	534
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	535 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	536 self, source: Gst.Element, dest: Gst.Element \| Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	537 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	538 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	539 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	540
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	541 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	542 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	543 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	544 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	545 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	546 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	547 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	548 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	549 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	550 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	551 log.error(f"Unexpected type for dest: {type(sink)}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	552 return False
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	553
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	554 return True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	555
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	556 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	557 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	558 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	559 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	560
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	561 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	562 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	563 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	564 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	565 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	566 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	567 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	568 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	569
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	570 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	571 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	572
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	573 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	574 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	575
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	576 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	577 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	578
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	579 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	580 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	581 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	582
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	583 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	584 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	585 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	586 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	587 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	588 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	589
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	590 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	591 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	592 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	593 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	594 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	595
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	596 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	597
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	598 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	599 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	600
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	601 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	602
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	603 # Check and log the original size of the video
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	604 width = s.get_int("width").value
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	605 height = s.get_int("height").value
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	606 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	607
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	608 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	609 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	610 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	611 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	612 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	613 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	614 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	615 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	616 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	617
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	618 conv = Gst.ElementFactory.make("videoconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	619 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	620 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	621 remote_video_sink = Gst.ElementFactory.make("appsink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	622
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	623 appsink_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	624 remote_video_sink.set_property("caps", appsink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	625
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	626 remote_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	627 remote_video_sink.set_property("drop", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	628 remote_video_sink.set_property("max-buffers", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	629 remote_video_sink.set_property("sync", True)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	630 remote_video_sink.connect("new-sample", self.appsink_data.remote_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	631 self.pipeline.add(remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	632 if self.sinks == SINKS_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	633 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	634
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	635 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	636
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	637 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	638 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	639 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	640 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	641 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	642 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	643
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	644 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	645 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	646 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	647 sink1_pad.set_property("height", local_height)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	648 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	649
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	650 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	651 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	652 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	653 remote_video_sink.set_property("zorder", 0)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	654
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	655 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	656 raise exceptions.InternalError(f'Unhandled "sinks" value: {self.sinks!r}')
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	657
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	658 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	659 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	660 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	661 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	662 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	663 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	664 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	665
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	666 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	667
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	668
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	669 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	670 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	671 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	672 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	673 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	674 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	675 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	676 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	677
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	678 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	679 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	680
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	681 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	682 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	683 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	684 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	685 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	686 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	687
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	688 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	689 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	690 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	691 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	692 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	693 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	694 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	695 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	696 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	697 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	698 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	699 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	700 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	701
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	702 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	703 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	704
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	705 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	706 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	707
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	708 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	709 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	710
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	711 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	712 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	713 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	714 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	715 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	716 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	717
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	718 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	719 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	720 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	721 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	722 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	723
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	724 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	725 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	726
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	727 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	728 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	729 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	730 assert self.callee
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	731 self.sid = await self.bridge.call_start(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	732 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	733 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	734 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	735 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	736 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	737 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	738 if self.pwd is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	739 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	740 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	741 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	742 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	743 ice_data[media_type] = {
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	744 "ufrag": self.ufrag,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	745 "pwd": self.pwd,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	746 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	747 }
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	748 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	749 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	750 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	751 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	752
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	753 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	754 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	755 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	756
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	757 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	758 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	759
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	760 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	761 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	762 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	763 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	764
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	765 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	766 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	767 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	768 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	769 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	770 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	771
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	772 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	773 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	774
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	775 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	776 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	777
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	778 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	779 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	780 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	781 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	782 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	783 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	784 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	785 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	786 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	787 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	788 assert "OPUS" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	789 await self.setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	790 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	791 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	792 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	793 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	794 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	795 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	796 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	797 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	798
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	799 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	800 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	801
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	802 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	803 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	804 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	805 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	806 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	807 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	808 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	809 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	810 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	811 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	812 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	813 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	814
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	815 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	816 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	817 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	818 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	819 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	820 else:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	821 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	822 assert sdp is not None
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	823 ufrag, pwd = self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	824 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	825 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	826 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	827 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	828 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	829 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	830 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	831
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	832 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	833 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	834
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	835 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	836 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	837
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	838 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	839 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	840 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	841 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	842 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	843 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	844 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	845 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	846 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	847 return
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	848 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	849 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	850 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	851 try:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	852 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	853 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	854 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	855 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	856 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	857 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	858 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	859 )
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	860
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	861 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	862 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	863 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	864
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	865 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	866 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	867 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	868 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	869 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	870
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	871 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	872 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	873
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	874 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	875 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	876 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	877 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	878 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	879 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	880
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	881 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	882 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	883
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	884 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	885 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	886 """
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	887 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	888
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	889 def on_audio_mute(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	890 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	891 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	892 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	893 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	894
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	895 def on_video_mute(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	896 if self.video_selector is not None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	897 # when muted, we switch to a black image and deactivate the camera
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	898 if not muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	899 self.video_src.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	900 pad = self.video_selector.get_static_pad("sink_1" if muted else "sink_0")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	901 self.video_selector.props.active_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	902 if muted:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	903 self.video_src.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	904 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	905 log.info(f"video is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	906
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	907 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	908 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation: Goffi <goffi@goffi.org> parents: diff changeset	909 self.reset_instance()

Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4142:783bbdbf8567