Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4205:17a8168966f9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
frontends (tools/webrtc): implement screensharing for Wayland + bug fixes: - Freedesktop Desktop Screenshare port is now used when Wayland is detected (needs `xdg-desktop-portal` with the implementation corresponding to desktop environment). - Add a binding feature to feedback state to application (e.g. if desktop sharing is cancelled from desktop environment, or at portal's permission request level). - fix misnaming of video source (was wrongly named `camera` instead of `video`). - fix desktop sharing pad selection in `input-selector` when it has been added once, then removed, then added again. rel 434
author Goffi <goffi@goffi.org>
date Thu, 18 Jan 2024 23:29:25 +0100
parents 879bad48cc2d
children fe29fbdabce6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 import gi
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
20 gi.require_versions({
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 "Gst": "1.0",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 "GstWebRTC": "1.0"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 })
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
24 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
25
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
26 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
27
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 except ImportError:
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
31 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
32 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
33 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 from dataclasses import dataclass
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 import logging
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
39 from random import randint
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
40 import re
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 from typing import Callable
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
42 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
43
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 from libervia.backend.tools.common import data_format
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
45 from libervia.frontends.tools import aio, display_servers
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
46
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
47 current_server = display_servers.detect()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
48 if current_server == display_servers.X11:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
49 # GSTreamer's ximagesrc documentation asks to run this function
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
50 import ctypes
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
51 ctypes.CDLL('libX11.so.6').XInitThreads()
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
53
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
54 log = logging.getLogger(__name__)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
55
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
56 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
57
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
58 SOURCES_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
59 SOURCES_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 SINKS_APP = "app"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 SINKS_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
62 SINKS_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
63
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
64
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
65 class ScreenshareError(Exception):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
66 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
67
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
68
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 @dataclass
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 class AppSinkData:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 local_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 remote_video_cb: Callable
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
73
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
74
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
75 class DesktopPortal:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
76
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
77 def __init__(self, webrtc: "WebRTC"):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
78 import dbus
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
79 from dbus.mainloop.glib import DBusGMainLoop
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
80 # we want monitors + windows, see https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html#org-freedesktop-portal-screencast-availablesourcetypes
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
81 self.dbus = dbus
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
82 self.webrtc = webrtc
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
83 self.sources_type = dbus.UInt32(7)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
84 DBusGMainLoop(set_as_default=True)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
85 self.session_bus = dbus.SessionBus()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
86 portal_object = self.session_bus.get_object(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
87 'org.freedesktop.portal.Desktop',
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
88 '/org/freedesktop/portal/desktop'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
89 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
90 self.screencast_interface = dbus.Interface(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
91 portal_object,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
92 'org.freedesktop.portal.ScreenCast'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
93 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
94 self.session_interface = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
95 self.session_signal = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
96 self.handle_counter = 0
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
97 self.session_handle = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
98 self.stream_data: dict|None = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
99
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
100 @property
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
101 def handle_token(self):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
102 self.handle_counter += 1
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
103 return f"libervia{self.handle_counter}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
104
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
105 def on_session_closed(self, details: dict) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
106 if self.session_interface is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
107 self.session_interface = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
108 self.webrtc.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
109 if self.session_signal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
110 self.session_signal.remove()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
111 self.session_signal = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
112
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
113
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
114 async def dbus_call(self, method_name: str, *args) -> dict:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
115 """Call a screenshare portal method
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
116
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
117 This method handle the signal response.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
118 @param method_name: method to call
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
119 @param args: extra args
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
120 `handle_token` will be automatically added to the last arg (option dict)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
121 @return: method result
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
122 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
123 if self.session_handle is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
124 self.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
125 method = getattr(self.screencast_interface, method_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
126 options = args[-1]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
127 reply_fut = asyncio.Future()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
128 signal_fut = asyncio.Future()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
129 # cf. https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
130 handle_token = self.handle_token
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
131 sender = self.session_bus.get_unique_name().replace(".", "_")[1:]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
132 path = f"/org/freedesktop/portal/desktop/request/{sender}/{handle_token}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
133 signal_match = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
134
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
135 def on_signal(response, results):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
136 assert signal_match is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
137 signal_match.remove()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
138 if response == 0:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
139 signal_fut.set_result(results)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
140 elif response == 1:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
141 signal_fut.set_exception(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
142 exceptions.CancelError("Cancelled by user.")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
143 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
144 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
145 signal_fut.set_exception(ScreenshareError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
146 f"Can't get signal result"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
147 ))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
148
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
149 signal_match = self.session_bus.add_signal_receiver(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
150 on_signal,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
151 signal_name="Response",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
152 dbus_interface="org.freedesktop.portal.Request",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
153 path=path
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
154 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
155
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
156 options["handle_token"] = handle_token
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
157
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
158 method(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
159 *args,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
160 reply_handler=reply_fut.set_result,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
161 error_handler=reply_fut.set_exception
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
162 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
163 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
164 await reply_fut
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
165 except Exception as e:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
166 raise ScreenshareError(f"Can't ask screenshare permission: {e}")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
167 return await signal_fut
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
168
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
169 async def request_screenshare(self) -> dict:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
170 session_data = await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
171 "CreateSession",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
172 {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
173 "session_handle_token": str(randint(1, 2**32)),
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
174 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
175 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
176 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
177 session_handle = session_data["session_handle"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
178 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
179 raise ScreenshareError("Can't get session handle")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
180 self.session_handle = session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
181
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
182
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
183 await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
184 "SelectSources",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
185 session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
186 {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
187 "multiple": True,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
188 "types": self.sources_type,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
189 "modal": True
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
190 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
191 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
192 screenshare_data = await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
193 "Start",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
194 session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
195 "",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
196 {}
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
197 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
198
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
199 session_object = self.session_bus.get_object(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
200 'org.freedesktop.portal.Desktop',
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
201 session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
202 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
203 self.session_interface = self.dbus.Interface(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
204 session_object,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
205 'org.freedesktop.portal.Session'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
206 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
207
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
208 self.session_signal = self.session_bus.add_signal_receiver(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
209 self.on_session_closed,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
210 signal_name="Closed",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
211 dbus_interface="org.freedesktop.portal.Session",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
212 path=session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
213 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
214
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
215 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
216 node_id, stream_data = screenshare_data["streams"][0]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
217 source_type = int(stream_data["source_type"])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
218 except (IndexError, KeyError):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
219 raise ScreenshareError("Can't parse stream data")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
220 self.stream_data = stream_data = {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
221 "session_handle": session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
222 "node_id": node_id,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
223 "source_type": source_type
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
224 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
225 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
226 height = int(stream_data["size"][0])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
227 weight = int(stream_data["size"][1])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
228 except (IndexError, KeyError):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
229 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
230 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
231 stream_data["size"] = (height, weight)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
232
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
233 return self.stream_data
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
234
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
235 def end_screenshare(self) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
236 """Close a running screenshare session, if any."""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
237 if self.session_interface is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
238 return
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
239 self.session_interface.Close()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
240 self.on_session_closed({})
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
241
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
242
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
243 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
244 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
245
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 This class encapsulates the WebRTC functionalities required for initiating and
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 handling audio and video calls.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
249
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
250 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
251 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
253 profile: str,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
254 sources: str = SOURCES_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
255 sinks: str = SINKS_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 appsink_data: AppSinkData | None = None,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
257 reset_cb: Callable | None = None,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
258 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
259 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
260 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
261 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
262 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
263 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
264 self._video_muted = False
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
265 self._desktop_sharing = False
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
266 self.desktop_sharing_data = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
267 self.sources = sources
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
268 self.sinks = sinks
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
269 if sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
270 self.appsink_data = appsink_data
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
271 elif appsink_data is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
272 raise exceptions.InternalError(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
273 "appsink_data can only be used for SINKS_APP sinks"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
274 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
275 self.reset_cb = reset_cb
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
276 if current_server == display_servers.WAYLAND:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
277 self.desktop_portal = DesktopPortal(self)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
278 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
279 self.desktop_portal = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
280 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
281
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
282 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
283 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
284 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
285
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
286 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
287 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
288 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
289 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
290 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
291
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
292 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
293 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
294 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
295
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
296 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
297 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
298 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
299 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
300 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
301
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
302 @property
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
303 def desktop_sharing(self):
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
304 return self._desktop_sharing
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
305
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
306 @desktop_sharing.setter
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
307 def desktop_sharing(self, active: bool) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
308 if active != self._desktop_sharing:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
309 self._desktop_sharing = active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
310 self.on_desktop_switch(active)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
311 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
312 cb = self.bindings["desktop_sharing"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
313 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
314 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
315 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
316 cb(active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
317
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
318 @property
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
319 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
320 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
321
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
322 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
323 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
324 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
325 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
326 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
327 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
328 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
329
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
330 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
331 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
332 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
333 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
334 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
335
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
336 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
337 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
338 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
339 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
340
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
341 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
342 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
343 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
344 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
345 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
346
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
347 def bind(self, **kwargs: Callable) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
348 self.bindings.clear()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
349 for key, cb in kwargs.items():
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
350 if key not in ("desktop_sharing",):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
351 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
352 'Only "desktop_sharing" is currently allowed for binding'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
353 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
354 self.bindings[key] = cb
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
355
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
356
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
357 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
358 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
359 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
360 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
361 with_timestamp: bool = True,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
362 bin_: Gst.Bin|None = None,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
363 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
364 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
365
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
366 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
367 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
368 See
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
369 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
370 for details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
371
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
372 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
373 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
374 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
375 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
376 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
377 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
378 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
379 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
380 if with_timestamp:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
381 timestamp = datetime.now().isoformat(timespec='milliseconds')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
382 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
383
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
384 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
385
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
386 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
387 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
388
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
389 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
390 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
391 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
392 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
393 return index
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
394 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
395
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
396 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
397 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
398
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
399 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
400 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
401 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
402 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
403 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
404
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
405 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
406 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
407 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
408 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
409
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
410 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
411
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
412 def _a_call(self, method, *args, **kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
413 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
414 aio.run_from_thread(method, *args, **kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
415
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
416 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
417 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
418 ) -> dict[str, int | None]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
419 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
420
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
421 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
422 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
423 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
424 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
425 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
426 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
427 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
428 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
429 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
430 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
431 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
432 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
433 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
434 continue
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
435 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
436 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
437 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
438 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
439 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
440 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
441 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
442 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
443 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
444
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
445 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
446 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
447
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
448 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
449 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
450 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
451 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
452 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
453 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
454 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
455 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
456 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
457 if match:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
458 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
459
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
460 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
461 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
462 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
463 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
464
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
465 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
466 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
467
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
468 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
469 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
470
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
471 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
472 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
473 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
474 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
475 return None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
476
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
477 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
478 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
479
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
480 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
481 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
482 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
483 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
484 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
485 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
486
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
487 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
488 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
489
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
490 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
491 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
492
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
493 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
494
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
495 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
496 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
497
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
498 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
499 @return: ufrag and pwd
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
500 @raise ValueError: Can't extract ufrag and password
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
501 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
502 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
503 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
504
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
505 if ufrag_line and pwd_line:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
506 ufrag = self.ufrag = ufrag_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
507 pwd = self.pwd = pwd_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
508 return ufrag, pwd
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
509 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
510 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
511 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
512
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
513 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
514 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
515 self.role: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
516 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
517 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
518 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
519 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
520 self.sid: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
521 self.offer: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
522 self.local_candidates_buffer = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
523 self.ufrag: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
524 self.pwd: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
525 self.callee: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
526 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
527 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
528 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
529 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
530 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
531 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
532 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
533 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
534 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
535 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
536 self.video_valve = None
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
537 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
538 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
539 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
540 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
541 self.bindings = {}
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
542 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
543 self.reset_cb()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
544
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
545
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
546 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
547 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
548 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
549 audio_pt: int | None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
550 video_pt: int | None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
551 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
552 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
553
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
554 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
555 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
556 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
557
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
558 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
559 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
560 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
561
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
562 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
563 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
564 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
565 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
566 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
567 if audio_pt is None or video_pt is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
568 raise NotImplementedError("None value is not handled yet")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
569
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
570 if self.sources == SOURCES_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
571 video_source_elt = "v4l2src"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
572 audio_source_elt = "pulsesrc"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
573 elif self.sources == SOURCES_TEST:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
574 video_source_elt = "videotestsrc is-live=true pattern=ball"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
575 audio_source_elt = "audiotestsrc"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
576 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
577 raise exceptions.InternalError(f'Unknown "sources" value: {self.sources!r}')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
578
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
579 extra_elt = ""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
580
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
581 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
582 local_video_sink_elt = (
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
583 "appsink name=local_video_sink emit-signals=true drop=true max-buffers=1 "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
584 "sync=True"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
585 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
586 elif self.sinks == SINKS_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
587 extra_elt = "compositor name=compositor ! autovideosink"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
588 local_video_sink_elt = """compositor.sink_1"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
589 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
590 raise exceptions.InternalError(f"Unknown sinks value {self.sinks!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
591
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
592 self.gst_pipe_desc = f"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
593 webrtcbin latency=100 name=sendrecv bundle-policy=max-bundle
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
594
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
595 input-selector name=video_selector
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
596 ! videorate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
597 ! video/x-raw,framerate=30/1
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
598 ! tee name=t
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
599
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
600 {extra_elt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
601
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
602 {video_source_elt} name=video_src ! queue leaky=downstream ! video_selector.
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
603 videotestsrc name=muted_src is-live=true pattern=black ! queue leaky=downstream ! video_selector.
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
604
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
605 t.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
606 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
607 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
608 ! vp8enc deadline=1 keyframe-max-dist=60
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
609 ! rtpvp8pay picture-id-mode=15-bit
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
610 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
611 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
612
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
613 t.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
614 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
615 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
616 ! {local_video_sink_elt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
617
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
618 {audio_source_elt} name=audio_src
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
619 ! valve
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
620 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
621 ! audioconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
622 ! audioresample
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
623 ! opusenc audio-type=voice
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
624 ! rtpopuspay
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
625 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
626 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
627 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
628
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
629 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
630
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
631 # Create the pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
632 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
633 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
634 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
635
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
636 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
637 self.video_src = self.pipeline.get_by_name("video_src")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
638 self.muted_src = self.pipeline.get_by_name("muted_src")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
639 self.video_selector = self.pipeline.get_by_name("video_selector")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
640 self.audio_valve = self.pipeline.get_by_name("audio_valve")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
641
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
642 if self.video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
643 self.on_video_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
644 if self.audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
645 self.on_audio_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
646
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
647 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
648 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
649 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
650 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
651
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
652 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
653 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
654 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
655 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
656 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
657 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
658 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
659 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
660 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
661 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
662 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
663 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
664 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
665 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
666 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
667 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
668 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
669 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
670
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
671 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
672 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
673
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
674 # local video feedback
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
675 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
676 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
677 local_video_sink = self.pipeline.get_by_name("local_video_sink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
678 local_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
679 local_video_sink.connect("new-sample", self.appsink_data.local_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
680 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
681 local_video_sink.set_property("caps", local_video_sink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
682
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
683 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
684 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
685 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
686 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
687 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
688
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
689 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
690 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
691 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
692 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
693 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
694 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
695 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
696 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
697 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
698 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
699 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
700 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
701
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
702 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
703 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
704 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
705 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
706
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
707 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
708 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
709 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
710 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
711 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
712 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
713 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
714
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
715 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
716 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
717 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
718 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
719 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
720 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
721 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
722 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
723 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
724 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
725 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
726 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
727 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
728 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
729 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
730 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
731
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
732 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
733 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
734
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
735 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
736 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
737 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
738 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
739 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
740 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
741 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
742 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
743 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
744 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
745 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
746 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
747
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
748 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
749 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
750 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
751 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
752
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
753 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
754 self, source: Gst.Element, dest: Gst.Element | Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
755 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
756 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
757 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
758
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
759 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
760 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
761 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
762 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
763 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
764 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
765 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
766 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
767 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
768 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
769 log.error(f"Unexpected type for dest: {type(sink)}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
770 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
771
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
772 return True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
773
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
774 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
775 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
776 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
777 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
778
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
779 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
780 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
781 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
782 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
783 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
784 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
785 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
786 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
787
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
788 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
789 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
790
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
791 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
792 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
793
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
794 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
795 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
796
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
797 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
798 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
799 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
800
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
801 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
802 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
803 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
804 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
805 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
806 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
807
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
808 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
809 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
810 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
811 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
812 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
813
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
814 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
815
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
816 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
817 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
818
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
819 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
820
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
821 # Check and log the original size of the video
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
822 width = s.get_int("width").value
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
823 height = s.get_int("height").value
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
824 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
825
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
826 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
827 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
828 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
829 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
830 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
831 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
832 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
833 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
834 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
835
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
836 conv = Gst.ElementFactory.make("videoconvert")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
837 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
838 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
839 remote_video_sink = Gst.ElementFactory.make("appsink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
840
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
841 appsink_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
842 remote_video_sink.set_property("caps", appsink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
843
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
844 remote_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
845 remote_video_sink.set_property("drop", True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
846 remote_video_sink.set_property("max-buffers", 1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
847 remote_video_sink.set_property("sync", True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
848 remote_video_sink.connect("new-sample", self.appsink_data.remote_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
849 self.pipeline.add(remote_video_sink)
4146
60d107f2178a frontends (tools/webrtc): fix `SINKS_APP` condition check.
Goffi <goffi@goffi.org>
parents: 4145
diff changeset
850 elif self.sinks == SINKS_AUTO:
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
851 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
852
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
853 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
854
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
855 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
856 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
857 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
858 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
859 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
860 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
861
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
862 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
863 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
864 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
865 sink1_pad.set_property("height", local_height)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
866 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
867
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
868 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
869 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
870 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
871 remote_video_sink.set_property("zorder", 0)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
872
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
873 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
874 raise exceptions.InternalError(f'Unhandled "sinks" value: {self.sinks!r}')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
875
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
876 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
877 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
878 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
879 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
880 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
881 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
882 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
883
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
884 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
885
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
886
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
887 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
888 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
889 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
890 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
891 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
892 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
893 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
894 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
895
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
896 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
897 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
898
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
899 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
900 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
901 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
902 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
903 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
904 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
905
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
906 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
907 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
908 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
909 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
910 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
911 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
912 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
913 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
914 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
915 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
916 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
917 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
918 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
919
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
920 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
921 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
922
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
923 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
924 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
925
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
926 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
927 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
928
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
929 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
930 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
931 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
932 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
933 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
934 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
935
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
936 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
937 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
938 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
939 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
940 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
941
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
942 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
943 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
944
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
945 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
946 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
947 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
948 assert self.callee
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
949 self.sid = await self.bridge.call_start(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
950 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
951 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
952 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
953 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
954 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
955 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
956 if self.pwd is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
957 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
958 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
959 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
960 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
961 ice_data[media_type] = {
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
962 "ufrag": self.ufrag,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
963 "pwd": self.pwd,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
964 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
965 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
966 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
967 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
968 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
969 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
970
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
971 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
972 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
973 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
974
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
975 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
976 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
977
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
978 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
979 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
980 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
981 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
982
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
983 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
984 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
985 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
986 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
987 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
988 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
989
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
990 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
991 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
992
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
993 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
994 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
995
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
996 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
997 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
998 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
999 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1000 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1001 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1002 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1003 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1004 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1005 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1006 assert "OPUS" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1007 await self.setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1008 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1009 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1010 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1011 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1012 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1013 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1014 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1015 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1016
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1017 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1018 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1019
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1020 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1021 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1022 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1023 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1024 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1025 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1026 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1027 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1028 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1029 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1030 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1031 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1032
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1033 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1034 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1035 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1036 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1037 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1038 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1039 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1040 assert sdp is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1041 ufrag, pwd = self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1042 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1043 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1044 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1045 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1046 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1047 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1048 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1049
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1050 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1051 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1052
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1053 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1054 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1055
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1056 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1057 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1058 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1059 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1060 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1061 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1062 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1063 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1064 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1065 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1066 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1067 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1068 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1069 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1070 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1071 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1072 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1073 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1074 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1075 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1076 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1077 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1078
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1079 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1080 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1081 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1082
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1083 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1084 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1085 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1086 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1087 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1088
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1089 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1090 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1091
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1092 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1093 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1094 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1095 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1096 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1097 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1098
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1099 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1100 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1101
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1102 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1103 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1104 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1105 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1106
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1107 def on_audio_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1108 """Handles (un)muting of audio.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1109
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1110 @param muted: True if audio is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1111 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1112 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1113 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1114 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1115 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1116
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1117 def on_video_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1118 """Handles (un)muting of video.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1119
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1120 @param muted: True if video is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1121 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1122 if self.video_selector is not None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1123 current_source = None if muted else "desktop" if self.desktop_sharing else "video"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1124 self.switch_video_source(current_source)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1125 state = "muted" if muted else "unmuted"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1126 log.info(f"Video is now {state}")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1127
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1128 def on_desktop_switch(self, desktop_active: bool) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1129 """Switches the video source between desktop and video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1130
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1131 @param desktop_active: True if desktop must be active. False for video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1132 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1133 if desktop_active and self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1134 aio.run_async(self.on_desktop_switch_portal(desktop_active))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1135 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1136 self.do_desktop_switch(desktop_active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1137
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1138 async def on_desktop_switch_portal(self, desktop_active: bool) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1139 """Call freedesktop screenshare portal and the activate the shared stream"""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1140 assert self.desktop_portal is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1141 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1142 screenshare_data = await self.desktop_portal.request_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1143 except exceptions.CancelError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1144 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1145 return
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1146 self.desktop_sharing_data = {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1147 "path": str(screenshare_data["node_id"])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1148 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1149 self.do_desktop_switch(desktop_active)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1150
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1151 def do_desktop_switch(self, desktop_active: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1152 if self.video_muted:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1153 # Update the active source state but do not switch
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1154 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1155 return
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1156
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1157 source = "desktop" if desktop_active else "video"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1158 self.switch_video_source(source)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1159 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1160
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1161 def switch_video_source(self, source: str|None) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1162 """Activates the specified source while deactivating the others.
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1163
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1164 @param source: 'desktop', 'video', 'muted' or None for muted source.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1165 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1166 if source is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1167 source = "muted"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1168 if source not in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1169 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1170 f"Invalid source: {source!r}, use one of {'video', 'muted', 'desktop'}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1171 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1172
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1173 self.pipeline.set_state(Gst.State.PAUSED)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1174
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1175 # Create a new desktop source if necessary
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1176 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1177 self._setup_desktop_source(self.desktop_sharing_data)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1178
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1179 # Activate the chosen source and deactivate the others
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1180 for src_name in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1181 src_element = self.pipeline.get_by_name(f"{src_name}_src")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1182 if src_name == source:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1183 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1184 src_element.set_state(Gst.State.PLAYING)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1185 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1186 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1187 if src_name == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1188 self._remove_desktop_source(src_element)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1189 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1190 src_element.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1191
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1192 # Set the video_selector active pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1193 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1194 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1195 pad = self.desktop_sink_pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1196 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1197 log.error(f"No desktop pad available")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1198 pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1199 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1200 pad_name = f"sink_{['video', 'muted'].index(source)}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1201 pad = self.video_selector.get_static_pad(pad_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1202
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1203 if pad is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1204 self.video_selector.props.active_pad = pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1206 self.pipeline.set_state(Gst.State.PLAYING)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1207
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1208 def _setup_desktop_source(self, properties: dict[str, object]|None) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1209 """Set up a new desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1210
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1211 @param properties: The properties to set on the desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1212 """
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1213 source_elt = "ximagesrc" if self.desktop_portal is None else "pipewiresrc"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1214 desktop_src = Gst.ElementFactory.make(source_elt, "desktop_src")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1215 if properties is None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1216 properties = {}
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1217 for key, value in properties.items():
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1218 log.debug(f"setting {source_elt} property: {key!r}={value!r}")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1219 desktop_src.set_property(key, value)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1220 video_convert = Gst.ElementFactory.make("videoconvert", "desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1221 queue = Gst.ElementFactory.make("queue", "desktop_queue")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1222 queue.set_property("leaky", "downstream")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1223
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1224 self.pipeline.add(desktop_src)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1225 self.pipeline.add(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1226 self.pipeline.add(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1227
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1228 desktop_src.link(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1229 video_convert.link(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1230
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1231 sink_pad_template = self.video_selector.get_pad_template("sink_%u")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1232 self.desktop_sink_pad = self.video_selector.request_pad(sink_pad_template, None, None)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1233 queue_src_pad = queue.get_static_pad("src")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1234 queue_src_pad.link(self.desktop_sink_pad)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1235
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1236 desktop_src.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1237 video_convert.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1238 queue.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1239
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1240 def _remove_desktop_source(self, desktop_src: Gst.Element) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1241 """Remove the desktop source from the pipeline.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1242
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1243 @param desktop_src: The desktop source to remove.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1244 """
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1245 # Remove elements for the desktop source
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1246 video_convert = self.pipeline.get_by_name("desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1247 queue = self.pipeline.get_by_name("desktop_queue")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1248
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1249 if video_convert:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1250 video_convert.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1251 desktop_src.unlink(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1252 self.pipeline.remove(video_convert)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1253
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1254 if queue:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1255 queue.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1256 self.pipeline.remove(queue)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1257
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1258 desktop_src.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1259 self.pipeline.remove(desktop_src)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1260
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1261 # Release the pad associated with the desktop source
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1262 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1263 self.video_selector.release_request_pad(self.desktop_sink_pad)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1264 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1265
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1266 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1267 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1268
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1269 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1270 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1271 self.reset_instance()