Mercurial > libervia-backend
annotate libervia/frontends/tools/webrtc_models.py @ 4240:79c8a70e1813
backend, frontend: prepare remote control:
This is a series of changes necessary to prepare the implementation of remote control
feature:
- XEP-0166: add a `priority` attribute to `ApplicationData`: this is needed when several
applications are working in a same session, to know which one must be handled first.
Will be used to make Remote Control have precedence over Call content.
- XEP-0166: `_call_plugins` is now async and is not used with `DeferredList` anymore: the
benefit to have methods called in parallels is very low, and it cause a lot of trouble
as we can't predict order. Methods are now called sequentially so workflow can be
predicted.
- XEP-0167: fix `senders` XMPP attribute <=> SDP mapping
- XEP-0234: preflight acceptance key is now `pre-accepted` instead of `file-accepted`, so
the same key can be used with other jingle applications.
- XEP-0167, XEP-0343: move some method to XEP-0167
- XEP-0353: use new `priority` feature to call preflight methods of applications according
to it.
- frontend (webrtc): refactor the sources/sink handling with a more flexible mechanism
based on Pydantic models. It is now possible to have has many Data Channel as necessary,
to have them in addition to A/V streams, to specify manually GStreamer sources and
sinks, etc.
- frontend (webrtc): rework of the pipeline to reduce latency.
- frontend: new `portal_desktop` method. Screenshare portal handling has been moved there,
and RemoteDesktop portal has been added.
- frontend (webrtc): fix `extract_ufrag_pwd` method.
rel 436
author | Goffi <goffi@goffi.org> |
---|---|
date | Sat, 11 May 2024 13:52:41 +0200 |
parents | d01b8d002619 |
children | 0d7bb4df2343 |
rev | line source |
---|---|
4233
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
1 #!/usr/bin/env python3 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
2 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
3 # Libervia WebRTC implementation |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org) |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
5 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
6 # This program is free software: you can redistribute it and/or modify |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
7 # it under the terms of the GNU Affero General Public License as published by |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
8 # the Free Software Foundation, either version 3 of the License, or |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
9 # (at your option) any later version. |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
10 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
11 # This program is distributed in the hope that it will be useful, |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
14 # GNU Affero General Public License for more details. |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
15 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
16 # You should have received a copy of the GNU Affero General Public License |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
17 # along with this program. If not, see <http://www.gnu.org/licenses/>. |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
18 |
4240
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
19 from collections.abc import Awaitable |
4233
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
20 from dataclasses import dataclass, field |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
21 from typing import Any, Callable |
4240
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
22 import uuid |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
23 import gi |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
24 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
25 gi.require_versions({"Gst": "1.0", "GstWebRTC": "1.0"}) |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
26 from gi.repository import GstWebRTC |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
27 from pydantic import BaseModel, Field |
4233
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
28 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
29 from libervia.frontends.tools import jid |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
30 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
31 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
32 @dataclass |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
33 class CallData: |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
34 callee: jid.JID |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
35 sid: str | None = None |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
36 action_id: str | None = None |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
37 kwargs: dict[str, Any] = field(default_factory=dict) |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
38 |
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
39 |
4240
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
40 class SourcesData(BaseModel): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
41 """Data for Sources""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
42 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
43 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
44 class SourcesNone(SourcesData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
45 """No source is used. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
46 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
47 This is used when the WebRTC connection will be used for data channels only.""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
48 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
49 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
50 class SourcesAuto(SourcesData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
51 """Automatic Sources (webcam/microphone)""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
52 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
53 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
54 class SourcesTest(SourcesData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
55 """Test Sources (pattern)""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
56 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
57 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
58 class SourcesDataChannel(SourcesData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
59 """Sources for transmitting data over Data Channel |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
60 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
61 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
62 @param dc_open_cb: Called when Data Channel is open. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
63 This callback will be run in a GStreamer thread. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
64 """ |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
65 name: str = Field(default_factory=lambda: str(uuid.uuid4())) |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
66 dc_open_cb: Callable[[GstWebRTC.WebRTCDataChannel], None] |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
67 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
68 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
69 class SourcesPipeline(SourcesData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
70 """Use custom pipeline description as a source. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
71 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
72 @param video_pipeline: pipeline description of video source. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
73 None to use automatic video source (same as SourcesAuto). |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
74 Empty string to disable video. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
75 @param audio_pipeline: pipeline description of audio source. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
76 None to use automatic audio source (same as SourcesAuto). |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
77 Empty string to disable audio. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
78 @param video_properties: Elements properties to set. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
79 @param audio_properties: Elements properties to set. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
80 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
81 """ |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
82 video_pipeline: str|None = None |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
83 audio_pipeline: str|None = None |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
84 video_properties: dict = Field(default_factory=lambda: {}) |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
85 audio_properties: dict = Field(default_factory=lambda: {}) |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
86 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
87 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
88 class SinksData(BaseModel): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
89 """Data for Sinks""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
90 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
91 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
92 class SinksNone(SinksData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
93 """No sink is used. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
94 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
95 This is used when the WebRTC connection will be used for data channels only.""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
96 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
97 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
98 class SinksAuto(SinksData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
99 """Automatic Sinks (create windows/default audio)""" |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
100 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
101 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
102 class SinksApp(SinksData): |
4233
d01b8d002619
cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff
changeset
|
103 local_video_cb: Callable |
4240
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
104 remote_video_cb: Callable | None |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
105 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
106 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
107 class SinksDataChannel(SinksData): |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
108 """Sinks for transmitting data over Data Channel |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
109 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
110 @param dc_on_data_channel: Called when Data Channel is created. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
111 This callback will be run in a GStreamer thread. |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
112 """ |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
113 |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
114 dc_on_data_channel: ( |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
115 Callable[[GstWebRTC.WebRTCDataChannel], Awaitable[None]] | None |
79c8a70e1813
backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents:
4233
diff
changeset
|
116 ) = None |