libervia-web: libervia/web/pages/calls/_browser/webrtc.py annotate

annotate libervia/web/pages/calls/_browser/webrtc.py @ 1602:6feac4a25e60

browser: Remote Control implementation: - Add `cbor-x` JS dependency. - In "Call" page, a Remote Control session can now be started. This is done by clicking on a search item 3 dots menu. Libervia Web will act as a controlling device. The call box is then adapted, and mouse/wheel and keyboard events are sent to remote, touch events are converted to mouse one. - Some Brython 3.12* related changes. rel 436

author	Goffi <goffi@goffi.org>
date	Sat, 11 May 2024 14:02:22 +0200
parents	0a4433a343a3
children	4a9679369856

rev	line source
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 import json
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	2 import re
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	3
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 from bridge import AsyncBridge as Bridge
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	5 from browser import aio, console as log, document, window, DOMNode
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	6 import dialog
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	7 from javascript import JSObject, NULL
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	8 import jid
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	9 from js_modules.cbor_x import CBOR
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 log.warning = log.warn
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 profile = window.profile or ""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 bridge = Bridge()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	14
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	16 class FileSender:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	17 CHUNK_SIZE = 64 * 1024
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	18
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	19 def __init__(self, session_id, file, data_channel):
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	20 self.session_id = session_id
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	21 self.file = file
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	22 self.data_channel = data_channel
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	23 data_channel.bind("open", self._on_open)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	24 self.offset = 0
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	25
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	26 def _on_open(self, __):
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	27 log.info(f"Data channel open, starting to send {self.file.name}.")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	28 self.send_file()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	29
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	30 def _on_reader_load(self, event):
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	31 self.data_channel.send(event.target.result)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	32 self.offset += self.CHUNK_SIZE
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	33 self.send_file()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	34
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	35 def send_file(self):
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	36 if self.offset < self.file.size:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	37 chunk = self.file.slice(self.offset, self.offset + self.CHUNK_SIZE)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	38 reader = window.FileReader.new()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	39 reader.onload = self._on_reader_load
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	40 reader.readAsArrayBuffer(chunk)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	41 else:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	42 log.info(f"file {self.file.name} sent.")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	43 self.data_channel.close()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	44 if self.session_id is not None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	45 aio.run(bridge.call_end(self.session_id, ""))
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	46
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	47
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	48 class FileReceiver:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	49
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	50 def __init__(self, session_id: str, data_channel, extra_data: dict) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	51 """Initializes the file receiver with a data channel.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	52
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	53 @param data_channel: The RTCDataChannel through which file data is received.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	54 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	55 self.session_id = session_id
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	56 self.data_channel = data_channel
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	57 self.received_chunks = []
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	58 self.file_data = extra_data.get("file_data", {})
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	59 data_channel.bind("message", self._on_message)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	60 data_channel.bind("close", self._on_close)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	61 log.debug("File receiver created.")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	62
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	63 def _on_message(self, event) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	64 """Handles incoming message events from the data channel.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	65
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	66 @param event: The event containing the data chunk.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	67 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	68 self.received_chunks.append(event.data)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	69
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	70 def _on_close(self, __) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	71 """Handles the data channel's close event.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	72
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	73 Assembles the received chunks into a Blob and triggers a file download.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	74 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	75 # The file is complete, we assemble the chunks in a blob
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	76 blob = window.Blob.new(self.received_chunks)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	77 url = window.URL.createObjectURL(blob)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	78
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	79 # and create the <a> element to download the file.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	80 a = document.createElement("a")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	81 a.href = url
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	82 a.download = self.file_data.get("name", "received_file")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	83 document.body.appendChild(a)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	84 a.click()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	85
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	86 # We now clean up.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	87 document.body.removeChild(a)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	88 window.URL.revokeObjectURL(url)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	89 log.info("File received.")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	90 aio.run(bridge.call_end(self.session_id, ""))
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	91
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	92
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	93 class RemoteControler:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	94 """Send input events to controlled device"""
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	95
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	96 def __init__(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	97 self,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	98 session_id: str,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	99 capture_elt: DOMNode,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	100 data_channel: JSObject,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	101 simulate_mouse: bool = True
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	102 ) -> None:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	103 """Initialize a RemoteControler instance.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	104
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	105 @param session_id: ID of the Jingle Session
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	106 @param capture_elt: element where the input events are captured.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	107 @param data_channel: WebRTCDataChannel instance linking to controlled device.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	108 @simulate_mouse: if True, touch event will be converted to mouse events.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	109 """
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	110 self.session_id = session_id
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	111 self.capture_elt = capture_elt
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	112 self.capture_elt.bind("click", self._on_capture_elt_click)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	113 self.data_channel = data_channel
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	114 data_channel.bind("open", self._on_open)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	115 self.simulate_mouse = simulate_mouse
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	116 self.last_mouse_position = (0, 0)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	117
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	118 def _on_capture_elt_click(self, __):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	119 self.capture_elt.focus()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	120
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	121 def _on_open(self, __):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	122 log.info(f"Data channel open, starting to send inputs.")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	123 self.start_capture()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	124
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	125 def start_capture(self) -> None:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	126 """Start capturing input events to send them to the controlled device."""
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	127 for event_name in [
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	128 "mousedown",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	129 "mouseup",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	130 "mousemove",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	131 "keydown",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	132 "keyup",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	133 "touchstart",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	134 "touchend",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	135 "touchmove",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	136 "wheel",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	137 ]:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	138 self.capture_elt.bind(event_name, self._send_event)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	139 self.capture_elt.focus()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	140
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	141 def get_stream_coordinates(self, client_x: float, client_y: float) -> tuple[float, float]:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	142 """Calculate coordinates relative to the actual video stream.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	143
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	144 This method calculates the coordinates relative to the video content inside the <video>
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	145 element, considering any scaling or letterboxing due to aspect ratio differences.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	146
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	147 @param client_x: The clientX value from the event, relative to the viewport.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	148 @param client_y: The clientY value from the event, relative to the viewport.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	149 @return: The x and y coordinates relative to the actual video stream.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	150 """
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	151 video_element = self.capture_elt
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	152 video_rect = video_element.getBoundingClientRect()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	153
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	154 # Calculate offsets relative to the capture element
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	155 element_x = client_x - video_rect.left
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	156 element_y = client_y - video_rect.top
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	157
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	158 element_width, element_height = video_rect.width, video_rect.height
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	159 stream_width, stream_height = video_element.videoWidth, video_element.videoHeight
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	160
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	161 if not all((element_width, element_height, stream_width, stream_height)):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	162 log.warning("Invalid dimensions for video or element, using offsets.")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	163 return element_x, element_y
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	164
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	165 element_aspect_ratio = element_width / element_height
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	166 stream_aspect_ratio = stream_width / stream_height
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	167
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	168 # Calculate scale and offset based on aspect ratio comparison
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	169 if stream_aspect_ratio > element_aspect_ratio:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	170 # Video is more "wide" than the element: letterboxes will be top and bottom
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	171 scale = element_width / stream_width
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	172 scaled_height = stream_height * scale
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	173 offset_x, offset_y = 0, (element_height - scaled_height) / 2
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	174 else:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	175 # Video is more "tall" than the element: letterboxes will be on the sides
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	176 scale = element_height / stream_height
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	177 scaled_width = stream_width * scale
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	178 offset_x, offset_y = (element_width - scaled_width) / 2, 0
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	179
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	180 # Mouse coordinates relative to the video stream
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	181 x = (element_x - offset_x) / scale
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	182 y = (element_y - offset_y) / scale
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	183
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	184 # Ensure the coordinates are within the bounds of the video stream
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	185 x = max(0.0, min(x, stream_width))
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	186 y = max(0.0, min(y, stream_height))
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	187
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	188 return x, y
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	189
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	190 def _send_event(self, event: JSObject) -> None:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	191 """Serialize and send the event to the controlled device through the data channel."""
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	192 event.preventDefault()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	193 serialized_event = self._serialize_event(event)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	194 # TODO: we should join events instead
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	195 self.data_channel.send(CBOR.encode(serialized_event))
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	196
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	197 def _serialize_event(self, event: JSObject) -> dict[str, object]:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	198 """Serialize event data for transmission.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	199
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	200 @param event: an input event.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	201 @return: event data to send to peer.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	202 """
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	203 if event.type.startswith("key"):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	204 ret = {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	205 "type": event.type,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	206 "key": event.key,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	207 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	208 if event.location:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	209 ret["location"] = event.location
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	210 return ret
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	211 elif event.type.startswith("mouse"):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	212 x, y = self.get_stream_coordinates(event.clientX, event.clientY)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	213 return {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	214 "type": event.type,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	215 "buttons": event.buttons,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	216 "x": x,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	217 "y": y,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	218 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	219 elif event.type.startswith("touch"):
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	220 touches = [
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	221 {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	222 "identifier": touch.identifier,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	223 **dict(zip(["x", "y"], self.get_stream_coordinates(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	224 touch.clientX,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	225 touch.clientY
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	226 ))),
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	227 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	228 for touch in event.touches
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	229 ]
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	230 if self.simulate_mouse:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	231 # In simulate mouse mode, we send mouse events.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	232 if touches:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	233 touch_data = touches[0]
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	234 x, y = touch_data["x"], touch_data["y"]
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	235 self.last_mouse_position = (x, y)
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	236 else:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	237 x, y = self.last_mouse_position
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	238
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	239 mouse_event: dict[str, object] = {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	240 "x": x,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	241 "y": y,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	242 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	243 if event.type == "touchstart":
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	244 mouse_event.update({
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	245 "type": "mousedown",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	246 "buttons": 1,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	247 })
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	248 elif event.type == "touchend":
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	249 mouse_event.update({
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	250 "type": "mouseup",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	251 "buttons": 1,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	252 })
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	253 elif event.type == "touchmove":
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	254 mouse_event.update({
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	255 "type": "mousemove",
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	256 })
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	257 return mouse_event
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	258 else:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	259 # Normal mode, with send touch events.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	260 return {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	261 "type": event.type,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	262 "touches": touches
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	263 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	264 elif event.type == "wheel":
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	265 return {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	266 "type": event.type,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	267 "deltaX": event.deltaX,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	268 "deltaY": event.deltaY,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	269 "deltaZ": event.deltaZ,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	270 "deltaMode": event.deltaMode,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	271 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	272 else:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	273 raise Exception(f"Internal Error: unexpected event {event.type=}")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	274
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	275
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	276 class WebRTC:
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	277
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	278 def __init__(
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	279 self,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	280 screen_sharing_cb=None,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	281 on_connection_established_cb=None,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	282 on_reconnect_cb=None,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	283 on_connection_lost_cb=None,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	284 on_video_devices=None,
1565 d282dbdd5ffd browser (calls): restore UI state on reset: Goffi <goffi@goffi.org> parents: 1564 diff changeset	285 on_reset_cb=None,
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	286 file_only: bool = False,
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	287 extra_data: dict \| None = None,
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	288 ):
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	289 """Initialise WebRTC instance.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	290
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	291 @param screen_sharing_cb: callable function for screen sharing event
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	292 @param on_connection_established_cb: callable function for connection established
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	293 event
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	294 @param on_reconnect_cb: called when a reconnection is triggered.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	295 @param on_connection_lost_cb: called when the connection is lost.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	296 @param on_video_devices: called when new video devices are set.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	297 @param on_reset_cb: called on instance reset.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	298 @param file_only: indicates a file transfer only session.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	299 @param extra_data: optional dictionary containing additional data.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	300 Notably used for file transfer, where ``file_data`` key is used.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	301 """
1565 d282dbdd5ffd browser (calls): restore UI state on reset: Goffi <goffi@goffi.org> parents: 1564 diff changeset	302 # reset
d282dbdd5ffd browser (calls): restore UI state on reset: Goffi <goffi@goffi.org> parents: 1564 diff changeset	303 self.on_reset_cb = on_reset_cb
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	304 self.reset_instance()
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	305
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	306 # ICE events
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	307 bridge.register_signal("ice_candidates_new", self._on_ice_candidates_new)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	308 bridge.register_signal("ice_restart", self._on_ice_restart)
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	309
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	310 # connection events callbacks
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	311 self.on_connection_established_cb = on_connection_established_cb
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	312 self.on_reconnect_cb = on_reconnect_cb
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	313 self.on_connection_lost_cb = on_connection_lost_cb
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	314
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	315 # video devices
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	316 self.on_video_devices = on_video_devices
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	317 self.video_devices = []
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	318 self.has_multiple_cameras = False
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	319 self.current_camera = None
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	320
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	321 self.file_only = file_only
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	322 if not file_only:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	323 # Initially populate the video devices list
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	324 aio.run(self._populate_video_devices())
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	325
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	326 # video elements
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	327 self.local_video_elt = document["local_video"]
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	328 self.remote_video_elt = document["remote_video"]
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	329 else:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	330 self.file_sender = None
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	331
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	332 # muting
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	333 self.is_audio_muted = None
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	334 self.is_video_muted = None
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	335
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	336 # screen sharing
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	337 self._is_sharing_screen = False
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	338 self.screen_sharing_cb = screen_sharing_cb
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	339
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	340 # extra
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	341 if extra_data is None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	342 extra_data = {}
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	343 self.extra_data = extra_data
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	344
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	345 @property
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	346 def is_sharing_screen(self) -> bool:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	347 return self._is_sharing_screen
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	348
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	349 @is_sharing_screen.setter
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	350 def is_sharing_screen(self, sharing: bool) -> None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	351 if sharing != self._is_sharing_screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	352 self._is_sharing_screen = sharing
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	353 if self.screen_sharing_cb is not None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	354 self.screen_sharing_cb(sharing)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	355
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	356 def reset_instance(self):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	357 """Inits or resets the instance variables to their default state."""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	358 self._peer_connection = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	359 self._media_types = None
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	360 self._media_types_inv = None
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	361 self.ufrag = None
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	362 self.pwd = None
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	363 self.sid = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	364 self.local_candidates = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	365 self.remote_stream = None
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	366 self.remote_candidates_buffer = {
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	367 "audio": {"candidates": []},
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	368 "video": {"candidates": []},
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	369 "application": {"candidates": []},
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	370 }
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	371 self.local_candidates_buffer = {}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	372 self.media_candidates = {}
1565 d282dbdd5ffd browser (calls): restore UI state on reset: Goffi <goffi@goffi.org> parents: 1564 diff changeset	373 if self.on_reset_cb is not None:
d282dbdd5ffd browser (calls): restore UI state on reset: Goffi <goffi@goffi.org> parents: 1564 diff changeset	374 self.on_reset_cb()
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	375
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	376 async def _populate_video_devices(self):
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	377 devices = await window.navigator.mediaDevices.enumerateDevices()
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	378 devices_ids = set()
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	379 self.video_devices.clear()
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	380 for device in devices:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	381 if device.kind != "videoinput":
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	382 continue
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	383 # we can have multiple devices with same IDs in some corner cases (e.g.
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	384 # infrared camera)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	385 device_id = device.deviceId
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	386 if device_id in devices_ids:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	387 continue
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	388 devices_ids.add(device_id)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	389 self.video_devices.append(device)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	390 self.has_multiple_cameras = len(self.video_devices) > 1
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	391 if self.on_video_devices is not None:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	392 self.on_video_devices(self.has_multiple_cameras)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	393 # Set the initial camera to the default (usually front on mobile)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	394 if self.video_devices:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	395 self.current_camera = self.video_devices[0].deviceId
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	396 log.debug(
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	397 f"devices populated: {self.video_devices=} {self.has_multiple_cameras=}"
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	398 )
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	399
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	400 @property
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	401 def media_types(self):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	402 if self._media_types is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	403 raise Exception("self._media_types should not be None!")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	404 return self._media_types
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	405
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	406 @media_types.setter
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	407 def media_types(self, new_media_types: dict) -> None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	408 self._media_types = new_media_types
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	409 self._media_types_inv = {v: k for k, v in new_media_types.items()}
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	410
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	411 @property
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	412 def media_types_inv(self) -> dict:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	413 if self._media_types_inv is None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	414 raise Exception("self._media_types_inv should not be None!")
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	415 return self._media_types_inv
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	416
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	417 def get_sdp_mline_index(self, media_type):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	418 """Gets the sdpMLineIndex for a given media type.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	419
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	420 @param media_type: The type of the media.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	421 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	422 for index, m_type in self.media_types.items():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	423 if m_type == media_type:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	424 return index
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	425 raise ValueError(f"Media type '{media_type}' not found")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	426
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	427 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	428 """Retrieves ICE password and user fragment for SDP offer.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	429
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	430 @param sdp: The Session Description Protocol offer string.
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	431 @return: ufrag and pwd
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	432 @raise ValueError: Can't extract ufrag and password
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	433 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	434 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	435 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	436
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	437 if ufrag_line and pwd_line:
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	438 ufrag = self.ufrag = ufrag_line.group(1)
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	439 pwd = self.pwd = pwd_line.group(1)
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	440 return ufrag, pwd
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	441 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	442 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	443 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	444
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	445 def extract_fingerprint_data(self, sdp):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	446 """Retrieves fingerprint data from an SDP offer.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	447
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	448 @param sdp: The Session Description Protocol offer string.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	449 @return: A dictionary containing the fingerprint data.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	450 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	451 fingerprint_line = re.search(r"a=fingerprint:(\S+)\s+(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	452 if fingerprint_line:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	453 algorithm, fingerprint = fingerprint_line.groups()
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	454 fingerprint_data = {"hash": algorithm, "fingerprint": fingerprint}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	455
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	456 setup_line = re.search(r"a=setup:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	457 if setup_line:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	458 setup = setup_line.group(1)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	459 fingerprint_data["setup"] = setup
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	460
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	461 return fingerprint_data
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	462 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	463 raise ValueError("fingerprint should not be missing")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	464
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	465 def parse_ice_candidate(self, candidate_string):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	466 """Parses the ice candidate string.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	467
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	468 @param candidate_string: The ice candidate string to be parsed.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	469 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	470 pattern = re.compile(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	471 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	472 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	473 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	474 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	475 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	476 match = pattern.match(candidate_string)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	477 if match:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	478 candidate_dict = match.groupdict()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	479
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	480 # Apply the correct types to the dictionary values
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	481 candidate_dict["component_id"] = int(candidate_dict["component_id"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	482 candidate_dict["priority"] = int(candidate_dict["priority"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	483 candidate_dict["port"] = int(candidate_dict["port"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	484
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	485 if candidate_dict["rel_port"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	486 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	487
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	488 if candidate_dict["generation"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	489 candidate_dict["generation"] = candidate_dict["generation"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	490
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	491 # Remove None values
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	492 return {k: v for k, v in candidate_dict.items() if v is not None}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	493 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	494 log.warning(f"can't parse candidate: {candidate_string!r}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	495 return None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	496
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	497 def build_ice_candidate(self, parsed_candidate):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	498 """Builds ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	499
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	500 @param parsed_candidate: Dictionary containing parsed ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	501 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	502 base_format = (
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	503 "candidate:{foundation} {component_id} {transport} {priority} "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	504 "{address} {port} typ {type}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	505 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	506
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	507 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	508 base_format += " raddr {rel_addr} rport {rel_port}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	509
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	510 if parsed_candidate.get("generation"):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	511 base_format += " generation {generation}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	512
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	513 return base_format.format(**parsed_candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	514
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	515 def on_ice_candidate(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	516 """Handles ICE candidate event
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	517
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	518 @param event: Event containing the ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	519 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	520 log.debug(f"on ice candidate {event.candidate=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	521 if event.candidate and event.candidate.candidate:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	522 parsed_candidate = self.parse_ice_candidate(event.candidate.candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	523 if parsed_candidate is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	524 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	525 try:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	526 media_type = self.media_types[event.candidate.sdpMLineIndex]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	527 except (TypeError, IndexError):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	528 log.error(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	529 f"Can't find media type.\n{event.candidate=}\n{self._media_types=}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	530 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	531 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	532 self.media_candidates.setdefault(media_type, []).append(parsed_candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	533 log.debug(f"ICE candidate [{media_type}]: {event.candidate.candidate}")
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	534 if self.sid is None:
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	535 log.debug("buffering candidate")
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	536 self.local_candidates_buffer.setdefault(media_type, []).append(
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	537 parsed_candidate
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	538 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	539 else:
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	540 ufrag, pwd = self.extract_ufrag_pwd(
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	541 self._peer_connection.localDescription.sdp
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	542 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	543
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	544 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	545 aio.run(
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	546 bridge.ice_candidates_add(
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	547 self.sid, json.dumps({media_type: ice_data})
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	548 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	549 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	550
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	551 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	552 log.debug("All ICE candidates gathered")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	553
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	554 def on_ice_connection_state_change(self, event):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	555 """Log ICE connection change, mainly used for debugging"""
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	556 state = self._peer_connection.iceConnectionState
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	557 log.info(f"ICE Connection State changed to: {state}")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	558
1561 7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	559 if state == "connected":
7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	560 if self.on_connection_established_cb is not None:
7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	561 self.on_connection_established_cb()
7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	562 elif state == "failed":
1563 e3449beac8d8 browser (calls): Add clear search + formatting Goffi <goffi@goffi.org> parents: 1561 diff changeset	563 log.error(
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	564 "ICE connection failed. Check network connectivity and ICE"
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	565 " configurations."
1563 e3449beac8d8 browser (calls): Add clear search + formatting Goffi <goffi@goffi.org> parents: 1561 diff changeset	566 )
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	567 elif state == "disconnected":
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	568 log.warning("ICE connection was disconnected.")
1561 7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	569 if self.on_connection_lost_cb is not None:
7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	570 self.on_connection_lost_cb()
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	571
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	572 def on_ice_candidate_error(self, event):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	573 """Log ICE error, useful for debugging"""
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	574 log.error(f"ICE Candidate Error: {event.errorText} (Code: {event.errorCode})")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	575 log.debug(
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	576 f"URL: {event.url}, Host candidate: {event.hostCandidate}, Port: {event.port}"
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	577 )
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	578
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	579 def _set_media_types(self, offer):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	580 """Sets media types from offer SDP
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	581
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	582 @param offer: RTC session description containing the offer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	583 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	584 sdp_lines = offer.sdp.splitlines()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	585 media_types = {}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	586 mline_index = 0
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	587
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	588 for line in sdp_lines:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	589 if line.startswith("m="):
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	590 media_types[mline_index] = line[2 : line.find(" ")]
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	591 mline_index += 1
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	592
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	593 self.media_types = media_types
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	594
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	595 def on_ice_gathering_state_change(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	596 """Handles ICE gathering state change
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	597
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	598 @param event: Event containing the ICE gathering state change
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	599 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	600 connection = event.target
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	601 log.debug(f"on_ice_gathering_state_change {connection.iceGatheringState=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	602 if connection.iceGatheringState == "complete":
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	603 log.info("ICE candidates gathering done")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	604
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	605 async def _create_peer_connection(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	606 self,
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	607 ) -> JSObject:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	608 """Creates peer connection"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	609 if self._peer_connection is not None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	610 raise Exception("create_peer_connection can't be called twice!")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	611
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	612 external_disco = json.loads(await bridge.external_disco_get(""))
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	613 ice_servers = []
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	614
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	615 for server in external_disco:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	616 ice_server = {}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	617 if server["type"] == "stun":
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	618 ice_server["urls"] = f"stun:{server['host']}:{server['port']}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	619 elif server["type"] == "turn":
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	620 ice_server["urls"] = (
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	621 f"turn:{server['host']}:{server['port']}?transport={server['transport']}"
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	622 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	623 ice_server["username"] = server["username"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	624 ice_server["credential"] = server["password"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	625 ice_servers.append(ice_server)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	626
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	627 rtc_configuration = {"iceServers": ice_servers}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	628
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	629 peer_connection = window.RTCPeerConnection.new(rtc_configuration)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	630 peer_connection.addEventListener("track", self.on_track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	631 peer_connection.addEventListener("negotiationneeded", self.on_negotiation_needed)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	632 peer_connection.addEventListener(
1563 e3449beac8d8 browser (calls): Add clear search + formatting Goffi <goffi@goffi.org> parents: 1561 diff changeset	633 "iceconnectionstatechange", self.on_ice_connection_state_change
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	634 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	635 peer_connection.addEventListener("icecandidate", self.on_ice_candidate)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	636 peer_connection.addEventListener("icecandidateerror", self.on_ice_candidate_error)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	637 peer_connection.addEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	638 "icegatheringstatechange", self.on_ice_gathering_state_change
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	639 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	640
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	641 self._peer_connection = peer_connection
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	642 window.pc = self._peer_connection
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	643 return peer_connection
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	644
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	645 async def _get_user_media(self, audio: bool = True, video: bool = True) -> None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	646 """
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	647 Gets user media (camera and microphone).
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	648
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	649 @param audio: True if an audio flux is required.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	650 @param video: True if a video flux is required.
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	651 """
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	652 media_constraints = {"audio": audio, "video": video}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	653 local_stream = await window.navigator.mediaDevices.getUserMedia(media_constraints)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	654
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	655 if not local_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	656 log.error("Failed to get the media stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	657 return
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	658
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	659 self.local_video_elt.srcObject = local_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	660
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	661 for track in local_stream.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	662 self._peer_connection.addTrack(track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	663
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	664 async def _replace_user_video(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	665 self,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	666 screen: bool = False,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	667 ) -> JSObject \| None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	668 """Replaces the user video track with either a camera or desktop sharing track.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	669
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	670 @param screen: True if desktop sharing is required. False will use the camera.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	671 @return: The local media stream or None if failed.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	672 """
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	673 if screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	674 media_constraints = {"video": {"cursor": "always"}}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	675 new_stream = await window.navigator.mediaDevices.getDisplayMedia(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	676 media_constraints
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	677 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	678 else:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	679 if self.local_video_elt.srcObject:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	680 for track in self.local_video_elt.srcObject.getTracks():
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	681 if track.kind == "video":
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	682 track.stop()
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	683
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	684 media_constraints = {
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	685 "video": (
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	686 {"deviceId": self.current_camera} if self.current_camera else True
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	687 )
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	688 }
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	689
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	690 new_stream = await window.navigator.mediaDevices.getUserMedia(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	691 media_constraints
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	692 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	693
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	694 if not new_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	695 log.error("Failed to get the media stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	696 return None
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	697
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	698 new_video_tracks = [
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	699 track for track in new_stream.getTracks() if track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	700 ]
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	701
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	702 if not new_video_tracks:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	703 log.error("Failed to retrieve the video track from the new stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	704 return None
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	705
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	706 # Retrieve the current local stream's video track.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	707 local_stream = self.local_video_elt.srcObject
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	708 if local_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	709 local_video_tracks = [
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	710 track for track in local_stream.getTracks() if track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	711 ]
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	712 if local_video_tracks:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	713 # Remove the old video track and add the new one to the local stream.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	714 local_stream.removeTrack(local_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	715 local_stream.addTrack(new_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	716
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	717 video_sender = next(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	718 (
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	719 sender
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	720 for sender in self._peer_connection.getSenders()
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	721 if sender.track and sender.track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	722 ),
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	723 None,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	724 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	725 if video_sender:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	726 await video_sender.replaceTrack(new_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	727
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	728 if screen:
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	729 # For screen sharing, we track the end event to properly stop the sharing
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	730 # when the user clicks on the browser's stop sharing dialog.
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	731 def on_track_ended(event):
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	732 aio.run(self.toggle_screen_sharing())
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	733
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	734 new_video_tracks[0].bind("ended", on_track_ended)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	735
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	736 self.is_sharing_screen = screen
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	737
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	738 return local_stream
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	739
1564 bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	740 async def switch_camera(self) -> None:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	741 """Switches to the next available camera.
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	742
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	743 This method cycles through the list of available video devices, replaces the
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	744 current video track with the next one in the user's local video element, and then
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	745 updates the sender's track in the peer connection. If there's only one camera or
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	746 if an error occurs while switching, the method logs the error and does nothing.
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	747 """
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	748 log.info("switching camera")
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	749 if not self.has_multiple_cameras:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	750 log.error("No multiple cameras to switch.")
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	751 return
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	752
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	753 current_camera_index = -1
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	754 for i, device_info in enumerate(self.video_devices):
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	755 if device_info.deviceId == self.current_camera:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	756 current_camera_index = i
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	757 break
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	758
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	759 if current_camera_index == -1:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	760 log.error("Current camera not found in available devices.")
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	761 return
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	762
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	763 # Switch to the next camera in the list
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	764 next_camera_index = (current_camera_index + 1) % len(self.video_devices)
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	765 self.current_camera = self.video_devices[next_camera_index].deviceId
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	766 log.debug(f"{next_camera_index=} {self.current_camera=}")
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	767
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	768 new_stream = await window.navigator.mediaDevices.getUserMedia(
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	769 {"video": {"deviceId": self.current_camera}}
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	770 )
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	771
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	772 new_video_tracks = [
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	773 track for track in new_stream.getTracks() if track.kind == "video"
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	774 ]
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	775
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	776 if not new_video_tracks:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	777 log.error("Failed to retrieve the video track from the new stream.")
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	778 return
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	779
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	780 # Update local video element's stream
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	781 local_stream = self.local_video_elt.srcObject
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	782 if local_stream:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	783 local_video_tracks = [
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	784 track for track in local_stream.getTracks() if track.kind == "video"
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	785 ]
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	786 if local_video_tracks:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	787 local_video_tracks[0].stop()
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	788 local_stream.removeTrack(local_video_tracks[0])
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	789 local_stream.addTrack(new_video_tracks[0])
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	790 self.local_video_elt.srcObject = local_stream
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	791
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	792 # update remove video stream
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	793 video_sender = next(
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	794 (
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	795 sender
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	796 for sender in self._peer_connection.getSenders()
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	797 if sender.track and sender.track.kind == "video"
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	798 ),
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	799 None,
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	800 )
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	801
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	802 if video_sender:
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	803 await video_sender.replaceTrack(new_video_tracks[0])
bd3c880f4a47 browser (calls): add camera switching: Goffi <goffi@goffi.org> parents: 1563 diff changeset	804
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	805 async def _gather_ice_candidates(self, is_initiator: bool, remote_candidates=None):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	806 """Get ICE candidates and wait to have them all before returning them
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	807
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	808 @param is_initiator: Boolean indicating if the user is the initiator of the connection
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	809 @param remote_candidates: Remote ICE candidates, if any
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	810 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	811 if self._peer_connection is None:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	812 raise Exception(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	813 "The peer connection must be created before gathering ICE candidates!"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	814 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	815
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	816 self.media_candidates.clear()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	817
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	818 if is_initiator:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	819 offer = await self._peer_connection.createOffer()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	820 self._set_media_types(offer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	821 await self._peer_connection.setLocalDescription(offer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	822 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	823 answer = await self._peer_connection.createAnswer()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	824 self._set_media_types(answer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	825 await self._peer_connection.setLocalDescription(answer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	826
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	827 if not is_initiator:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	828 log.debug(self._peer_connection.localDescription.sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	829 log.debug(self._peer_connection.localDescription.sdp)
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	830 ufrag, pwd = self.extract_ufrag_pwd(self._peer_connection.localDescription.sdp)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	831 return {
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	832 "ufrag": ufrag,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	833 "pwd": pwd,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	834 "candidates": self.media_candidates,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	835 }
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	836
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	837 async def accept_call(self, session_id: str, sdp: str, profile: str) -> None:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	838 """Call has been accepted, connection can be established
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	839
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	840 @param session_id: Session identifier
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	841 @param sdp: Session Description Protocol data
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	842 @param profile: Profile associated
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	843 """
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	844 await self._peer_connection.setRemoteDescription({"type": "answer", "sdp": sdp})
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	845 await self.on_ice_candidates_new(self.remote_candidates_buffer)
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	846 self.remote_candidates_buffer.clear()
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	847
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	848 def _on_ice_candidates_new(self, sid: str, candidates_s: str, profile: str) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	849 """Called when new ICE candidates are received
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	850
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	851 @param sid: Session identifier
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	852 @param candidates_s: ICE candidates serialized
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	853 @param profile: Profile associated with the action
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	854 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	855 if sid != self.sid:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	856 log.debug(f"ignoring peer ice candidates for {sid=} ({self.sid=}).")
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	857 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	858 candidates = json.loads(candidates_s)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	859 aio.run(self.on_ice_candidates_new(candidates))
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	860
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	861 def _on_ice_restart(self, sid: str, side: str, profile: str):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	862 if sid != self.sid:
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	863 log.debug(f"ignoring peer ice candidates for {sid=} ({self.sid=}).")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	864 return
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	865 log.debug("ICE has been restarted")
1561 7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	866 if self.on_reconnect_cb is not None:
7dbb131bbb9e browser (calls): update status on various events (connection established, connection lost, etc.) Goffi <goffi@goffi.org> parents: 1559 diff changeset	867 self.on_reconnect_cb()
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	868
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	869 async def on_ice_candidates_new(self, candidates: dict) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	870 """Called when new ICE canidates are received from peer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	871
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	872 @param candidates: Dictionary containing new ICE candidates
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	873 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	874 log.debug(f"new peer candidates received: {candidates}")
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	875
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	876 # try:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	877 # # FIXME: javascript.NULL must be used here, once we move to Brython 3.12.3+
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	878 # remoteDescription_is_none = self._peer_connection.remoteDescription is None
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	879 # except Exception as e:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	880 # # FIXME: should be fine in Brython 3.12.3+
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	881 # log.debug("Workaround for Brython bug activated.")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	882 # remoteDescription_is_none = True
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	883
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	884 if (
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	885 self._peer_connection is None
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	886 or self._peer_connection.remoteDescription is NULL
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	887 ):
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	888 for media_type in ("audio", "video", "application"):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	889 media_candidates = candidates.get(media_type)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	890 if media_candidates:
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	891 buffer = self.remote_candidates_buffer[media_type]
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	892 buffer["candidates"].extend(media_candidates["candidates"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	893 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	894 for media_type, ice_data in candidates.items():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	895 for candidate in ice_data["candidates"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	896 candidate_sdp = self.build_ice_candidate(candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	897 try:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	898 sdp_mline_index = self.get_sdp_mline_index(media_type)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	899 except Exception as e:
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	900 log.warning(f"Can't get sdp_mline_index: {e}")
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	901 continue
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	902 else:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	903 ice_candidate = window.RTCIceCandidate.new(
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	904 {"candidate": candidate_sdp, "sdpMLineIndex": sdp_mline_index}
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	905 )
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	906 await self._peer_connection.addIceCandidate(ice_candidate)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	907
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	908 def on_track(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	909 """New track has been received from peer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	910
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	911 @param event: Event associated with the new track
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	912 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	913 if event.streams and event.streams[0]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	914 remote_stream = event.streams[0]
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	915 self.remote_video_elt.srcObject = remote_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	916 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	917 if self.remote_stream is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	918 self.remote_stream = window.MediaStream.new()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	919 self.remote_video_elt.srcObject = self.remote_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	920 self.remote_stream.addTrack(event.track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	921
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	922 def on_negotiation_needed(self, event) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	923 log.debug(f"on_negotiation_needed {event=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	924 # TODO
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	925
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	926 def _on_data_channel(self, event) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	927 """Handles the data channel event from the peer connection.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	928
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	929 @param event: The event associated with the opening of a data channel.
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	930 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	931 data_channel = event.channel
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	932 self.file_receiver = FileReceiver(self.sid, data_channel, self.extra_data)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	933
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	934 async def answer_call(self, sid: str, offer_sdp: str, profile: str):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	935 """We respond to the call"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	936 log.debug("answering call")
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	937 if sid != self.sid:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	938 raise Exception(f"Internal Error: unexpected sid: {sid=} {self.sid=}")
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	939 await self._create_peer_connection()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	940
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	941 await self._peer_connection.setRemoteDescription(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	942 {"type": "offer", "sdp": offer_sdp}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	943 )
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	944 await self.on_ice_candidates_new(self.remote_candidates_buffer)
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	945 self.remote_candidates_buffer.clear()
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	946 if self.file_only:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	947 self._peer_connection.bind("datachannel", self._on_data_channel)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	948 else:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	949 await self._get_user_media()
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	950
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	951 # Gather local ICE candidates
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	952 local_ice_data = await self._gather_ice_candidates(False)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	953 self.local_candidates = local_ice_data["candidates"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	954
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	955 await bridge.call_answer_sdp(sid, self._peer_connection.localDescription.sdp)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	956
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	957 async def _get_call_data(self) -> dict:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	958 """Start a WebRTC call"""
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	959 await self._gather_ice_candidates(True)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	960
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	961 return {"sdp": self._peer_connection.localDescription.sdp}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	962
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	963 async def _send_buffered_local_candidates(self) -> None:
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	964 if self.local_candidates_buffer:
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	965 log.debug(
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	966 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	967 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	968 assert self.pwd is not None
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	969 ice_data = {}
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	970 for media_type, candidates in self.local_candidates_buffer.items():
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	971 ice_data[media_type] = {
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	972 "ufrag": self.ufrag,
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	973 "pwd": self.pwd,
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	974 "candidates": candidates,
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	975 }
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	976 await bridge.ice_candidates_add(
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	977 self.sid,
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	978 json.dumps(ice_data),
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	979 )
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	980 self.local_candidates_buffer.clear()
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	981
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	982 async def make_call(
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	983 self, callee_jid: jid.JID, audio: bool = True, video: bool = True
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	984 ) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	985 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	986 @param audio: True if an audio flux is required
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	987 @param video: True if a video flux is required
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	988 """
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	989 await self._create_peer_connection()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	990 await self._get_user_media(audio, video)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	991 call_data = await self._get_call_data()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	992 log.info(f"calling {callee_jid!r}")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	993 self.sid = await bridge.call_start(str(callee_jid), json.dumps(call_data))
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	994 log.debug(f"Call SID: {self.sid}")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	995 await self._send_buffered_local_candidates()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	996
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	997 async def start_remote_control(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	998 self, callee_jid: jid.JID, audio: bool = True, video: bool = True
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	999 ) -> None:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1000 """Starts a Remote Control session.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1001
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1002 If both audio and video are False, no screenshare will be done, the input will be
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1003 sent without feedback.
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1004 @param audio: True if an audio flux is required
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1005 @param video: True if a video flux is required
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1006 """
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1007 if audio or not video:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1008 raise NotImplementedError("Only video screenshare is supported for now.")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1009 peer_connection = await self._create_peer_connection()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1010 if video:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1011 peer_connection.addTransceiver("video", {"direction": "recvonly"})
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1012 data_channel = peer_connection.createDataChannel("remote-control")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1013
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1014 call_data = await self._get_call_data()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1015
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1016 try:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1017 remote_control_data = json.loads(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1018 await bridge.remote_control_start(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1019 str(callee_jid),
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1020 json.dumps(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1021 {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1022 "devices": {
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1023 "keyboard": {},
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1024 "mouse": {},
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1025 "wheel": {}
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1026 },
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1027 "call_data": call_data,
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1028 }
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1029 ),
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1030 )
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1031 )
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1032 except Exception as e:
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1033 dialog.notification.show(f"Can't start remote control: {e}", level="error")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1034 return
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1035
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1036 self.sid = remote_control_data["session_id"]
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1037
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1038 log.debug(f"Remote Control SID: {self.sid}")
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1039 await self._send_buffered_local_candidates()
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1040 self.remote_controller = RemoteControler(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1041 self.sid, self.remote_video_elt, data_channel
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1042 )
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1043
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1044 def _on_opened_data_channel(self, event):
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1045 log.info("Datachannel has been opened.")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1046
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1047 async def send_file(self, callee_jid: jid.JID, file: JSObject) -> None:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1048 assert self.file_only
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1049 peer_connection = await self._create_peer_connection()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1050 data_channel = peer_connection.createDataChannel("file")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1051 call_data = await self._get_call_data()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1052 log.info(f"sending file to {callee_jid!r}")
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1053 file_meta = {"size": file.size}
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1054 if file.type:
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1055 file_meta["media_type"] = file.type
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1056
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1057 try:
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1058 file_data = json.loads(
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1059 await bridge.file_jingle_send(
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1060 str(callee_jid),
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1061 "",
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1062 file.name,
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1063 "",
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1064 json.dumps({"webrtc": True, "call_data": call_data, **file_meta}),
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1065 )
6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1066 )
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1067 except Exception as e:
1602 6feac4a25e60 browser: Remote Control implementation: Goffi <goffi@goffi.org> parents: 1600 diff changeset	1068 dialog.notification.show(f"Can't send file: {e}", level="error")
1600 0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1069 return
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1070
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1071 self.sid = file_data["session_id"]
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1072
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1073 log.debug(f"File Transfer SID: {self.sid}")
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1074 await self._send_buffered_local_candidates()
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1075 self.file_sender = FileSender(self.sid, file, data_channel)
0a4433a343a3 browser (calls): implement WebRTC file sharing: Goffi <goffi@goffi.org> parents: 1577 diff changeset	1076
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1077 async def end_call(self) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1078 """Stop streaming and clean instance"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1079 if self._peer_connection is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1080 log.debug("There is currently no call to end.")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1081 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1082 self._peer_connection.removeEventListener("track", self.on_track)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1083 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1084 "negotiationneeded", self.on_negotiation_needed
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1085 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1086 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1087 "icecandidate", self.on_ice_candidate
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1088 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1089 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1090 "icegatheringstatechange", self.on_ice_gathering_state_change
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1091 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1092
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1093 # Base64 encoded 1x1 black pixel image
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1094 # this is a trick to reset the image displayed, so we don't see last image of
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1095 # last stream
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1096 black_image_data = (
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1097 "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0"
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1098 "lEQVR42mP8/wcAAwAB/uzNq7sAAAAASUVORK5CYII="
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1099 )
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1100
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1101 local_video = self.local_video_elt
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1102 remote_video = self.remote_video_elt
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1103 if local_video.srcObject:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1104 for track in local_video.srcObject.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1105 track.stop()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1106 local_video.src = black_image_data
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1107
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1108 if remote_video.srcObject:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1109 for track in remote_video.srcObject.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1110 track.stop()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1111 remote_video.src = black_image_data
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1112
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1113 self._peer_connection.close()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1114 self.reset_instance()
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1115
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1116 def toggle_media_mute(self, media_type: str) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1117 """Toggle mute/unmute for media tracks.
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1118
1566 e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	1119 @param media_type: "audio" or "video". Determines which media tracks
e65d2ef1ded4 browser (calls/webrtc): send ICE candidates when received: Goffi <goffi@goffi.org> parents: 1565 diff changeset	1120 to process.
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1121 """
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1122 assert media_type in ("audio", "video"), "Invalid media type"
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1123
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1124 local_video = self.local_video_elt
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1125 is_muted_attr = f"is_{media_type}_muted"
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1126
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1127 if local_video.srcObject:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1128 track_getter = getattr(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1129 local_video.srcObject, f"get{media_type.capitalize()}Tracks"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1130 )
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1131 for track in track_getter():
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1132 track.enabled = not track.enabled
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1133 setattr(self, is_muted_attr, not track.enabled)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1134
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1135 media_name = self.media_types_inv.get(media_type)
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1136 if media_name is not None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1137 extra = {"name": str(media_name)}
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1138 aio.run(
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1139 bridge.call_info(
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1140 self.sid,
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1141 "mute" if getattr(self, is_muted_attr) else "unmute",
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1142 json.dumps(extra),
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1143 )
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1144 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1145
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1146 return getattr(self, is_muted_attr)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1147
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1148 def toggle_audio_mute(self) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1149 """Toggle mute/unmute for audio tracks."""
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1150 return self.toggle_media_mute("audio")
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1151
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1152 def toggle_video_mute(self) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1153 """Toggle mute/unmute for video tracks."""
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	1154 return self.toggle_media_mute("video")
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1155
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1156 async def toggle_screen_sharing(self):
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1157 log.debug(f"toggle_screen_sharing {self._is_sharing_screen=}")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1158
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1159 if self._is_sharing_screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1160 await self._replace_user_video(screen=False)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1161 else:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	1162 await self._replace_user_video(screen=True)

Mercurial > libervia-web

annotate libervia/web/pages/calls/_browser/webrtc.py @ 1602:6feac4a25e60