libervia-web: libervia/web/pages/calls/_browser/webrtc.py annotate

annotate libervia/web/pages/calls/_browser/webrtc.py @ 1559:410064b31dca

browser (calls): add some logs useful for debugging

author	Goffi <goffi@goffi.org>
date	Wed, 16 Aug 2023 16:52:53 +0200
parents	83c2a6faa2ae
children	7dbb131bbb9e

rev	line source
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	1 import json
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	2 import re
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	3
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	4 from bridge import AsyncBridge as Bridge
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	5 from browser import aio, console as log, document, timer, window
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	6 import dialog
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	7 import errors
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	8 from javascript import JSObject
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	9 import jid
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	10
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	11 log.warning = log.warn
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	12 profile = window.profile or ""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	13 bridge = Bridge()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	14 GATHER_TIMEOUT = 10000
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	15
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	16
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	17 class WebRTC:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	18 def __init__(self):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	19 self.reset_instance()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	20 bridge.register_signal("ice_candidates_new", self._on_ice_candidates_new)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	21 bridge.register_signal("ice_restart", self._on_ice_restart)
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	22 self.is_audio_muted = None
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	23 self.is_video_muted = None
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	24 self._is_sharing_screen = False
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	25 self.screen_sharing_cb = None
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	26 self.local_video_elt = document["local_video"]
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	27 self.remote_video_elt = document["remote_video"]
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	28
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	29 @property
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	30 def is_sharing_screen(self) -> bool:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	31 return self._is_sharing_screen
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	32
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	33 @is_sharing_screen.setter
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	34 def is_sharing_screen(self, sharing: bool) -> None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	35 if sharing != self._is_sharing_screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	36 self._is_sharing_screen = sharing
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	37 if self.screen_sharing_cb is not None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	38 self.screen_sharing_cb(sharing)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	39
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	40 def reset_instance(self):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	41 """Inits or resets the instance variables to their default state."""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	42 self._peer_connection = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	43 self._media_types = None
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	44 self._media_types_inv = None
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	45 self._callee = None
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	46 self.sid = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	47 self.local_candidates = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	48 self.remote_stream = None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	49 self.candidates_buffer = {
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	50 "audio": {"candidates": []},
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	51 "video": {"candidates": []},
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	52 }
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	53 self.media_candidates = {}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	54 self.candidates_gathered = aio.Future()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	55
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	56 @property
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	57 def media_types(self):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	58 if self._media_types is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	59 raise Exception("self._media_types should not be None!")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	60 return self._media_types
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	61
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	62 @media_types.setter
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	63 def media_types(self, new_media_types: dict) -> None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	64 self._media_types = new_media_types
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	65 self._media_types_inv = {v: k for k, v in new_media_types.items()}
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	66
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	67 @property
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	68 def media_types_inv(self) -> dict:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	69 if self._media_types_inv is None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	70 raise Exception("self._media_types_inv should not be None!")
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	71 return self._media_types_inv
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	72
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	73 def get_sdp_mline_index(self, media_type):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	74 """Gets the sdpMLineIndex for a given media type.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	75
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	76 @param media_type: The type of the media.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	77 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	78 for index, m_type in self.media_types.items():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	79 if m_type == media_type:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	80 return index
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	81 raise ValueError(f"Media type '{media_type}' not found")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	82
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	83 def extract_pwd_ufrag(self, sdp):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	84 """Retrieves ICE password and user fragment for SDP offer.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	85
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	86 @param sdp: The Session Description Protocol offer string.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	87 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	88 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	89 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	90
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	91 if ufrag_line and pwd_line:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	92 return ufrag_line.group(1), pwd_line.group(1)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	93 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	94 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	95 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	96
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	97 def extract_fingerprint_data(self, sdp):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	98 """Retrieves fingerprint data from an SDP offer.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	99
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	100 @param sdp: The Session Description Protocol offer string.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	101 @return: A dictionary containing the fingerprint data.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	102 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	103 fingerprint_line = re.search(r"a=fingerprint:(\S+)\s+(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	104 if fingerprint_line:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	105 algorithm, fingerprint = fingerprint_line.groups()
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	106 fingerprint_data = {"hash": algorithm, "fingerprint": fingerprint}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	107
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	108 setup_line = re.search(r"a=setup:(\S+)", sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	109 if setup_line:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	110 setup = setup_line.group(1)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	111 fingerprint_data["setup"] = setup
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	112
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	113 return fingerprint_data
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	114 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	115 raise ValueError("fingerprint should not be missing")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	116
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	117 def parse_ice_candidate(self, candidate_string):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	118 """Parses the ice candidate string.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	119
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	120 @param candidate_string: The ice candidate string to be parsed.
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	121 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	122 pattern = re.compile(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	123 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	124 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	125 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	126 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	127 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	128 match = pattern.match(candidate_string)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	129 if match:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	130 candidate_dict = match.groupdict()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	131
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	132 # Apply the correct types to the dictionary values
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	133 candidate_dict["component_id"] = int(candidate_dict["component_id"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	134 candidate_dict["priority"] = int(candidate_dict["priority"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	135 candidate_dict["port"] = int(candidate_dict["port"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	136
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	137 if candidate_dict["rel_port"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	138 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	139
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	140 if candidate_dict["generation"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	141 candidate_dict["generation"] = candidate_dict["generation"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	142
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	143 # Remove None values
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	144 return {k: v for k, v in candidate_dict.items() if v is not None}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	145 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	146 log.warning(f"can't parse candidate: {candidate_string!r}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	147 return None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	148
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	149 def build_ice_candidate(self, parsed_candidate):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	150 """Builds ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	151
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	152 @param parsed_candidate: Dictionary containing parsed ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	153 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	154 base_format = (
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	155 "candidate:{foundation} {component_id} {transport} {priority} "
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	156 "{address} {port} typ {type}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	157 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	158
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	159 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	160 base_format += " raddr {rel_addr} rport {rel_port}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	161
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	162 if parsed_candidate.get("generation"):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	163 base_format += " generation {generation}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	164
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	165 return base_format.format(**parsed_candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	166
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	167 def on_ice_candidate(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	168 """Handles ICE candidate event
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	169
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	170 @param event: Event containing the ICE candidate
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	171 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	172 log.debug(f"on ice candidate {event.candidate=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	173 if event.candidate and event.candidate.candidate:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	174 window.last_event = event
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	175 parsed_candidate = self.parse_ice_candidate(event.candidate.candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	176 if parsed_candidate is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	177 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	178 try:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	179 media_type = self.media_types[event.candidate.sdpMLineIndex]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	180 except (TypeError, IndexError):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	181 log.error(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	182 f"Can't find media type.\n{event.candidate=}\n{self._media_types=}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	183 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	184 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	185 self.media_candidates.setdefault(media_type, []).append(parsed_candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	186 log.debug(f"ICE candidate [{media_type}]: {event.candidate.candidate}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	187 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	188 log.debug("All ICE candidates gathered")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	189
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	190 def on_ice_connection_state_change(self, event):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	191 """Log ICE connection change, mainly used for debugging"""
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	192 state = self._peer_connection.iceConnectionState
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	193 log.info(f"ICE Connection State changed to: {state}")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	194
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	195 if state == "failed":
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	196 log.error("ICE connection failed. Check network connectivity and ICE configurations.")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	197 elif state == "disconnected":
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	198 log.warning("ICE connection was disconnected.")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	199
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	200 def on_ice_candidate_error(self, event):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	201 """Log ICE error, useful for debugging"""
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	202 log.error(f"ICE Candidate Error: {event.errorText} (Code: {event.errorCode})")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	203 log.debug(
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	204 f"URL: {event.url}, Host candidate: {event.hostCandidate}, Port: {event.port}"
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	205 )
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	206
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	207 def _set_media_types(self, offer):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	208 """Sets media types from offer SDP
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	209
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	210 @param offer: RTC session description containing the offer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	211 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	212 sdp_lines = offer.sdp.splitlines()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	213 media_types = {}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	214 mline_index = 0
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	215
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	216 for line in sdp_lines:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	217 if line.startswith("m="):
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	218 media_types[mline_index] = line[2 : line.find(" ")]
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	219 mline_index += 1
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	220
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	221 self.media_types = media_types
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	222
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	223 def on_ice_gathering_state_change(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	224 """Handles ICE gathering state change
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	225
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	226 @param event: Event containing the ICE gathering state change
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	227 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	228 connection = event.target
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	229 log.debug(f"on_ice_gathering_state_change {connection.iceGatheringState=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	230 if connection.iceGatheringState == "complete":
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	231 log.info("ICE candidates gathering done")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	232 self.candidates_gathered.set_result(None)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	233
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	234 async def _create_peer_connection(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	235 self,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	236 ):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	237 """Creates peer connection"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	238 if self._peer_connection is not None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	239 raise Exception("create_peer_connection can't be called twice!")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	240
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	241 external_disco = json.loads(await bridge.external_disco_get(""))
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	242 ice_servers = []
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	243
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	244 for server in external_disco:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	245 ice_server = {}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	246 if server["type"] == "stun":
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	247 ice_server["urls"] = f"stun:{server['host']}:{server['port']}"
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	248 elif server["type"] == "turn":
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	249 ice_server[
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	250 "urls"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	251 ] = f"turn:{server['host']}:{server['port']}?transport={server['transport']}"
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	252 ice_server["username"] = server["username"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	253 ice_server["credential"] = server["password"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	254 ice_servers.append(ice_server)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	255
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	256 rtc_configuration = {"iceServers": ice_servers}
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	257
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	258 peer_connection = window.RTCPeerConnection.new(rtc_configuration)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	259 peer_connection.addEventListener("track", self.on_track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	260 peer_connection.addEventListener("negotiationneeded", self.on_negotiation_needed)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	261 peer_connection.addEventListener(
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	262 "iceconnectionstatechange",
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	263 self.on_ice_connection_state_change
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	264 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	265 peer_connection.addEventListener("icecandidate", self.on_ice_candidate)
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	266 peer_connection.addEventListener("icecandidateerror", self.on_ice_candidate_error)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	267 peer_connection.addEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	268 "icegatheringstatechange", self.on_ice_gathering_state_change
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	269 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	270
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	271 self._peer_connection = peer_connection
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	272 window.pc = self._peer_connection
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	273
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	274 async def _get_user_media(self, audio: bool = True, video: bool = True) -> None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	275 """
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	276 Gets user media (camera and microphone).
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	277
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	278 @param audio: True if an audio flux is required.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	279 @param video: True if a video flux is required.
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	280 """
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	281 media_constraints = {"audio": audio, "video": video}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	282 local_stream = await window.navigator.mediaDevices.getUserMedia(media_constraints)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	283
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	284 if not local_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	285 log.error("Failed to get the media stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	286 return
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	287
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	288 self.local_video_elt.srcObject = local_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	289
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	290 for track in local_stream.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	291 self._peer_connection.addTrack(track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	292
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	293 async def _replace_user_video(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	294 self,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	295 screen: bool = False,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	296 ) -> JSObject \| None:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	297 """Replaces the user video track with either a camera or desktop sharing track.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	298
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	299 @param screen: True if desktop sharing is required. False will use the camera.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	300 @return: The local media stream or None if failed.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	301 """
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	302 if screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	303 media_constraints = {"video": {"cursor": "always"}}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	304 new_stream = await window.navigator.mediaDevices.getDisplayMedia(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	305 media_constraints
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	306 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	307 else:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	308 if self.local_video_elt.srcObject:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	309 for track in self.local_video_elt.srcObject.getTracks():
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	310 if track.kind == "video":
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	311 track.stop()
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	312 media_constraints = {"video": True}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	313 new_stream = await window.navigator.mediaDevices.getUserMedia(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	314 media_constraints
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	315 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	316
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	317 if not new_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	318 log.error("Failed to get the media stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	319 return None
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	320
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	321 new_video_tracks = [
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	322 track for track in new_stream.getTracks() if track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	323 ]
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	324
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	325 if not new_video_tracks:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	326 log.error("Failed to retrieve the video track from the new stream.")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	327 return None
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	328
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	329 # Retrieve the current local stream's video track.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	330 local_stream = self.local_video_elt.srcObject
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	331 if local_stream:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	332 local_video_tracks = [
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	333 track for track in local_stream.getTracks() if track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	334 ]
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	335 if local_video_tracks:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	336 # Remove the old video track and add the new one to the local stream.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	337 local_stream.removeTrack(local_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	338 local_stream.addTrack(new_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	339
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	340 video_sender = next(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	341 (
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	342 sender
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	343 for sender in self._peer_connection.getSenders()
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	344 if sender.track and sender.track.kind == "video"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	345 ),
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	346 None,
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	347 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	348 if video_sender:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	349 await video_sender.replaceTrack(new_video_tracks[0])
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	350
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	351 if screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	352 # For screen sharing, we track the end event to properly stop the sharing when
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	353 # the user clicks on the browser's stop sharing dialog.
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	354 def on_track_ended(event):
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	355 aio.run(self.toggle_screen_sharing())
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	356
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	357 new_video_tracks[0].bind("ended", on_track_ended)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	358
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	359 self.is_sharing_screen = screen
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	360
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	361 return local_stream
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	362
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	363 async def _gather_ice_candidates(self, is_initiator: bool, remote_candidates=None):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	364 """Get ICE candidates and wait to have them all before returning them
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	365
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	366 @param is_initiator: Boolean indicating if the user is the initiator of the connection
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	367 @param remote_candidates: Remote ICE candidates, if any
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	368 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	369 if self._peer_connection is None:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	370 raise Exception(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	371 "The peer connection must be created before gathering ICE candidates!"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	372 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	373
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	374 self.media_candidates.clear()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	375 gather_timeout = timer.set_timeout(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	376 lambda: self.candidates_gathered.set_exception(
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	377 errors.TimeoutError("ICE gathering time out")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	378 ),
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	379 GATHER_TIMEOUT,
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	380 )
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	381
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	382 if is_initiator:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	383 offer = await self._peer_connection.createOffer()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	384 self._set_media_types(offer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	385 await self._peer_connection.setLocalDescription(offer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	386 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	387 answer = await self._peer_connection.createAnswer()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	388 self._set_media_types(answer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	389 await self._peer_connection.setLocalDescription(answer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	390
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	391 if not is_initiator:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	392 log.debug(self._peer_connection.localDescription.sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	393 await self.candidates_gathered
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	394 log.debug(self._peer_connection.localDescription.sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	395 timer.clear_timeout(gather_timeout)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	396 ufrag, pwd = self.extract_pwd_ufrag(self._peer_connection.localDescription.sdp)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	397 return {
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	398 "ufrag": ufrag,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	399 "pwd": pwd,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	400 "candidates": self.media_candidates,
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	401 }
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	402
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	403 async def accept_call(self, session_id: str, sdp: str, profile: str) -> None:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	404 """Call has been accepted, connection can be established
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	405
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	406 @param session_id: Session identifier
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	407 @param sdp: Session Description Protocol data
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	408 @param profile: Profile associated
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	409 """
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	410 await self._peer_connection.setRemoteDescription({"type": "answer", "sdp": sdp})
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	411 await self.on_ice_candidates_new(self.candidates_buffer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	412 self.candidates_buffer.clear()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	413
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	414 def _on_ice_candidates_new(self, sid: str, candidates_s: str, profile: str) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	415 """Called when new ICE candidates are received
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	416
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	417 @param sid: Session identifier
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	418 @param candidates_s: ICE candidates serialized
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	419 @param profile: Profile associated with the action
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	420 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	421 if sid != self.sid:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	422 log.debug(f"ignoring peer ice candidates for {sid=} ({self.sid=}).")
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	423 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	424 candidates = json.loads(candidates_s)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	425 aio.run(self.on_ice_candidates_new(candidates))
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	426
1559 410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	427 def _on_ice_restart(self, sid: str, side: str, profile: str):
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	428 if sid != self.sid:
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	429 log.debug(f"ignoring peer ice candidates for {sid=} ({self.sid=}).")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	430 return
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	431 log.debug("ICE has been restarted")
410064b31dca browser (calls): add some logs useful for debugging Goffi <goffi@goffi.org> parents: 1553 diff changeset	432
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	433 async def on_ice_candidates_new(self, candidates: dict) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	434 """Called when new ICE canidates are received from peer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	435
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	436 @param candidates: Dictionary containing new ICE candidates
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	437 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	438 log.debug(f"new peer candidates received: {candidates}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	439 if (
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	440 self._peer_connection is None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	441 or self._peer_connection.remoteDescription is None
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	442 ):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	443 for media_type in ("audio", "video"):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	444 media_candidates = candidates.get(media_type)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	445 if media_candidates:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	446 buffer = self.candidates_buffer[media_type]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	447 buffer["candidates"].extend(media_candidates["candidates"])
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	448 return
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	449 for media_type, ice_data in candidates.items():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	450 for candidate in ice_data["candidates"]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	451 candidate_sdp = self.build_ice_candidate(candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	452 try:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	453 sdp_mline_index = self.get_sdp_mline_index(media_type)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	454 except Exception as e:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	455 log.warning(e)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	456 continue
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	457 ice_candidate = window.RTCIceCandidate.new(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	458 {"candidate": candidate_sdp, "sdpMLineIndex": sdp_mline_index}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	459 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	460 await self._peer_connection.addIceCandidate(ice_candidate)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	461
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	462 def on_track(self, event):
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	463 """New track has been received from peer
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	464
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	465 @param event: Event associated with the new track
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	466 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	467 if event.streams and event.streams[0]:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	468 remote_stream = event.streams[0]
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	469 self.remote_video_elt.srcObject = remote_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	470 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	471 if self.remote_stream is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	472 self.remote_stream = window.MediaStream.new()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	473 self.remote_video_elt.srcObject = self.remote_stream
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	474 self.remote_stream.addTrack(event.track)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	475
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	476 def on_negotiation_needed(self, event) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	477 log.debug(f"on_negotiation_needed {event=}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	478 # TODO
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	479
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	480 async def answer_call(self, sid: str, offer_sdp: str, profile: str):
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	481 """We respond to the call"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	482 log.debug("answering call")
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	483 if sid != self.sid:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	484 raise Exception(f"Internal Error: unexpected sid: {sid=} {self.sid=}")
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	485 await self._create_peer_connection()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	486
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	487 await self._peer_connection.setRemoteDescription(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	488 {"type": "offer", "sdp": offer_sdp}
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	489 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	490 await self.on_ice_candidates_new(self.candidates_buffer)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	491 self.candidates_buffer.clear()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	492 await self._get_user_media()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	493
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	494 # Gather local ICE candidates
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	495 local_ice_data = await self._gather_ice_candidates(False)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	496 self.local_candidates = local_ice_data["candidates"]
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	497
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	498 await bridge.call_answer_sdp(sid, self._peer_connection.localDescription.sdp)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	499
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	500 async def make_call(
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	501 self, callee_jid: jid.JID, audio: bool = True, video: bool = True
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	502 ) -> None:
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	503 """Start a WebRTC call
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	504
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	505 @param audio: True if an audio flux is required
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	506 @param video: True if a video flux is required
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	507 """
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	508 await self._create_peer_connection()
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	509 await self._get_user_media(audio, video)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	510 await self._gather_ice_candidates(True)
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	511
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	512 call_data = {"sdp": self._peer_connection.localDescription.sdp}
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	513 log.info(f"calling {callee_jid!r}")
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	514 self.sid = await bridge.call_start(str(callee_jid), json.dumps(call_data))
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	515 log.debug(f"Call SID: {self.sid}")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	516
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	517 async def end_call(self) -> None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	518 """Stop streaming and clean instance"""
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	519 if self._peer_connection is None:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	520 log.debug("There is currently no call to end.")
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	521 else:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	522 self._peer_connection.removeEventListener("track", self.on_track)
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	523 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	524 "negotiationneeded", self.on_negotiation_needed
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	525 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	526 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	527 "icecandidate", self.on_ice_candidate
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	528 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	529 self._peer_connection.removeEventListener(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	530 "icegatheringstatechange", self.on_ice_gathering_state_change
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	531 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	532
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	533 # Base64 encoded 1x1 black pixel image
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	534 # this is a trick to reset the image displayed, so we don't see last image of
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	535 # last stream
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	536 black_image_data = (
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	537 "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0"
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	538 "lEQVR42mP8/wcAAwAB/uzNq7sAAAAASUVORK5CYII="
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	539 )
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	540
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	541 local_video = self.local_video_elt
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	542 remote_video = self.remote_video_elt
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	543 if local_video.srcObject:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	544 for track in local_video.srcObject.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	545 track.stop()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	546 local_video.src = black_image_data
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	547
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	548 if remote_video.srcObject:
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	549 for track in remote_video.srcObject.getTracks():
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	550 track.stop()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	551 remote_video.src = black_image_data
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	552
b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	553 self._peer_connection.close()
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	554 self.reset_instance()
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	555
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	556 def toggle_media_mute(self, media_type: str) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	557 """Toggle mute/unmute for media tracks.
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	558
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	559 @param media_type: 'audio' or 'video'. Determines which media tracks
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	560 to process.
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	561 """
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	562 assert media_type in ("audio", "video"), "Invalid media type"
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	563
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	564 local_video = self.local_video_elt
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	565 is_muted_attr = f"is_{media_type}_muted"
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	566
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	567 if local_video.srcObject:
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	568 log.debug(f"{local_video.srcObject=}")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	569 track_getter = getattr(
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	570 local_video.srcObject, f"get{media_type.capitalize()}Tracks"
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	571 )
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	572 log.debug("track go")
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	573 for track in track_getter():
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	574 log.debug(f"{track=}")
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	575 track.enabled = not track.enabled
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	576 setattr(self, is_muted_attr, not track.enabled)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	577
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	578 media_name = self.media_types_inv.get(media_type)
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	579 if media_name is not None:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	580 extra = {"name": str(media_name)}
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	581 aio.run(
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	582 bridge.call_info(
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	583 self.sid,
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	584 "mute" if getattr(self, is_muted_attr) else "unmute",
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	585 json.dumps(extra),
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	586 )
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	587 )
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	588
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	589 return getattr(self, is_muted_attr)
1517 b8ed9726525b browser: "calls" implementation, first draft: Goffi <goffi@goffi.org> parents: diff changeset	590
1549 e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	591 def toggle_audio_mute(self) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	592 """Toggle mute/unmute for audio tracks."""
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	593 return self.toggle_media_mute("audio")
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	594
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	595 def toggle_video_mute(self) -> bool:
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	596 """Toggle mute/unmute for video tracks."""
e47c24204449 browser (calls): update call to handle search, control buttons, and better UI/UX: Goffi <goffi@goffi.org> parents: 1518 diff changeset	597 return self.toggle_media_mute("video")
1553 83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	598
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	599 async def toggle_screen_sharing(self):
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	600 log.debug(f"toggle_screen_sharing {self._is_sharing_screen=}")
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	601
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	602 if self._is_sharing_screen:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	603 await self._replace_user_video(screen=False)
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	604 else:
83c2a6faa2ae browser (calls): screen sharing implementation: Goffi <goffi@goffi.org> parents: 1549 diff changeset	605 await self._replace_user_video(screen=True)

Mercurial > libervia-web

annotate libervia/web/pages/calls/_browser/webrtc.py @ 1559:410064b31dca