fix(agent): remote-control input now works (data channel + nut-js API)
Two latent bugs that broke input control on any setup: - Data channel was created by the viewer (the answerer), so the agent's offer had no SCTP m-line and the channel never opened -> no input reached the agent. The agent (offerer) now creates the 'input' channel; the viewer receives it. - inject.js used nut.screen.getResolution() which doesn't exist in this nut-js; switched to screen.width()/height() with per-session caching. Verified end-to-end locally: screen streams + mouse injection moves the remote cursor. Also commits desktop/ + mobile/ package-lock.json from client installs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -104,13 +104,13 @@ async function startStreaming() {
|
||||
pc = new RTCPeerConnection({ iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] });
|
||||
localStream.getTracks().forEach((t) => pc.addTrack(t, localStream));
|
||||
|
||||
// Viewer creates the input data channel; we receive it here.
|
||||
pc.ondatachannel = (ev) => {
|
||||
const ch = ev.channel;
|
||||
ch.onmessage = (msg) => {
|
||||
let evt; try { evt = JSON.parse(msg.data); } catch { return; }
|
||||
window.agent.injectInput(evt); // -> main process -> OS injection
|
||||
};
|
||||
// The agent is the OFFERER, so it must create the input data channel — otherwise the
|
||||
// SCTP m-line is absent from the offer and the channel never negotiates (viewer's stays
|
||||
// closed, so no input arrives). The viewer receives this channel via ondatachannel.
|
||||
const inputCh = pc.createDataChannel('input', { ordered: true });
|
||||
inputCh.onmessage = (msg) => {
|
||||
let evt; try { evt = JSON.parse(msg.data); } catch { return; }
|
||||
window.agent.injectInput(evt); // -> main process -> OS injection
|
||||
};
|
||||
|
||||
pc.onicecandidate = (ev) => {
|
||||
|
||||
Reference in New Issue
Block a user