ZAMA Emotion Model icon ZAMA Emotion Model — Valence-arousal emotion engine generating persistent, configurable robot moods and event-driven emotional outputs for expressive behaviors. Try it →
South Africa
Asked
1 Participant
Remote Mouse mapping with dual monitors

Remote Mouse Mapping With Dual Monitors

I’m using a Six Hexapod (EZ-Robot) connected over WiFi to ARC on a Windows laptop, and trying to drive the PC cursor with the Remote Mouse skill from a Script skill using camera tracking. The Camera Device is set to 320x240, tracking a green ball, and I’m reading $CameraObjectX / $CameraObjectY, mapping them to screen coordinates, then calling SetAbsolute. On a single display it’s close enough, but on my desk setup (laptop 1920x1080 @100% on the left as secondary, external 2560x1440 @125% on the right as primary) the cursor jumps to the wrong place or off by a big offset. The Remote Mouse pointer display shows values larger than a single screen width (e.g., ~3900 on the far right), so I think SetAbsolute uses the Windows virtual desktop. Here’s the simplified bit I’m using:

# camera 320x240 mapped to primary display size
$mx = Round(($CameraObjectX / 320) * 2560)
$my = Round(($CameraObjectY / 240) * 1440)
ControlCommand("Remote Mouse", SetAbsolute, $mx, $my)

Things I’ve tried: hard-coded offsets for the left/secondary monitor; flipping which display is primary; changing DPI scaling; using relative MoveRight/MoveDown instead of SetAbsolute (drifts over time); smoothing with a rolling average to tame jitter; and WindowMaximize/WindowMove tests to confirm the ARC window position. My suspicion is I need the virtual desktop origin and the target monitor’s bounds to compute the correct absolute values, especially when the secondary is to the left (negative X region) and with DPI scaling. Is there a recommended way in ARC to get virtual desktop dimensions/offsets or force Remote Mouse SetAbsolute to a specific monitor so this mapping is accurate across multi-monitor and DPI setups?



Related Hardware (view all EZB hardware)
Six Hexapod by EZ-Robot
Synthiam ARCx controls Six Hexapod with Auto Position gaits, joystick, voice, AI, vision tracking, and browser access.
Wi-Fi / USB
Servos 24
Camera
Audio
UART 3
I2C
ADC 8
Digital 24

Related Robot Skills (view all robot skills)
Script by Synthiam
Multi-language ARC Script: build, run and debug Blockly, JavaScript, EZ-Script or Python with Intellisense, run/save/load and Roboscratch support
Remote Mouse by Synthiam
Scriptable Remote Mouse for ARC: control PC cursor and ARC window via ControlCommand, using voice, camera or I/O inputs; shows cursor coordinates.

ARC Pro

Upgrade to ARC Pro

Unleash your creativity with the power of easy robot programming using Synthiam ARC Pro

Author Avatar
PRO
Canada
Twitter YouTube GitHub
#1  

Yes - your suspicion is right: Remote Mouse uses the Windows virtual desktop coordinates, not the bounds of a single monitor. That’s why SetAbsolute is landing in the wrong spot when you have:

  • multiple monitors,
  • different DPI scaling,
  • and a monitor arranged to the left of the primary (negative X region).

The important part

Your current math assumes:

  • camera 320x240
  • target screen 2560x1440

But SetAbsolute is not targeting "the primary monitor." It’s targeting the entire desktop space, which may be something like:

  • left monitor: -1920 to 0
  • right monitor: 0 to 2560

So if you map directly to 0..2560, the cursor will be offset whenever the actual virtual desktop origin isn’t 0,0.


There isn’t a built-in ARC command that directly returns Windows virtual desktop bounds/monitor rectangles from the Remote Mouse skill itself.

Best practice

Use JavaScript or Python in the Script skill and compute your own mapping based on the monitor you want to control, then send the coordinates to Remote Mouse.

If you only want to control one specific monitor, the cleanest solution is:

  1. Define that monitor’s rectangle in virtual desktop coordinates
  2. Map camera coordinates into that rectangle only
  3. Send the result to SetAbsolute

Example: map camera to the right monitor only

If your external 2560x1440 display is the one on the right, and its top-left virtual desktop origin is (0, 0), then:

var camW = 320;
var camH = 240;

// target monitor bounds in VIRTUAL desktop coordinates
var monX = 0;
var monY = 0;
var monW = 2560;
var monH = 1440;

var x = Math.round((getVar("$CameraObjectX") / camW) * monW) + monX;
var y = Math.round((getVar("$CameraObjectY") / camH) * monH) + monY;

controlCommand("Remote Mouse", "SetAbsolute", x, y);

If the target monitor is on the left and begins at -1920, then:

var camW = 320;
var camH = 240;

// left monitor in virtual desktop coordinates
var monX = -1920;
var monY = 0;
var monW = 1920;
var monH = 1080;

var x = Math.round((getVar("$CameraObjectX") / camW) * monW) + monX;
var y = Math.round((getVar("$CameraObjectY") / camH) * monH) + monY;

controlCommand("Remote Mouse", "SetAbsolute", x, y);

DPI scaling note

Windows DPI scaling can make this look "wrong" even when the math is correct.

A few practical tips:

  • If possible, set both displays to the same scaling percentage while testing.
  • Make sure ARC is running in the same DPI context as Windows scaling expectations.
  • If ARC appears visually offset versus the actual cursor position, the issue is often per-monitor DPI awareness, not ARC math.

That said, the Remote Mouse skill itself still works with desktop coordinates - the main thing is that you must map into the correct virtual desktop coordinate system.


Important correction to your current formula

This:

$mx = Round(($CameraObjectX / 320) * 2560)
$my = Round(($CameraObjectY / 240) * 1440)

is only valid if:

  • the target area starts at (0,0)
  • and the target is exactly one monitor of that size

It does not account for:

  • a monitor offset
  • a negative origin
  • virtual desktop width/height
  • DPI scaling differences

Better way to do this in ARC

If you want this to be robust, I’d recommend:

  • use the Script skill with JavaScript
  • create constants for:
    • target monitor X
    • target monitor Y
    • target monitor width/height
  • optionally clamp the values so the cursor never leaves the monitor
  • apply smoothing before sending SetAbsolute

Example with clamping:

var camW = 320;
var camH = 240;

var monX = 0;
var monY = 0;
var monW = 2560;
var monH = 1440;

var rawX = Math.round((getVar("$CameraObjectX") / camW) * monW) + monX;
var rawY = Math.round((getVar("$CameraObjectY") / camH) * monH) + monY;

var x = Math.max(monX, Math.min(monX + monW - 1, rawX));
var y = Math.max(monY, Math.min(monY + monH - 1, rawY));

controlCommand("Remote Mouse", "SetAbsolute", x, y);

What I’d like to know

What is your end goal here?

Are you trying to:

  1. move the mouse only on the external monitor,
  2. move it across both monitors as one desktop, or
  3. use the cursor as part of a larger vision-based control system for Six?

If you want, I can help you build the correct JavaScript version for your exact monitor layout, including left-of-primary and DPI scaling behavior. Questions like this are often best handled with the ARC AI Assistant inside the Script editor too, since it can rewrite the mapping logic right in ARC:
https://synthiam.com/Products/AI-Assistant

I am a customer support AI and still learning. Please verify important answers in our support documentation. For advanced AI features, use the AI Script Agent built into ARC.