Sharp IR Radar icon Sharp IR Radar Radar-style Sharp IR scanner for EZB ADC: sweeps sensor, displays distance dots, logs ADC voltage, and can steer/avoid obstacles via Movement Panel Try it →
Asked

Remote Mouse DPI Scaling On Ohmni

I’m running ARC directly on an Ohmnilabs Ohmni telepresence robot (Windows 10 Pro on the UpBoard, ARC build 2024.05.XX). I use the Ohmnilabs Standalone Movement Panel to drive and added the Remote Mouse skill to automate a couple of on-screen clicks (launching a browser-based kiosk and hitting a "Join" button). I’m doing SetAbsolute moves via a Script/Python block so the operator doesn’t need to fiddle with the touch UI while telepresent. The issue: pointer coordinates drift when the display scaling or resolution changes. With Windows display scaling set to 125% on the Ohmni, SetAbsolute lands ~60-100 px off from where it should; if Windows auto-switches to 1024x768 after an RDP disconnect, everything is way off again. I tried: forcing Windows scaling to 100%, locking resolution to 1920x1080, enabling the ARC compatibility setting "Override high DPI scaling behavior" (tested Application/System/Enhanced), and always calling ControlCommand("Remote Mouse", WindowMaximize) on ARC before moves. I also attempted to compute relative coordinates in Python by reading the screen size:

import ctypes
w = ctypes.windll.user32.GetSystemMetrics(0)
h = ctypes.windll.user32.GetSystemMetrics(1)
# then map 92%/95% of w/h to SetAbsolute

But SetAbsolute still seems to use unscaled coordinates when Windows is at 125% DPI, so my math is still off unless scaling is 100%. Is there a DPI-aware way to use Remote Mouse so SetAbsolute targets the effective (scaled) desktop, or a percent-based command I’m missing, or does ARC expose the current scaled screen width/height I can read reliably and recompute on resolution changes?



Related Hardware (view all EZB hardware)
Ohmni telepresence robot with 13MP Supercam and low-latency streaming; run Synthiam ARC on Windows 10, use Ohmnilabs Standalone Skill and Exosphere.
Wi-Fi
Servos 1
Camera
Audio
UART
I2C
ADC
Digital

Related Robot Skill (view all robot skills)
Remote Mouse by Synthiam
Scriptable Remote Mouse for ARC: control PC cursor and ARC window via ControlCommand, using voice, camera or I/O inputs; shows cursor coordinates.

ARC Pro

Upgrade to ARC Pro

Synthiam ARC Pro is a new tool that will help unleash your creativity with programming robots in just seconds!

Author Avatar
PRO
Canada
Twitter YouTube GitHub
#1  

Yes - what you’re seeing is consistent with Windows DPI scaling vs. physical desktop coordinates on the Ohmni’s Windows 10 UpBoard.

Short answer

Remote Mouse uses screen coordinates, not percent-based coordinates, and ARC does not currently expose a separate "scaled desktop" coordinate system for that skill. So if Windows changes scaling or resolution, SetAbsolute will be affected unless you compensate for it yourself.

What’s happening

When Windows is at 125% scaling, the desktop coordinate space reported by some APIs can differ from the actual clickable surface ARC is interacting with. That’s why:

  • SetAbsolute lands correctly at 100%
  • it drifts at 125%
  • it shifts again when the resolution drops to 1024x768

This is a Windows display scaling behavior, not really an ARC mouse skill bug.

Best approach

Option 1: Keep the robot UI at a fixed display mode

For telepresence kiosk-style operation, the most reliable setup is:

  • Resolution fixed to 1920x1080
  • Scaling fixed to 100%
  • avoid auto-switching to 1024x768 after RDP disconnect

If the machine is dropping to 1024x768 after disconnect, that usually means the display session is being renegotiated by Windows or the remote display adapter. In that case, the most stable workflow is to avoid RDP for the live kiosk session if possible.

Option 2: Use image-based targeting instead of fixed coordinates

If the "Join" button can move depending on browser UI, zoom level, or scaling, coordinate clicking will always be fragile.

A better ARC workflow is:

  • use the Camera Device skill or screen-capture-based logic if appropriate
  • detect the button visually
  • click based on the detected location

That is much more robust than hard-coded pixel positions.

Option 3: Read the current desktop size and map coordinates yourself

You can build a workaround in Python/JavaScript by reading the current screen size and converting percentages to absolute coordinates. However, this only helps if the values you read reflect the same desktop coordinate space that Remote Mouse expects.

If you want to test this, I’d recommend using a browser windowed state first and compare:

import ctypes

user32 = ctypes.windll.user32
w = user32.GetSystemMetrics(0)
h = user32.GetSystemMetrics(1)

print("Screen size:", w, h)

x = int(w * 0.92)
y = int(h * 0.95)

ControlCommand("Remote Mouse", "SetAbsolute", x, y)

But if Windows scaling is affecting what GetSystemMetrics() reports vs. what Remote Mouse is using internally, the mismatch can still happen.

Things to try in Windows

On the Ohmni PC, check these:

  1. Display Settings

    • Resolution: 1920x1080
    • Scale: 100%
  2. Advanced scaling settings

    • Enable Let Windows try to fix apps so they’re not blurry
  3. Compatibility settings for ARC

    • You already tested the DPI override modes, which is good
  4. Avoid session drops

    • If RDP disconnect changes the display mode, that’s likely the source of the 1024x768 fallback

Recommendation for your use case

For a telepresence kiosk, I’d suggest:

  • keep ARC and the browser in a single known resolution
  • avoid RDP changing the session resolution
  • use windowed browser positioning if possible
  • if you need to automate UI clicks reliably, consider a robot skill you create to handle browser/UI state transitions instead of pure coordinate clicking

If a built-in feature doesn’t cover your exact workflow, Synthiam’s path for that is to create a robot skill: https://synthiam.com/Support/Create-Robot-Skill/Overview

One question so I can narrow this down

Are you launching the browser in:

  1. full screen kiosk mode, or
  2. a normal window maximized inside ARC/Windows?

That detail matters because the best coordinate strategy changes depending on whether the browser is occupying the whole desktop or just a window.


I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.