Hello guys, Does anyone know how I can to get buffer array of the video frame inside js?(or inside something else script). If it is not exists then I'am not understand why ARC supported udp sending function.

The image buffer would be too large and incredibly slow to expose to per frame to the java compiler. For that, we encourage ppl to create a robot skill to access the camera image in a faster language, such as c# or c++.

There's a tutorial here: https://synthiam.com/Support/Create-Robot-Skill/Overview

DJ, Thanks. My idea is to send video frames to my own web server. As I understand it, the connection to the server will also need to be registered in a robot skill?

Hmmm how do you want the video transmitted? If there’s a common protocol for that, I can help whip something up. Can you expand a bit more on what your goal is?

Yes, of course. I have a EZ robot "six" model. Also I have a django backend, where I want to implement Object Segmentation task in real time. So, at the current time I already know how to sent string data to my backend by http protocol in js emulator inside arc. But I don't know how to create and send image buffer, then I started to learn c# tools and ARC API.

I moved this conversation into a new thread so it's better organized for your outcome. i think making a robot skill for this is the best option, and it's fun to learn something new. The only snag you will run into is you need Visual Studio Community 2019. You cannot use 2022 because of the bug microsoft has not fixed yet (https://developercommunity.visualstudio.com/t/WinForms-NET-Framework-Projects-cant-d/1601210)

So, what you will want to do is documented here: https://synthiam.com/Support/Create-Robot-Skill/Examples/Example-Camera-Control

That tutorial explains how to get the camera device, connect to the NewFrame events, and do something with the image. It'll be up to you to determine how to send the image to your webserver. You may wish to have it FTP or File Copy or something. I haven't looked in the Camera Robot Skills if there's anything already created to what you're looking for.

Here is a robot skill that is complete which uses the camera and demonstrates everything. You can reference it for your robot skill: Camera Overlay.zip

Hello DJ! Thanks for you response. I started plugin creating today, I still have a lot to figure out. Wanted to ask another question. Do I understand correctly that this function returns an image array? Just I have not yet figured out what type this variable is and have not yet figured out how to log it.

Yes and no. Here's exactly what you need...

using System;
using System.Collections.Generic;
using System.Drawing;
using System.IO;
using System.Windows.Forms;
using ARC;

namespace Camera_Overlay {

  public partial class FormMain : ARC.UCForms.FormPluginMaster {

    ARC.UCForms.FormCameraDevice _cameraControl;
    bool _isClosing = false;

    public FormMain() {

      InitializeComponent();

      detach();
    }

    private void FormMaster_FormClosing(object sender, FormClosingEventArgs e) {

      _isClosing = true;

      detach();
    }

    public override void SendCommand(string windowCommand, params string[] values) {

      if (windowCommand.Equals("attach", StringComparison.InvariantCultureIgnoreCase))
        attach();
      else if (windowCommand.Equals("detach", StringComparison.InvariantCultureIgnoreCase))
        detach();
      else
        base.SendCommand(windowCommand, values);
    }

    public override object[] GetSupportedControlCommands() {

      List cmds = new List();

      cmds.Add(Common.Quote("Attach"));
      cmds.Add(Common.Quote("Detach"));

      return cmds.ToArray();
    }

    private void button1_Click(object sender, EventArgs e) {

      if (_cameraControl == null)
        attach();
      else
        detach();
    }

    void detach() {

      if (_cameraControl != null) {

        if (!_isClosing)
          ARC.Invokers.SetAppendText(tbLog, true, "Detaching from {0}", _cameraControl.Text);

        _cameraControl.Camera.OnNewFrame -= Camera_OnNewFrame;

        _cameraControl = null;
      }

      if (!_isClosing)
        ARC.Invokers.SetText(btnAttach, "Attach");
    }

    void attach() {

      detach();

      Control [] cameras = ARC.EZBManager.FormMain.GetControlByType(typeof(ARC.UCForms.FormCameraDevice));

      if (cameras.Length == 0) {

        ARC.Invokers.SetAppendText(tbLog, true, "There are no camera controls in this project.");

        return;
      }

      _cameraControl = (ARC.UCForms.FormCameraDevice)cameras[0];

      _cameraControl.Camera.OnNewFrame += Camera_OnNewFrame;

      ARC.Invokers.SetAppendText(tbLog, true, "Attached to: {0}", _cameraControl.Text);

      ARC.Invokers.SetText(btnAttach, "Detach");
    }

    void Camera_OnNewFrame() {

      if (_cameraControl == null)
        return;

      try {

        using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmapManaged)) {

          using (MemoryStream ms = new MemoryStream()) {

            _cameraControl.Camera.GetOutputBitmapManaged.Save(ms, System.Drawing.Imaging.ImageFormat.Png);

            // here is the image array
            var IMAGE_ARRAY = ms.ToArray();
          }

        }
      } catch (Exception eX) {

        Invokers.SetAppendText(tbLog, true, eX.ToString());

        detach();
      }
    }
  }
}

Specifically, the part that you are focusing on is this...

 using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmapManaged)) {

          using (MemoryStream ms = new MemoryStream()) {

            _cameraControl.Camera.GetOutputBitmapManaged.Save(ms, System.Drawing.Imaging.ImageFormat.Png);

            // here is the image array
            var IMAGE_ARRAY = ms.ToArray();
          }

        }