Behavior Control
This control is developed by:
AvatarDJ Sures
Created
Updated
Help Index

Servo And Camera Server

How To Download and Use This Control

  1. Make sure you have the latest version of EZ-Builder installed.
  2. Select Download This Control button below to download the file.
  3. Double click the downloaded file to open it.
  4. The installer will automatically add this control to EZ-Builder.
Download This Control

This is a servo & camera video server which allows a remote client to move servos and receive video stream from a camera device. This is specifically useful for those creating Unity apps that operate as a client to EZ-Builder, where the camera video stream can be received and servo positions can be sent. The camera must be streaming for the servo positions to transmit.


Example Client App Source Code
Here is an example test app src that connects to localhost (127.0.0.1), moves a servo on port D2 and displays the camera video stream. The sample app is C# .Net source-code and can be downloaded at this link: Test App v2.zip


Rasperry Pi
This plugin will work on a Raspberry Pi EZ-Builder, but will require SUDO privilege. This means you will have to launch EZ-Builder with sudo mono EZ-Builder.exe. This is because the plugin binds to a TCP port and therefore requires root privilege to become a network server program


Test Unity Project
I have included a test Unity project for example reference. The example rotates a cube on the screen using the ARROW keys. The cube projects the texture from the camera stream onto it. The arrow keys will also move the servos connected to port D0 and D1 relative to the rotation of the cube. You can download the project here: Servo Server.zip



Use In Unity
The stream client files in the "Test App" can be included in a Unity project to receive the video and move servos. The test app demonstrates how to move the servos using the methods, and how to display the video on Windows. To display the video in Unity, follow the steps below. The video works by becoming a Texture2D that can be applied to any material.

1) To use this in your Unity App, copy the files from the Test App\ServoServerClient\*.cs into your Unity project.
User-inserted image

2) Here is an example bit of unity code that can be attached to an object in Unity

Code:


using EZ_Robot_Unity_DLL;
using UnityEngine;

namespace SceneRobotView {

  public class RobotView : MonoBehaviour {

    // The stream client
    servoServerClient _streamClient;

    // We want to ensure the scene has been initialized by at least one Update() event occuring before writing to a material.
    // This is because we don't want to write to material that hasn't been initialized yet.
    bool _initialized = false;

    // To be friendly on the garbage collector, we re-use ths same texture object.
    // The object gets initialized in the Start() for this scene.
    Texture2D _texture;

    // The data for the camera will arrive at a different rate than the scene.
    // To ensure there is synchronization between the two updates, we will store the camera video in this array
    // and load it into the texture on every Update()
    volatile byte [] _toDisplay = new byte[]{ };

    void Start() {

      // Initialize the texture with the video format from the servo Server
      _texture = new Texture2D(640, 480, TextureFormat.RGB24, false);

      // The stream client will connect to the servo Server
      // Probably a good idea to use the address as a user-configurable item rather than hard coded;)
      _streamClient = new servoServerClient ();
      _streamClient.OnImageDataReady += _streamClient_OnImageDataReady;
      _streamClient.Start("192.168.1.50", 8282);
    }

    // Every time the camera sends a frame, this event is raised
    private void _streamClient_OnImageDataReady(byte[] imageData) {

      // Exit if the scene has not been initialized yet
      if (!_initialized)
        return;

      // Set the image data to an array which we will use in the Update() on the material
      _toDisplay = imageData;
    }

    void OnDisable() {

      // Stop the streaming because we're closing this scene.
      _streamClient.Stop();
    }

    // Called with every update frame of the scene
    void Update() {

      _initialized = true;

      // If there is data in the camera array...
      if (_toDisplay.Length > 0) {

        // ... set the texture from the camera array data
        _texture.LoadImage(_toDisplay);

        // Set the material to the current component
        var material = GetComponent<Renderer>().material;
        material.mainTexture = _texture;
        material.mainTextureScale = new Vector2(-1, -1);
      }
    }
  }
}

Yes I used that method. I think you are correct something needs to be done to those values to smooth out the motion. For sure needs some tweeking. But a great start.
— Updated
The title says it all. Step right up ladies and gentlemen! Spin the cube and watch wormy's mesmerizing dance!
That seems to be super fluid...are you using the default code, or did you do anything to those servo values already?
Thats default. And i think that the cube it spinning so fast that the servos are forced to update faster. But if i slow down the movement its very jerky. Almost as if the steps are larger than a degree or two.
Ah OK, I was thinking its only my setup that does this...