France
Asked — Edited

Get Matrix From Camera

Hi,

I'm using the camera above the Six Hexapod. I want to create my own plugin in which I would like to process the returned matrix from the video Stream. I followed tutorials and successfuly managed to create my plugin in which I'm attaching the camera and displaying messages :

 using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace Spidy
{
    public partial class MainForm : EZ_Builder.UCForms.FormPluginMaster
    {
        EZ_Builder.UCForms.FormCameraDevice _cameraControl;

        public MainForm()
        {
            InitializeComponent();
            detach();
        }

        private void FormMaster_FormClosing(object sender, FormClosingEventArgs e)
        {

            detach();
        }

        private void MainForm_Load(object sender, EventArgs e)
        {

        }

        /// 
        /// This method is called by ARC when it requests the configuration for your  plugin. 
        /// ARC will request the configuration when the control is created and when the project is saved.
        /// The data set in this configuration will be serialized and saved in the ARC user's project.
        /// Custom data may be stored in the STORAGE dictionary.
        /// 
        public override EZ_Builder.Config.Sub.PluginV1 GetConfiguration()
        {

            return base.GetConfiguration();
        }

        /// 
        /// This method is called by ARC when a project is loaded or when the control is added to a project.
        /// Set your default data in here!
        /// The configuration from the user's project file will be set using this method.
        /// *Note: The   plugin must call Base.SetConfiguration(cf) in your override. See the example!
        /// Also note that this is a good place to ensure all required configuration data exists.
        /// In the case of a newer version of your   plugin where different configuration data may be required outside of the users configuration file, set it here.
        /// 
        public override void SetConfiguration(EZ_Builder.Config.Sub.PluginV1 cf)
        {

            base.SetConfiguration(cf);
        }

        /// 
        /// This method is called by ARC when another control sends a command to this control using the  EZ-Script   ControlCommand() function.
        /// The available   ControlCommand() functions for this control should be returned in the GetSupportedControlCommandsMethodsForSlave() override.
        /// 
        public override void SendCommand(string windowCommand, params string[] values)
        {
            if (windowCommand.Equals("attach", StringComparison.InvariantCultureIgnoreCase))
                attach();
            else if (windowCommand.Equals("detach", StringComparison.InvariantCultureIgnoreCase))
                detach();
            else
                base.SendCommand(windowCommand, values);

            //base.SendCommand(windowCommand, values);
        }

        /// 
        /// This method is called by ARC for the   Cheat Sheet to receive all available   ControlCommand() syntax for this control.
        /// When the   ControlCommand() is called for this function, see the SendCommand() override.
        /// 
        public override object[] GetSupportedControlCommands()
        {

            List cmds = new List();

            cmds.Add("Attach");
            cmds.Add("Detach");

            return cmds.ToArray();

            //return base.GetSupportedControlCommands();

        }

        private void button1_Click(object sender, EventArgs e)
        {
            if (_cameraControl == null)
                attach();
            else
                detach();

            /* int sensor_value = EZ_Builder.EZBManager.GetADC(EZ_B.ADC.ADCPortEnum.ADC7);
             if (sensor_value < 100)
             {
                 EZ_Builder.EZBManager.EZBs[0].Servo.SetServoPosition(EZ_B.Servo.ServoPortEnum.D0,10);
                 label1.Text = sensor_value.ToString();
             } else
             {

                 //label1.Text = "Not moving";
                 label1.Text = EZ_Builder.EZBManager.EZBs[0].Servo.GetServoPosition(EZ_B.Servo.ServoPortEnum.D0).ToString();
             }*/

        }

        void detach()
        {

            if (_cameraControl != null)
            {

                EZ_Builder.Invokers.SetAppendText(textBox1, true, "Detaching from {0}", _cameraControl.Text);
               _cameraControl.Camera.OnNewFrame -= Camera_OnNewFrame;

                _cameraControl = null;
            }

            EZ_Builder.Invokers.SetText(button1, "Attach");
        }

        void attach()
        {
            detach();

            Control[] cameras = EZ_Builder.EZBManager.FormMain.GetControlByType(typeof(EZ_Builder.UCForms.FormCameraDevice));

            if (cameras.Length == 0)
            {

                EZ_Builder.Invokers.SetAppendText(textBox1, true, "There are no camera controls in this project.");
                return;
            }

            _cameraControl = (EZ_Builder.UCForms.FormCameraDevice)cameras[0];

            _cameraControl.Camera.OnNewFrame += Camera_OnNewFrame;

            EZ_Builder.Invokers.SetAppendText(textBox1, true, "Attached to: {0}", _cameraControl.Text);

            EZ_Builder.Invokers.SetText(button1, "Detach");
        }

        void Camera_OnNewFrame()
        {

            if (_cameraControl == null)
                return;

            using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmap.ToManagedImage(false)))
            {

                if (EZ_Builder.Scripting.VariableManager.GetVariable("$CameraIsTracking") == "0")
                {

                    g.DrawString("No object detected", SystemFonts.CaptionFont, Brushes.Red, 0, 0);

                    return;
                }

                int objectX = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectX"));
                int objectY = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectY"));
                int objectCenterX = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectCenterX"));
                int objectCenterY = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectCenterY"));
                int objectWidth = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectWidth"));
                int objectHeight = Convert.ToInt32(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectHeight"));

                // Crosshair
                AForge.Imaging.Drawing.Line(_cameraControl.Camera.GetOutputBitmap, new AForge.IntPoint(objectCenterX - 8, objectCenterY), new AForge.IntPoint(objectCenterX + 8, objectCenterY), Color.FromArgb(200, 240, 100, 200));
                AForge.Imaging.Drawing.Line(_cameraControl.Camera.GetOutputBitmap, new AForge.IntPoint(objectCenterX, objectCenterY - 8), new AForge.IntPoint(objectCenterX, objectCenterY + 8), Color.FromArgb(200, 240, 100, 200));

                // filled rectangle with opacity
                AForge.Imaging.Drawing.FillRectangle(_cameraControl.Camera.GetOutputBitmap, new Rectangle(objectX, objectY, objectWidth, objectHeight), Color.FromArgb(100, 50, 50, 250));


                if (EZ_Builder.Scripting.VariableManager.GetVariable("$CameraTrackingType") == "\"Object\"")
                    g.DrawString(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectName"), SystemFonts.CaptionFont, Brushes.White, objectX, objectY + objectHeight + 5);
            }
        }
    }

}

My question is, how to get the initial video Stream like the rgb matrix and process it with librairies such as openCV ?

Thanks :)


ARC Pro

Upgrade to ARC Pro

Become a Synthiam ARC Pro subscriber to unleash the power of easy and powerful robot programming

PRO
USA
#1  

you can capture the bitmap within Camera_OnNewFrame:


void Camera_OnNewFrame()
{

    if (_cameraControl == null)
        return;

    using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmap.ToManagedImage(false)))
    {

        var bitmap = _cameraControl.Camera.GetOutputBitmap.ToManagedImage(false);

        //other stuff...
    }
}

France
#2  

Hi,

If I understand correctly, the bitmap is a matrix of every values. If I want to apply a change on the bitmap, can I "send" the new bitmap into the frame ? So I can see the results in live.

(Sorry if it's seems obvious to you, I'm kind of new to C# and ezRobot technologies)

Thanks.

PRO
USA
#3  

http://www.aforgenet.com/framework/docs/html/86a3d7d8-0f95-e651-7dec-c2bfb000ad06.htm


var bitmap = _cameraControl.Camera.GetOutputBitmap.ToManagedImage(false);

passing false you have the original image content (bytes) wrapped in a Bitmap https://docs.microsoft.com/en-us/dotnet/api/system.drawing.bitmap?view=netframework-4.6.1 structure.


using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmap.ToManagedImage(false)))
{
...
}

The graphics g object allows you to write directly to the original (false parameter) image.

So every modification will be visible everywhere e.g. camera object and or other plugins, because all of them will receive the original image through .net event delegate.

If you cloned the image (parameter true) you will need to have a picture control to show the result.

PRO
Synthiam
#4  

There’s a section in the plugin tutorial with absolutely the exact example you’re inquiring. Look here: https://synthiam.com/Tutorials/UserTutorials/146/17

Absolutely everything you asked about is demonstrated with explanation and code examples on that page:) almost like we read your mind!

France
#5  

Hi,

First, I'm really sorry for such a late answer. I couldn't work on my project sooner.

Thanks for the help example, however I can't find the link you gave me. Seems like you've changed a lot on the website and I can't reach it.

Thanks

France
#6  

About your answer ptp,  I've tried :

var bitmap = _cameraControl.Camera.GetOutputBitmap.ToManagedImage(false);

I putted just after the attach(); Here's the error :

System.Exception : 'Attempting to read GetOutputBitmap while locked. This means you are attempting to read the output bitmap outside of the OnNewFrame event!'

I guess I'm not doing it correctly however your link doesn't work either :C

Thanks

PRO
Synthiam
#7  

Click the link i provided earlier - or - click this link: https://synthiam.com/Tutorial/How-To-Make-An-EZ-Builder-Plugin-16328/17

#8  

Links provided are not clickable to me. I have to highlight, cut and paste into address bar to follow. Maybe it's because I'm using Chrome?