Example: Camera Control
Your plugin can access the camera by attaching itself to an existing camera control. Attaching to an existing camera control is a friendly way of sharing the camera video stream with other plugins and maintaining the camera control's features.
This is demonstrating one method of finding a control within the workspace. There are a number of methods that make this easy. Take a look at the Example: Finding Other Behavior Control section of this tutorial for a more detailed explanation.
In this example, we will search for an existing camera control and attach to frame event.
Step 1 - Declare a global camera control variable
Your plugin will need to search the project to find a camera control. When it finds the camera control, it will need to keep a reference to it for future actions. Specifically, your plugin will need to keep the camera control reference so it can detach from the video frame event during dispose or close.
namespace Camera_Example {
public partial class FormMain : EZ_Builder.UCForms.FormPluginMaster {
// Global variable within your plugin class
ARC.UCForms.FormCameraDevice _cameraControl;
public FormMain() {
InitializeComponent();
}
}
}
Step 2 - Create an attach() method with Frame Event
This method will search the project for an existing camera control, add a reference to it and attach a method to the frame event.
void attach() {
// detach in case there is already an attachment to an existing camera control
detach();
// get all camera controls in the project
Control [] cameras = ARC.EZBManager.FormMain.GetControlByType(typeof(ARC.UCForms.FormCameraDevice));
// if there are no camera controls, inform the user
if (cameras.Length == 0) {
MessageBox.Show("There are no camera controls in this project.");
return;
}
// get a reference to the first camera control we find, in the case there are many
_cameraControl = (ARC.UCForms.FormCameraDevice)cameras[0];
// attach to the New Frame event
_cameraControl.Camera.OnNewFrame += Camera_OnNewFrame;
ARC.EZBManager.Log("Attached to: {0}", _cameraControl.Text);
}
// We will add code to this method later in this example
void Camera_OnNewFrame() {
}
Step 3 - Create a detach() method to detach from the camera control
Now that we have created an attach() method, there will need to be a method to detach as well. The detach is necessary specifically for your plugin's Closing event. Be sure to add the detach to the Closing event as in this example.
private void FormMain_FormClosing(object sender, FormClosingEventArgs e) {
// detach the event handler from this plugin when it closes
detach();
}
void detach() {
// only perform detach if there is a reference to a camera control
if (_cameraControl != null) {
ARC.EZBManager.Log("Detaching from {0}", _cameraControl.Text);
// Detach the from the camera event handler
_cameraControl.Camera.OnNewFrame -= Camera_OnNewFrame;
// Set the camera control reference to null
_cameraControl = null;
}
}
Step 4 - Create a button to Attach()
Your plugin will need some way to let the user attach to the camera control. You could do this when the plugin is initiated, but you may wish to give the user the ability to attach/detach on their own. Just in case they do not wish for it to be active right away. Create a button on your plugin which will have this code in the OnClick event. This code will attach() if there is no reference to a camera control, and detach if there is. The code will work as a toggle between the two states.
private void btnAttach_Click(object sender, EventArgs e) {
if (_cameraControl == null)
attach();
else
detach();
}
Step 5 - Create ControlCommand() attach/detach
Be friendly to your users by giving them the ability to attach and detach your plugin via EZ-Script ControlCommand() syntax. These two override methods will do just that...
// Override is called when ControlCommand() is sent to this control
// If an applicable command is passed, execute it. Otherwise execute the base which throws an exception to the user
public override void SendCommand(string windowCommand, params string[] values) {
if (windowCommand.Equals("attach", StringComparison.InvariantCultureIgnoreCase))
attach();
else if (windowCommand.Equals("detach", StringComparison.InvariantCultureIgnoreCase))
detach();
else
base.SendCommand(windowCommand, values);
}
// Return a list of available ControlCommands() to the ARC UI
// This list is presented to the user when they are editing EZ-Script and viewing the Cheat Sheet
public override object[] GetSupportedControlCommands() {
List cmds = new List();
cmds.Add("Attach");
cmds.Add("Detach");
return cmds.ToArray();
}
Step 6 - Do something in the video frame event
Now that you have successfully created attach and detach methods to the camera control, let's do something with the video frame event. The video frame event was created earlier in this project, and was empty. Now let's simply add some code to get the image and draw on it.
Add a reference to the aforge.dll in the ARC program folder. This is the same folder which you added the ARC.exe and EZ_B.dll during the plugin setup. The aforge.dll library is a fantastic open-source project with direct memory access to image buffer for manipulation, which is faster than using GDI (System.Drawing).
In this example frame event code, we will draw a solid square around the object which was detected during tracking. For testing, you will need to enable a tracking type.
void Camera_OnNewFrame() {
// Exit this method if there is no reference to a camera control
// This may only happen during a dipose/detach if the frame is already executing
if (_cameraControl == null)
return;
// If no object is detected, display a message to the user in the camera image
if (ARC.Scripting.VariableManager.GetVariable("$CameraIsTracking") == "0") {
using (Graphics g = Graphics.FromImage(_cameraControl.Camera.GetOutputBitmap.ToManagedImage(false)))
g.DrawString("No object detected", SystemFonts.CaptionFont, Brushes.Red, 0, 0);
return;
}
// If we got this far, an object must be detected
// Get the camera control variables for the detected object location and size
int objectX = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectX"));
int objectY = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectY"));
int objectCenterX = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectCenterX"));
int objectCenterY = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectCenterY"));
int objectWidth = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectWidth"));
int objectHeight = Convert.ToInt32(ARC.Scripting.VariableManager.GetVariable("$CameraObjectHeight"));
// Draw a Crosshair using the aforge library fast draw functions
AForge.Imaging.Drawing.Line(_cameraControl.Camera.GetOutputBitmap, new AForge.IntPoint(objectCenterX - 8, objectCenterY), new AForge.IntPoint(objectCenterX + 8, objectCenterY), Color.FromArgb(200, 240, 100, 200));
AForge.Imaging.Drawing.Line(_cameraControl.Camera.GetOutputBitmap, new AForge.IntPoint(objectCenterX, objectCenterY - 8), new AForge.IntPoint(objectCenterX, objectCenterY + 8), Color.FromArgb(200, 240, 100, 200));
// draw a filled rectangle with opacity
AForge.Imaging.Drawing.FillRectangle(_cameraControl.Camera.GetOutputBitmap, new Rectangle(objectX, objectY, objectWidth, objectHeight), Color.FromArgb(100, 50, 50, 250));
}
You're Done!
You have successfully created methods required to attach/detach from existing camera controls and perform drawing on the frame image!
Is this out of date? There doesn't seem to be a GetConfiguration function within EZ_Builder.Config.Sub.PluginV1
Look at the tutorial step titled [color=#ce3991][size=3][font=OpenSans, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"]Code: Saving/Loading Configuration
[color=#ce3991][size=3][font=OpenSans, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"]The get and set configuration methods are overrides of the form. There’s a great video on the first step of this tutorial that demonstrates the step by step of building a plugin. I recommend watching that because it helps fill in any steps that were missed. [/font][/size][/color]
[color=#ce3991][size=3][font=OpenSans, -apple-system, BlinkMacSystemFont, Segoe UI, Roboto, Helvetica Neue, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol, Noto Color Emoji]When you’ve done it once, it makes sense and voila, you can rinse and repeat :)[/font][/size][/color][/font][/size][/color]
Excellent, thanks - I will do. I really must learn not to just jump ahead in the process :)
Hey no problem - I do it all the time, and end up frustrated because I dont know what it was that I missed. Excitement gets the best of me
Trying to follow the tutorials but can't find where the plugin page has gone. How do I add a new plugin to the ez-robot / Synthiam site to get the XML?
Never mind. Just found the "Create skill control" link :)
I am trying to follow the instructions for adding my own plugin but I cannot seem to find the place to register the plugin based on the instructions.
Any help is appreciated.
Thanks
The new button to create a plugin skill control is less than an inch below the button you pressed to create this question. :)
Thanks for the quick response.
This what happens when you are working on robotics when its way past your bed time.
You miss the obvious
No problem - i get it :D
Please follow the tutorial. It’s impossible to know why you’re plug-in isn’t showing up without asking you if you followed each step of the tutorial :). Reviewing your screenshots, it doesn’t appear as if any of the tutorial steps have been followed.
Hi i fixed it. thanks
Hello, I am trying the tutorial to get the robot to speak. I am using Visual studio. Currently, the sound is output from the pc instead of the robot. Is there a code I can attach so that the sound comes from the robot speakers instead of the pc?
Look in this tutorial for the step labeled "output audio from ezb". It’s lower down in the list. There’s instruction examples for either playing audio (ie mp3) or text to speech.
Error: the referenced component" EZ_builder,EZ_B" could not be found, DJ Sure i hope you can help me !
The error cannot read the COM file, I downloaded it and when I follow the instructions, I get an error, while other files read normally. .
sorry for me but i tried many different ways but still show the error,I couldn't find EZ_B.dll file even though I downloaded it
None of the required references are in your list. Please follow the tutorial. It explains exactly how to click the browse button and navigate to the folder and select the files.
Sorry, but the reason I can't reference is because there is no file in the EZ_B folder and there is an error : this folder is empty , I am trying to solve it. I would like to thank DJ sure for answering my superfluous questions and I'm sorry for bothering you
I need to playback 5 Serial Bus servos in sequence.
What protocol is it? A "serial bus" is a generic term for anything using a UART that's chained together sharing the same RX line. Also, why did you add a photo with the question text added in your response?
Are you planning on making a skill control to do this? You wrote the question in the skill control thread in a comment - I'd like to make sure your question is in the right place to help you out.
To begin, I would recommend starting with servo Script control so you can make the serial bus protocol work - then consider making a skill control only if you're planning on distributing the effort to others: https://synthiam.com/Products/Controls/Scripting/Servo-Script-19068
He seems rather demanding as well. I guess he needs the benefit of the doubt as English may not be his native language....
I have installed all the software dependencies and still ARC does not detect that I have Visual Studio installed. OS is Windows 10, .NET 4.8 or newer, Visual Studio Community 2019.
When the popup says it doesn’t detect visual studio, you can still skip and continue. I wonder why it’s not detecting it? We had a hard time trying to find a proper way of detecting - even Microsoft’s suggestion didn’t actually work eye roll
ill look into it a bit further and see if we can find a better way of detecting
@DJ: It's easy to find the Visual Studio 2017 and up: Microsoft: https://github.com/Microsoft/vswhere/wiki/Find-MSBuild
Some quick c# code to use with .NET: https://github.com/ppedro74/Utils/blob/master/FindVisualStudio/Program.cs
We went this route and it didn’t work on my computer - because I had a preview of visual studio installed which isn’t in that directory path. Microsoft had numerous suggestions of detecting visual studio. The one which worked for our various installations was a registry check.
apparently with the above individual, the registry didn’t work either. I’ll have to combine a few methods.
everything looks simple from the outside - until you have a hundred thousand+ installations of your software. That’s when you run into things like this lol
@DJ: I agree sometimes the things go out of script easily.
I avoid going through the registry keys, unless is recommended by the vendor. A lot of people blame the changes (keys, entries are renamed etc), but, that is normal if I own my product is my business and is part of the software evolution. Some products you can break the support contract agreement if you query directly the database, or if you look elsewhere outside of the public API.
Is true story some years ago a "rogue" developer on my team released a Sharepoint integration using a mix of APIs and database queries, everything worked well with multiple clients, until one day the Microsoft Black suits visit one of the customers to follow up on an unrelated support ticket, and they basically used "unsupported" card and left the client hanging, and we had problems too, unfortunately the Rogue developer went to another galaxy ... and the team suffered the consequences.
That does not mean I'm not tempted to do it... :)
I used the vswhere before and I would say is almost 99% bulletproof, is used with Xamarin, NVIDIA, Intel setups. If you add vswhere.exe to your project (nuget package) you cover scenarios where the tool is not present or have been deleted (broken uninstalls).
The other fallback could be ask the user the visual studio version.
The other reason to avoid registry is due to Visual Studio uses a private exclusive registry keys to store more stuff: http://www.visualstudioextensibility.com/2017/07/15/about-the-new-privateregistry-bin-file-of-visual-studio-2017/
So the things are getting more complex.
The above post is only part of the "Full solution" for example I have one setup with visual studio 2017 c# installed and Visual studio 2019 with Python and C++, vswhere will return 2019 version, but my c# is done with VS2017.
If you are generating customized vs version project files, maybe a fallback (ask the VS version) will cover more bases.
Yes - Microsoft has a few pages on how to identify visual studio and we tried them all during testing - the one we went with was with registry. I'm going to combine the two as using only one method apparently doesn't work for all cases.