Welcome to Synthiam!

The easiest way to program the most powerful robots. Use technologies by leading industry experts. ARC is a free-to-use robot programming software that makes servo automation, computer vision, autonomous navigation, and artificial intelligence easy.

Get Started
Belgium
Asked — Edited

Camera Facedetection Threadable?

Hello all,

I have a question concerning the camera usage in visual studio 10, C#.

Is it possible to use this with a thread?

The problem is that my program is starting to run very very slowly when the camera is activated for facial detections

This is my code

Code:



            if (!ezb.Camera.IsActive)
			{
				return;
			}

			ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

			if (!objectLocation.isFound)
			{
				turnAway = true;
				timer2.Start();
				timer4.Stop();
				timer4.Dispose();
			}

			if (objectLocation.isFound)
			{
				interactie();
				textBox1.AppendText(Environment.NewLine);
				textBox1.AppendText("Face found");
				textBox1.AppendText(Environment.NewLine);
			}



When the program sees a face it goes to the method "interact();" or in English "interaction();"

But then the program seems to halt and running slowly

Thanks in advance for the help.

Cheers,

Laurens


ARC Pro

Upgrade to ARC Pro

Synthiam ARC Pro is a cool new tool that will help unleash your creativity with programming robots in just seconds!

PRO
Synthiam
#1  
That is how you would do it, yes. The speed of your program depends on the entire program, not just that sniplet. It matters how you are executing that bit of code. Is that being executed in a timer? Is it in a loop? How many times a second are you calling it? Is any throttling mechanism in place to prevent it from eating up all available cpu cycles within the loop?

If you paste more code, i can help you
#2  
Hi DJ, Thx for helping me.

The code for the cam is being executed by a timer.
The cam must be active whenever it comes across an object.
Then it will scan if it is a human. When I run it the program "works" but when the cam is active the cam stops working, but the program continuous to work.

But the cam won't activate anymore

This is my whole code:

Code:


using System;
using System.Drawing;
using System.Windows.Forms;

using EZ_B;
using FaceDetect;

using Microsoft.DirectX;
using Microsoft.DirectX.DirectInput;

using System.Threading;

namespace WindowsFormsApplication1
{
	public partial class Form1 : Form
	{
		/// 
		/// Initialisatie
		/// 
        /// 
        private EZB ezb = new EZB();

        // Bools
		private bool isProgamRunning = false;
		private bool direction = true;
        private bool turnAway = false;

		private int distanceObject = 0;
		private int turnDirection = 0;

        private string videoCaptureDeviceName = "";

		private Random randomTurnDirection = new Random();

        // Timers
        private System.Windows.Forms.Timer timer1 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer2 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer3 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer4 = new System.Windows.Forms.Timer(); //CAM

		//Woordenlijst
		private string[] phraseList =
		{
			"go forward",		// 0
			"go right",			// 1
			"go reverse",	    // 2
            "go left",			// 3
            "stop",				// 4
            "look up",			// 5
            "look down",		// 6
            "left arm down",	// 7
            "left arm up",	    // 8
            "right arm up",     // 9
            "right arm down",   // 10
			"teddy speak",		// 11
            "sing",             // 12
            "resume",           // 13
			"goodnight",        // 14
        }; 

		public Form1()
		{
			InitializeComponent();

			btn2.Enabled = false;
			comboBoxCam.Enabled = false;

			textBox1.Text = Resource1.message1;

			comboBoxPorts.Items.Clear();
			comboBoxPorts.Items.AddRange(ezb.GetAvailableCommunicationPorts());
			comboBoxCam.Items.Clear();
			
            foreach (string name in ezb.Camera.GetVideoCaptureDeviceList())
			{
				comboBoxCam.Items.Add(name);
			}

            label2.Text = "Afstand = " + (distanceObject * 2.54).ToString() + " cm";
		}

		private void Form1_Load(object sender, EventArgs e)
		{
			ezb.Movement.MovementType = Movement.MovementTypeEnum.ModifiedServo;
			ezb.Movement.ServoWheelLeftModifiedPort = Servo.ServoPortEnum.D13;
			ezb.Movement.ServoWheelRightModifiedPort = Servo.ServoPortEnum.D14;

			turnDirection = randomTurnDirection.Next(0, 2);

			timer2.Interval = 2000;
			timer2.Tick += new EventHandler(timer2_Tick);
			timer3.Interval = 1000;
			timer3.Tick += new EventHandler(timer3_Tick);
			timer4.Tick += new EventHandler(timer4_Tick);
		}

		protected override void OnClosed(EventArgs e)
		{
			ezb.Servo.ReleaseAllServos();
			timer1.Stop();
			timer2.Stop();
			ezb.Disconnect();
			ezb.Dispose();
		}

		private void comboBoxCam_SelectedIndexChanged(object sender, EventArgs e)
		{
			videoCaptureDeviceName = comboBoxCam.SelectedItem.ToString();
			ezb.Camera.StartCamera(videoCaptureDeviceName, panel1);

			timer4.Interval = 100;
            timer4.Enabled = true;
		}

		/// 
		/// Connecteren en Disconnecteren met EZB
		/// 
		private void btn1_Click(object sender, EventArgs e)
		{
            try
            {
                if (ezb.IsConnected)
                {
                    ezb.Disconnect();
                }

                else
                {
                    ezb.Connect(comboBoxPorts.SelectedItem.ToString());
                    comboBoxCam.Enabled = false;
                }

                if (ezb.IsConnected)
                {
                    btn1.Text = "Disconnect";
                    btn2.Enabled = true;
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Connected !");
                    comboBoxCam.Enabled = true;
                }

                else
                {
                    btn1.Text = "Connect";
                    btn2.Enabled = false;
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Disconnected !");
                    comboBoxCam.Enabled = false;
                }
            }
            
            catch(Exception ex)
            {
                MessageBox.Show("Error = " + ex);
            }
		}

		/// 
		/// De knop die instaat voor het starten en stoppen van het programma.
		/// Via een boolean is er controlle op de status van het programma.
		/// 
		private void btn2_Click(object sender, EventArgs e)
		{
			if (!isProgamRunning)
			{
				isProgamRunning = true;
				btn2.Text = "Stop";

                //Run_Program();
                Run_Program_Delegate rpd = new Run_Program_Delegate(Run_Program);
                Run_Program();
                
			}

			else
			{
				isProgamRunning = false;
				btn2.Text = "Run";

			    ezb.Servo.ReleaseAllServos();
			    timer1.Stop();
                timer1.Dispose();
			    timer2.Stop();
                timer2.Dispose();
			    timer3.Stop();
                timer3.Dispose();
			    timer4.Stop();
                timer4.Dispose();
			    ezb.Dispose();
			}
		}

		private void Run_Program()
		{
            timer1.Start();
			timer1.Tick += new EventHandler(timer1_Tick);
		}

		/// 
		/// TIMER1 = PING SENSOR
		/// 
		/// 
		/// 
		private void timer1_Tick(object sender , EventArgs e)
		{
            distanceObject = ezb.HC_SR04.GetDistance(Digital.DigitalPortEnum.D0, Digital.DigitalPortEnum.D0); //afstand in Inches
            progressBar1.Value = (int)(distanceObject * 2.54);

            label2.Text = ("Afstand = " + (distanceObject * 2.54).ToString() + " cm");

            if (distanceObject 
		/// TIMER2 = STOP
		/// 
		/// 
		/// 
		private void timer2_Tick(object sender, EventArgs e)
		{
			timer3.Start();
			timer2.Stop();
            timer2.Dispose();
			timer1.Start();
		}

		/// 
		/// TIMER3 = DRAAIEN
		/// 
		/// 
		/// 
		private void timer3_Tick(object sender, EventArgs e)
		{
			if (turnDirection == 0)
			{
				ezb.Movement.GoStop();
				ezb.Movement.GoLeft();
			}

			else
			{
				ezb.Movement.GoStop();
				ezb.Movement.GoRight();
			}
			timer3.Stop();
            timer3.Dispose();
		}

		/// 
		/// TIMER4 = CAMERA
		/// 
		/// 
		/// 
        private void timer4_Tick(object sender, EventArgs e)
        {
            if (!ezb.Camera.IsActive)
            {
                isProgamRunning = false;
                btn2.Text = "Run";

                ezb.Servo.ReleaseAllServos();
                timer1.Stop();
                timer1.Dispose();
                timer2.Stop();
                timer2.Dispose();
                timer3.Stop();
                timer3.Dispose();
                timer4.Stop();
                timer4.Dispose();
                ezb.Dispose();

                MessageBox.Show("Geen camera geconnecteerd.nProgramma wordt gestopt!");
            }

            else
            {
                ezb.Camera.UpdateImage();
                ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

                if (objectLocation.isFound == false)
                {
                    turnAway = true;

                    timer2.Start();
                    timer4.Stop();
                    timer4.Dispose();
                }

                else
                {
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Face detected!");

                    interactie();
                }
            }
		}

		private void interactie()
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);
            ezb.SpeechSynth.StartListening();
			ezb.SpeechSynth.OnPhraseRecognized += new SpeechSynth.OnPhraseRecognizedEvent(SpeechSynth_OnPhraseRecognized);
		}

		private void SpeechSynth_OnPhraseRecognized(string text, float confidence)
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);

			try
			{
                if (confidence < 0.85)
                {
                    return;
                }

                else
                {
                    if (text == phraseList[0])
                    {
                        ezb.Movement.GoForward();
                    }

                    else if (text == phraseList[1])
                    {
                        ezb.Movement.GoRight();
                    }

                    else if (text == phraseList[2])
                    {
                        ezb.Movement.GoReverse();
                    }

                    else if (text == phraseList[3])
                    {
                        ezb.Movement.GoLeft();
                    }

                    else if (text == phraseList[4])
                    {
                        ezb.Movement.GoStop();
                    }

                    else if (text == phraseList[5])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 50);
                    }

                    else if (text == phraseList[6])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 25);
                    }

                    else if (text == phraseList[7])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 50);
                    }

                    else if (text == phraseList[8])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 25);
                    }

                    else if (text == phraseList[9])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 50);
                    }

                    else if (text == phraseList[10])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 25);
                    }

                    else if (text == phraseList[11])
                    {
                        ezb.SpeechSynth.Say("Hello my name is teddy Asimov, what is your name?"); 
                    }

                    else if (text == phraseList[12])
                    {
                        ezb.Tone.RandomTone(Digital.DigitalPortEnum.D8, 20, 10);
                    }

                    else if (text == phraseList[13])
                    {
                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer2.Stop();
                        timer3.Stop();
                        timer4.Stop();
                        ezb.Dispose();

                        Run_Program();
                    }

                    else if (text == phraseList[14])
                    {
                        int i;

                        for (i=0 ; i<=1 ; i++)
                        {
                            ezb.SpeechSynth.Say("Just what do you think you're doing Dave?");
                            i++;
                        }

                        btn2.Text = "Run";

                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer2.Stop();
                        timer3.Stop();
                        timer4.Stop();
                        ezb.Dispose();

                        isProgamRunning = false;
                    }
                }
			}

			catch(Exception ex)
			{
				MessageBox.Show("Error speech = " + ex);
			}
		}
	}

    delegate void Run_Program_Delegate();
}
PRO
Synthiam
#3  
There are a lot of thread timers. timers that are unused. primarily because they all share the same thread anyway. Have you looked at Omnibot's code example? I think that is more of what you are trying to do. You only need one timer to do everything.
#4  
Allright, thx.

We will look into that.

A small side-question: Is it possible to use more then one class?
Like the form only as the representation level and other classes for face detection, voice interaction, movement?
PRO
Synthiam
#5  
What do you mean use more than one class?
#6  
Like you have your Form1.cs and Program.cs classes.
Now we only need to improve the performance of our code

The program now is using 1 timer and running smoother then before!
Thx for the reference!
#7  
The code is working much better now, but we still have a small problem with the voice recognition.

When the robot ping sensor sees an object (<= 4 inches) it stops and checks with the camera is the object is a human.
If the object is not a human, the robot turn left or right and goes on.

This works, but when the robot detects a human face. The program "jumps" to the interaction method for voice detection/synthesis.

The robot does this, but is stuck in a loop and repeats the same command over & over again
(i.e. you say "sing" the robot sings, but when the robot says your face again, it start to sing all over again)
#8  
To make it easier to follow the program here is my new code:

Code:


        private void timer1_Tick_1(object sender, EventArgs e)
        {
            distanceObject = ezb.HC_SR04.GetDistance(Digital.DigitalPortEnum.D0, Digital.DigitalPortEnum.D0); //afstand in Inches
            progressBar1.Value = (int)(distanceObject * 2.54);

            label2.Text = ("Afstand = " + (distanceObject * 2.54).ToString() + " cm");

            if (ezb.Camera.IsActive)
            {
                ezb.Camera.UpdateImage();
            }

            // <= 4 inches = 10,16cm
            if (distanceObject <= 4)
            {
                ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

                if (objectLocation.isFound == true)
                {
                    if (interactionDone == false)
                    {
                        ezb.Movement.GoStop();

                        textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Face detected!");

                        interaction();
                        System.Threading.Thread.Sleep(50);
                    }

                    else
                    {
                        interactionDone = false;
                    }
                }

                else
                {
                    turning();
                    System.Threading.Thread.Sleep(50);
                }

                ezb.Movement.GoStop();
                System.Threading.Thread.Sleep(50);
            }

            else
            {
                ezb.Movement.GoForward();
                System.Threading.Thread.Sleep(50);
            }
        }

        private void turning()
        {
            if (turnDirection == 0)
            {
                ezb.Movement.GoLeft();
            }

            else
            {
                ezb.Movement.GoRight();
            }
        }

		private void interaction()
		{
            textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Into interaction mode");

            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);
            ezb.SpeechSynth.StartListening();
			ezb.SpeechSynth.OnPhraseRecognized += new SpeechSynth.OnPhraseRecognizedEvent(SpeechSynth_OnPhraseRecognized);
		}

		private void SpeechSynth_OnPhraseRecognized(string text, float confidence)
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);

			try
			{
                if (confidence < 0.85)
                {
                    return;
                }

                else
                {
                    interactionDone = true;

                    // Go forward
                    if (text == phraseList[0])
                    {
                        ezb.Movement.GoForward();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go right
                    else if (text == phraseList[1])
                    {
                        ezb.Movement.GoRight();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go revese
                    else if (text == phraseList[2])
                    {
                        ezb.Movement.GoReverse();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go left
                    else if (text == phraseList[3])
                    {
                        ezb.Movement.GoLeft();
                        System.Threading.Thread.Sleep(50);
                    }

                    // STOP
                    else if (text == phraseList[4])
                    {
                        ezb.Movement.GoStop();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Look up
                    else if (text == phraseList[5])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Look down
                    else if (text == phraseList[6])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Left arm up
                    else if (text == phraseList[7])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Left arm down
                    else if (text == phraseList[8])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Right arm up
                    else if (text == phraseList[9])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Right arm down
                    else if (text == phraseList[10])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Speak
                    else if (text == phraseList[11])
                    {
                        ezb.SpeechSynth.Say("Hello my name is teddy Asimov, what is your name?");
                        System.Threading.Thread.Sleep(50);
                    }

                    // Sing
                    else if (text == phraseList[12])
                    {
                        ezb.Tone.RandomTone(Digital.DigitalPortEnum.D8, 20, 10);
                        System.Threading.Thread.Sleep(50);

                        timer1.Start();
                    }

                    // Resume program
                    else if (text == phraseList[13])
                    {
                        ezb.Servo.ReleaseAllServos();
                        ezb.Dispose();

                        timer1.Start();
                    }

                    // End program
                    else if (text == phraseList[14])
                    {
                        ezb.SpeechSynth.Say("Just what do you think you're doing Dave?");

                        btn2.Text = "Run";

                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer1.Dispose();
                        ezb.Dispose();

                        isProgamRunning = false;
                    }

                    // Say nothing, return
                    else
                    {
                        timer1.Start();
                        //return;
                    }
                }
			}

			catch(Exception ex)
			{
				MessageBox.Show("Error speech = " + ex);
			}
		}
PRO
Synthiam
#9  
Everytime you run interaction(), there is a new method being assigned to the event. By adding multiple methods, there will be multiple peices of code executed one on command. The stack wiill continue to build up and up.

ZIP up your project file and attach it. i'll take a look at it if i have a moment
#10  
Please see the attachment.

And thank you for your help!

V6.zip


EDIT:

NOTE: that when you say "sing" the speaker that is connected to the EZ-B Board starts to play. But whenever you can go out the loop of the speech recognition/ synthesis, the speaker starts to play the random tones
#11  
Hi, I've cleaned up the code and the code is once again better then before.
Now we only have 1 problem, and that problem is when you say "resume" to the robot to continue on his own ("automaton") the robot won't start.

When I debug the code I see that the code returns to the "SpeechSynth_OnPhraseRecognized(, )" method,
however in the method for resuming the robot, it clearly states to start the timer

Enclosed is the project.

Thank you for your help!

V7.zip
#12  
Hey,
I'm trying to make a camera scanner like the ping radar scanner.
Not sure what I'm doing wrong here but I'm using the c# script editor that comes with the add control menu. I'm trying to read the face detection coordinates from the camera but am getting this error:

I'm able to read a servo position and have the machine speak it but the line of code to read the camera doesn't work. I don't have MS Visual Studio or any other compilers so I'm trying to familiarize myself with the that script editor being that I don't see any commands in the EZ-Script for reading from the camera etc. Prior to my most recent update, I used another program that would read from the debug window for face detection data and then it would manually move the controles in the ARC. But the Float controls option has now been removed. But learning the other scripting methods seemed easier in the long run.

Code:

using System;
using System.IO;
using System.Windows.Forms;
using EZ_B;

namespace VName {

  public class VClass {

    public void Main(EZB ezb0, EZB ezb1, EZB ezb2, EZB ezb3, EZB ezb4) {

int servoPos = ezb0.Servo.GetServoPosition(Servo.ServoPortEnum.D5);
int Facewhere = ezb0.CameraDetection.FaceDetection.GetFaceDetection;
ezb0.SpeechSynth.Say("Servo 5 is at position " + servoPos.ToString());

ezb0.SpeechSynth.Say("Face Found at " + Facewhere.ToString());


}
}
}


Here's the error I'm getting:
Cannot compile assembly. 1 Errors:
Line: 13 - 'EZ_B.EZB' does not contain a definition for 'CameraDetection'

If anyone has some sample code that works in the ARC for reading the camera that will solve everything.

Thanks
PRO
Synthiam
#13  
I'll make an example for you:)

I removed the Float Windows because I didn't know anyone used it lol. Some people complained about it even being there. It caused more issues during development than it was worth.
#14  
Thanks a lot DJ. BTW I so love the auto positioner and now the dance routines... What a scream!

Something I do in cases like that when I remove features for our providers, I'll create a ctrl key command for those who ask, "where's my ?" I'll tell them to hit CTRL ALT F or something.

Best,
Bill
Login to post a comment