Belgium
Asked — Edited

Camera Facedetection Threadable?

Skip to comments   Jump to end

Hello all,

I have a question concerning the camera usage in visual studio 10, C#.

Is it possible to use this with a thread?

The problem is that my program is starting to run very very slowly when the camera is activated for facial detections

This is my code



            if (!ezb.Camera.IsActive)
			{
				return;
			}

			ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

			if (!objectLocation.isFound)
			{
				turnAway = true;
				timer2.Start();
				timer4.Stop();
				timer4.Dispose();
			}

			if (objectLocation.isFound)
			{
				interactie();
				textBox1.AppendText(Environment.NewLine);
				textBox1.AppendText("Face found");
				textBox1.AppendText(Environment.NewLine);
			}

When the program sees a face it goes to the method "interact();" or in English "interaction();"

But then the program seems to halt and running slowly

Thanks in advance for the help.

Cheers,

Laurens


ARC Pro

Upgrade to ARC Pro

Join the ARC Pro community and gain access to a wealth of resources and support, ensuring your robot's success.

PRO
Synthiam
#1  

That is how you would do it, yes. The speed of your program depends on the entire program, not just that sniplet. It matters how you are executing that bit of code. Is that being executed in a timer? Is it in a loop? How many times a second are you calling it? Is any throttling mechanism in place to prevent it from eating up all available cpu cycles within the loop?

If you paste more code, i can help you

#2  

Hi DJ, Thx for helping me.

The code for the cam is being executed by a timer. The cam must be active whenever it comes across an object. Then it will scan if it is a human. When I run it the program "works" but when the cam is active the cam stops working, but the program continuous to work.

But the cam won't activate anymore

This is my whole code:


using System;
using System.Drawing;
using System.Windows.Forms;

using EZ_B;
using FaceDetect;

using Microsoft.DirectX;
using Microsoft.DirectX.DirectInput;

using System.Threading;

namespace WindowsFormsApplication1
{
	public partial class Form1 : Form
	{
		/// 
		/// Initialisatie
		/// 
        /// 
        private EZB ezb = new EZB();

        // Bools
		private bool isProgamRunning = false;
		private bool direction = true;
        private bool turnAway = false;

		private int distanceObject = 0;
		private int turnDirection = 0;

        private string videoCaptureDeviceName = "";

		private Random randomTurnDirection = new Random();

        // Timers
        private System.Windows.Forms.Timer timer1 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer2 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer3 = new System.Windows.Forms.Timer();
        private System.Windows.Forms.Timer timer4 = new System.Windows.Forms.Timer(); //CAM

		//Woordenlijst
		private string[] phraseList =
		{
			"go forward",		// 0
			"go right",			// 1
			"go reverse",	    // 2
            "go left",			// 3
            "stop",				// 4
            "look up",			// 5
            "look down",		// 6
            "left arm down",	// 7
            "left arm up",	    // 8
            "right arm up",     // 9
            "right arm down",   // 10
			"teddy speak",		// 11
            "sing",             // 12
            "resume",           // 13
			"goodnight",        // 14
        }; 

		public Form1()
		{
			InitializeComponent();

			btn2.Enabled = false;
			comboBoxCam.Enabled = false;

			textBox1.Text = Resource1.message1;

			comboBoxPorts.Items.Clear();
			comboBoxPorts.Items.AddRange(ezb.GetAvailableCommunicationPorts());
			comboBoxCam.Items.Clear();
			
            foreach (string name in ezb.Camera.GetVideoCaptureDeviceList())
			{
				comboBoxCam.Items.Add(name);
			}

            label2.Text = "Afstand = " + (distanceObject * 2.54).ToString() + " cm";
		}

		private void Form1_Load(object sender, EventArgs e)
		{
			ezb.Movement.MovementType = Movement.MovementTypeEnum.ModifiedServo;
			ezb.Movement.ServoWheelLeftModifiedPort = Servo.ServoPortEnum.D13;
			ezb.Movement.ServoWheelRightModifiedPort = Servo.ServoPortEnum.D14;

			turnDirection = randomTurnDirection.Next(0, 2);

			timer2.Interval = 2000;
			timer2.Tick += new EventHandler(timer2_Tick);
			timer3.Interval = 1000;
			timer3.Tick += new EventHandler(timer3_Tick);
			timer4.Tick += new EventHandler(timer4_Tick);
		}

		protected override void OnClosed(EventArgs e)
		{
			ezb.Servo.ReleaseAllServos();
			timer1.Stop();
			timer2.Stop();
			ezb.Disconnect();
			ezb.Dispose();
		}

		private void comboBoxCam_SelectedIndexChanged(object sender, EventArgs e)
		{
			videoCaptureDeviceName = comboBoxCam.SelectedItem.ToString();
			ezb.Camera.StartCamera(videoCaptureDeviceName, panel1);

			timer4.Interval = 100;
            timer4.Enabled = true;
		}

		/// 
		/// Connecteren en Disconnecteren met EZB
		/// 
		private void btn1_Click(object sender, EventArgs e)
		{
            try
            {
                if (ezb.IsConnected)
                {
                    ezb.Disconnect();
                }

                else
                {
                    ezb.Connect(comboBoxPorts.SelectedItem.ToString());
                    comboBoxCam.Enabled = false;
                }

                if (ezb.IsConnected)
                {
                    btn1.Text = "Disconnect";
                    btn2.Enabled = true;
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Connected !");
                    comboBoxCam.Enabled = true;
                }

                else
                {
                    btn1.Text = "Connect";
                    btn2.Enabled = false;
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Disconnected !");
                    comboBoxCam.Enabled = false;
                }
            }
            
            catch(Exception ex)
            {
                MessageBox.Show("Error = " + ex);
            }
		}

		/// 
		/// De knop die instaat voor het starten en stoppen van het programma.
		/// Via een boolean is er controlle op de status van het programma.
		/// 
		private void btn2_Click(object sender, EventArgs e)
		{
			if (!isProgamRunning)
			{
				isProgamRunning = true;
				btn2.Text = "Stop";

                //Run_Program();
                Run_Program_Delegate rpd = new Run_Program_Delegate(Run_Program);
                Run_Program();
                
			}

			else
			{
				isProgamRunning = false;
				btn2.Text = "Run";

			    ezb.Servo.ReleaseAllServos();
			    timer1.Stop();
                timer1.Dispose();
			    timer2.Stop();
                timer2.Dispose();
			    timer3.Stop();
                timer3.Dispose();
			    timer4.Stop();
                timer4.Dispose();
			    ezb.Dispose();
			}
		}

		private void Run_Program()
		{
            timer1.Start();
			timer1.Tick += new EventHandler(timer1_Tick);
		}

		/// 
		/// TIMER1 = PING SENSOR
		/// 
		/// 
		/// 
		private void timer1_Tick(object sender , EventArgs e)
		{
            distanceObject = ezb.HC_SR04.GetDistance(Digital.DigitalPortEnum.D0, Digital.DigitalPortEnum.D0); //afstand in Inches
            progressBar1.Value = (int)(distanceObject * 2.54);

            label2.Text = ("Afstand = " + (distanceObject * 2.54).ToString() + " cm");

            if (distanceObject 
		/// TIMER2 = STOP
		/// 
		/// 
		/// 
		private void timer2_Tick(object sender, EventArgs e)
		{
			timer3.Start();
			timer2.Stop();
            timer2.Dispose();
			timer1.Start();
		}

		/// 
		/// TIMER3 = DRAAIEN
		/// 
		/// 
		/// 
		private void timer3_Tick(object sender, EventArgs e)
		{
			if (turnDirection == 0)
			{
				ezb.Movement.GoStop();
				ezb.Movement.GoLeft();
			}

			else
			{
				ezb.Movement.GoStop();
				ezb.Movement.GoRight();
			}
			timer3.Stop();
            timer3.Dispose();
		}

		/// 
		/// TIMER4 = CAMERA
		/// 
		/// 
		/// 
        private void timer4_Tick(object sender, EventArgs e)
        {
            if (!ezb.Camera.IsActive)
            {
                isProgamRunning = false;
                btn2.Text = "Run";

                ezb.Servo.ReleaseAllServos();
                timer1.Stop();
                timer1.Dispose();
                timer2.Stop();
                timer2.Dispose();
                timer3.Stop();
                timer3.Dispose();
                timer4.Stop();
                timer4.Dispose();
                ezb.Dispose();

                MessageBox.Show("Geen camera geconnecteerd.nProgramma wordt gestopt!");
            }

            else
            {
                ezb.Camera.UpdateImage();
                ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

                if (objectLocation.isFound == false)
                {
                    turnAway = true;

                    timer2.Start();
                    timer4.Stop();
                    timer4.Dispose();
                }

                else
                {
                    textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Face detected!");

                    interactie();
                }
            }
		}

		private void interactie()
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);
            ezb.SpeechSynth.StartListening();
			ezb.SpeechSynth.OnPhraseRecognized += new SpeechSynth.OnPhraseRecognizedEvent(SpeechSynth_OnPhraseRecognized);
		}

		private void SpeechSynth_OnPhraseRecognized(string text, float confidence)
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);

			try
			{
                if (confidence < 0.85)
                {
                    return;
                }

                else
                {
                    if (text == phraseList[0])
                    {
                        ezb.Movement.GoForward();
                    }

                    else if (text == phraseList[1])
                    {
                        ezb.Movement.GoRight();
                    }

                    else if (text == phraseList[2])
                    {
                        ezb.Movement.GoReverse();
                    }

                    else if (text == phraseList[3])
                    {
                        ezb.Movement.GoLeft();
                    }

                    else if (text == phraseList[4])
                    {
                        ezb.Movement.GoStop();
                    }

                    else if (text == phraseList[5])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 50);
                    }

                    else if (text == phraseList[6])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 25);
                    }

                    else if (text == phraseList[7])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 50);
                    }

                    else if (text == phraseList[8])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 25);
                    }

                    else if (text == phraseList[9])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 50);
                    }

                    else if (text == phraseList[10])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 25);
                    }

                    else if (text == phraseList[11])
                    {
                        ezb.SpeechSynth.Say("Hello my name is teddy Asimov, what is your name?"); 
                    }

                    else if (text == phraseList[12])
                    {
                        ezb.Tone.RandomTone(Digital.DigitalPortEnum.D8, 20, 10);
                    }

                    else if (text == phraseList[13])
                    {
                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer2.Stop();
                        timer3.Stop();
                        timer4.Stop();
                        ezb.Dispose();

                        Run_Program();
                    }

                    else if (text == phraseList[14])
                    {
                        int i;

                        for (i=0 ; i<=1 ; i++)
                        {
                            ezb.SpeechSynth.Say("Just what do you think you're doing Dave?");
                            i++;
                        }

                        btn2.Text = "Run";

                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer2.Stop();
                        timer3.Stop();
                        timer4.Stop();
                        ezb.Dispose();

                        isProgamRunning = false;
                    }
                }
			}

			catch(Exception ex)
			{
				MessageBox.Show("Error speech = " + ex);
			}
		}
	}

    delegate void Run_Program_Delegate();
}
PRO
Synthiam
#3  

There are a lot of thread timers. timers that are unused. primarily because they all share the same thread anyway. Have you looked at Omnibot's code example? I think that is more of what you are trying to do. You only need one timer to do everything.

#4  

Allright, thx.

We will look into that.

A small side-question: Is it possible to use more then one class? Like the form only as the representation level and other classes for face detection, voice interaction, movement?

PRO
Synthiam
#5  

What do you mean use more than one class?

#6  

Like you have your Form1.cs and Program.cs classes. Now we only need to improve the performance of our code

The program now is using 1 timer and running smoother then before! Thx for the reference!

#7  

The code is working much better now, but we still have a small problem with the voice recognition.

When the robot ping sensor sees an object (<= 4 inches) it stops and checks with the camera is the object is a human. If the object is not a human, the robot turn left or right and goes on.

This works, but when the robot detects a human face. The program "jumps" to the interaction method for voice detection/synthesis.

The robot does this, but is stuck in a loop and repeats the same command over & over again (i.e. you say "sing" the robot sings, but when the robot says your face again, it start to sing all over again)

#8  

To make it easier to follow the program here is my new code:


        private void timer1_Tick_1(object sender, EventArgs e)
        {
            distanceObject = ezb.HC_SR04.GetDistance(Digital.DigitalPortEnum.D0, Digital.DigitalPortEnum.D0); //afstand in Inches
            progressBar1.Value = (int)(distanceObject * 2.54);

            label2.Text = ("Afstand = " + (distanceObject * 2.54).ToString() + " cm");

            if (ezb.Camera.IsActive)
            {
                ezb.Camera.UpdateImage();
            }

            // <= 4 inches = 10,16cm
            if (distanceObject <= 4)
            {
                ObjectLocation objectLocation = ezb.CameraFaceDetection.GetFaceDetection();

                if (objectLocation.isFound == true)
                {
                    if (interactionDone == false)
                    {
                        ezb.Movement.GoStop();

                        textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Face detected!");

                        interaction();
                        System.Threading.Thread.Sleep(50);
                    }

                    else
                    {
                        interactionDone = false;
                    }
                }

                else
                {
                    turning();
                    System.Threading.Thread.Sleep(50);
                }

                ezb.Movement.GoStop();
                System.Threading.Thread.Sleep(50);
            }

            else
            {
                ezb.Movement.GoForward();
                System.Threading.Thread.Sleep(50);
            }
        }

        private void turning()
        {
            if (turnDirection == 0)
            {
                ezb.Movement.GoLeft();
            }

            else
            {
                ezb.Movement.GoRight();
            }
        }

		private void interaction()
		{
            textBox1.AppendText(Environment.NewLine + Environment.NewLine + "Into interaction mode");

            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);
            ezb.SpeechSynth.StartListening();
			ezb.SpeechSynth.OnPhraseRecognized += new SpeechSynth.OnPhraseRecognizedEvent(SpeechSynth_OnPhraseRecognized);
		}

		private void SpeechSynth_OnPhraseRecognized(string text, float confidence)
		{
            //spraakherkenning woordenboek inladen!
            ezb.SpeechSynth.SetDictionaryOfPhrases(phraseList);

			try
			{
                if (confidence < 0.85)
                {
                    return;
                }

                else
                {
                    interactionDone = true;

                    // Go forward
                    if (text == phraseList[0])
                    {
                        ezb.Movement.GoForward();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go right
                    else if (text == phraseList[1])
                    {
                        ezb.Movement.GoRight();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go revese
                    else if (text == phraseList[2])
                    {
                        ezb.Movement.GoReverse();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Go left
                    else if (text == phraseList[3])
                    {
                        ezb.Movement.GoLeft();
                        System.Threading.Thread.Sleep(50);
                    }

                    // STOP
                    else if (text == phraseList[4])
                    {
                        ezb.Movement.GoStop();
                        System.Threading.Thread.Sleep(50);
                    }

                    // Look up
                    else if (text == phraseList[5])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Look down
                    else if (text == phraseList[6])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D6, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Left arm up
                    else if (text == phraseList[7])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Left arm down
                    else if (text == phraseList[8])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D2, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Right arm up
                    else if (text == phraseList[9])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 50);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Right arm down
                    else if (text == phraseList[10])
                    {
                        ezb.Servo.SetServoPosition(Servo.ServoPortEnum.D4, 25);
                        System.Threading.Thread.Sleep(50);
                    }

                    // Speak
                    else if (text == phraseList[11])
                    {
                        ezb.SpeechSynth.Say("Hello my name is teddy Asimov, what is your name?");
                        System.Threading.Thread.Sleep(50);
                    }

                    // Sing
                    else if (text == phraseList[12])
                    {
                        ezb.Tone.RandomTone(Digital.DigitalPortEnum.D8, 20, 10);
                        System.Threading.Thread.Sleep(50);

                        timer1.Start();
                    }

                    // Resume program
                    else if (text == phraseList[13])
                    {
                        ezb.Servo.ReleaseAllServos();
                        ezb.Dispose();

                        timer1.Start();
                    }

                    // End program
                    else if (text == phraseList[14])
                    {
                        ezb.SpeechSynth.Say("Just what do you think you're doing Dave?");

                        btn2.Text = "Run";

                        ezb.Servo.ReleaseAllServos();
                        timer1.Stop();
                        timer1.Dispose();
                        ezb.Dispose();

                        isProgamRunning = false;
                    }

                    // Say nothing, return
                    else
                    {
                        timer1.Start();
                        //return;
                    }
                }
			}

			catch(Exception ex)
			{
				MessageBox.Show("Error speech = " + ex);
			}
		}
Login to post a comment