Ez-face

JustinRatliff

USA
robot video thumbnail

EZ-Face is the first in what I plan to develop into a suite of supporting application for ARC and other robotics applications. EZ-Face performs multiple face recognition. It has a interface for training faces and assigning names. When the application sees faces that are recognized the names are displayed and visually you'll see boxes around the faces with the names assigned. If a face is detected but not recognized there will be a display of a box around the face with no name. The more pictures of a face you train, the easier it is for the application to recognize a face.

User-inserted image

This is a stand alone application developed in C# under Visual Studio.NET 2013. You should have .NET framework 4.5 and it is for Windows7 and Windows8.x systems.

This project showcase explains the technology behind the application and highlights development milestones.

Technology: Developed in .NET Visual Studio 2013 (you can use express versions with the source code) Designed to work with ARC but could be integrated into other software or robotic systems Is a standalone application Is open source, source code is included Uses emgu cv wrapper for .NET (Open CV)

Resources: (Things I found helpful in creating the application) ARC Telnet interface tutorial (Enable Telnet as the first part Shows, this is used to test communications manually to ARC via TCP/IP): https://synthiam.com/Tutorials/Help.aspx?id=159 If you do not have Telnet installed on your system go to this site: http://technet.microsoft.com/en-us/library/cc771275 ARC SDK Tutorial 52: https://synthiam.com/Community/Questions/4952&page=1 ARC script for listening to the TCP/IP port for variables: https://synthiam.com/Community/Questions/5255

Acknowledgements: DJ Sures, for making EZ-Robot and ARC so robust Rich, for his help with ARC scripting Sergio, for his emgu cv examples

Basic Usage Direction (after download and install): 1.) Open ARC and load the included EZ-Face example 2.) Click on the Script start button (this sets up the communications from the ARC side of things) 3.) Open the EZ-Face application 4.) Refresh your camera list (click the button) 5.) Select your camera (in the drop down list) 6.) Click the "1. Detect and recognize" button 7.) Train at least one face 8.) Change the local address and port number as needed (the local IP address may not be your computer's address - you can enter "localhost" and leave the port set to 6666 unless you changed that setting in ARC) 9.) Click File and select Save User Settings (to store your changes) 10.) Click Connection (this opens the communication line to ARC from the EZ-Face app side 11.) Allow EZ-Face to recognize the face you trained - then with your computer speakers turned on ARC should speak "Hello (the name of the face you trained)" 12.) If the example work - integrate in your EZ-Robot applications as you see fit

Tips: 1.) If after training several faces if you get false recognition of faces (faces recognized with the wrong name) - to correct this you should train the incorrectly recognized faces with the correct name. After a couple of training pictures are stored the accuracy of the face recognition will improve. 2.) Do not train faces with one camera, then switch to another camera for face recognition - recognition accuracy will drop.

Using Two Cameras: What I found worked best was to start ARC, select the camera I wanted and started the camera feed, then I started EZ-Face. If I reverses the process (even though I was selecting a different camera) I would get a black image in ARC.

2.26.14 Update: I still have several improvements I want to make before I upload the first public version of the application.

3.2.14 Update: The first public version is ready for release and is posted at the link below. This version has many user improvements to allow you store many settings, including http and port address, camera device, logging of faces in a text file (up to 1mb of data before the file auto deletes), face variable output to ARC, face training and more.

3.3.14 Update: I updated the script, version 3.3.14 has the HTTP server panel (which is not used - you don't need to start it) but it does show you your computer's IP address so you can enter it in EZ-Face. Remember to save your settings under the File Menu. I also changed the script so it will no only speak for variable values greater than "" or NULL.
EZ-Face3.3.14.EZB

3.7.14 Update: I updated the EZ-Face application: "localhost" is now the default address, new option for auto connect, functions to receive commands from ARC or other 3rd party application to stop and start the camera feed within EZ-Face. There is also a new ARC project with several new scripts to test out the functions. Please go to my site to download the latest version. You will also find a video there that demonstrations the new functions and provides directions for setup and usages.

Download: The latest version will be published here: http://www.j2rscientific.com/software For support and reporting any errors please use the ContactUs feature from http://www.j2rscientific.com with the subject line "EZ-Face".

I welcome any and all feedback!
Thank you

By — Last update
Jump to end

ARC Pro

Upgrade to ARC Pro

ARC Pro is your passport to a world of endless possibilities in robot programming, waiting for you to explore.

#73  

Hi, how to stop the script when a face is detected? Thanks

United Kingdom
#74  

I'll be posting some of my script examples soon with a few features you can all build on.

If you want the script to stop on face detect, either replace the Say("hello" + $facename) line with Halt() or just add a Halt() after the line.

The thing to remember here is you have free reign with what this app does. The Example Project supplied with it only does one thing. It would be impossible to cover everything that can be done. There is so much that can be done from the $FaceName variable.

Brazil
#75  

friends.. 2 questions. How do I setup ARC AND Ez-face to share the same camera ?! My robot uses one single camera tha I would like to keep doing the tracking job, and the recognition too... At this point, I was only able to make one works, if I disable the other...

And, How do I setup the Ez-Face to automatically RUN AND CONNECT , after load when I turn the robot on (it uses an onboard ITX computer) ?

That's because the robot is set to load, and connect ARC at the windows start up..

THANKS!

United Kingdom
#76  

You can't use the same camera on two applications, this is not a limitation to EZ-Face only.

I did offer a suggestion to Justin a few days ago about using the HTTP server control and grabbing the image however it is a static image not a video feed so I am guessing he is still working on that for a possible future upgrade.

You can have EZ-Face start automatically a few ways. I would suggest using ARC to start it within an init script.

Exec()

Which, if installed in the default location should be;

Exec("C:\BotBrain\EZ-Face\bin\Debug\MultiFaceRec.exe")
#77  

@Rich, thank you it works. And thank you a thousand times @JustinRatliff. It's really great

#78  

@Rich is right, the including EX-Builder script is just a basic example of passing a variable from EZ-Face to ARC. I'm looking forward to seeing the scripts Rich produces to make a more robust script.

@tevans Single camera usage is something I am working on. The new functionality will allow you to call up EZ-Face to run on demand (open, use the camera, release the camera and close) and you can call it up as often you like. That functionality is not included in the first version.

@Rich EZ-Face is built on the EMGU CV framework. The problem with the HTTP server function in ARC is it provides an HTTP IP address for a camera image (static is currently the only option from ARC, but that is not the main issue). The problem is two fold. One there is not a "clean" way for EMGU to accept HTTP IP addresses as a camera feed. From everything I have seen it really seems designed to pull only from traditional capture device. The examples I have seen for IP address video feeds have included 3rd party support files that must be purchased (no open source solutions).

One way to get around all this is to use what is in ARC already, which is the picture snapshot function. Have a script look for a face in ARC, take a snap shot or two, send a variable to EZ-Face, EZ-Face would look for new pictures, run recognition, store faces to variable, send variable to ARC where ARC could do the needful. But the EMGU functionality utilized in EZ-Face is designed for video, not pictures. So adding that functionality is not straight forward and I anticipate that requiring a fair amount of work.

However, if ARC had an option to export a video clip - that functionality I could add to EZ-Face without much effort. @DJSures - do you guys have adding video clip output on your radar by any chance?

United Kingdom
#79  

Here are a few different quick examples. I haven't had much time to play with it yet but these should be a starter for you to use and adapt to suit your requirements...

Greet user only on change of user while ignoring no recognition/non detection.

# Make sure the variable exists and is reset
$FaceName = "null"
$user0 = "null"
$user1 = "null"

# Make an endless loop
:loop

# Wait until the variable changes
WaitForChange($FaceName)

# Check facename isn't blank
IF ($FaceName != "")
  $user1 = $user0
  $user0 = $FaceName

  # Check detected face is different from previously detected
  IF($user0 != $user1)
    # Script commands to be run on change of facename variable
    Say("Hello " + $user0)
  EndIf

EndIf

# Loop back to the start
Goto(loop)

Enable/Disable Speech Recognition on detected user (note: change "Rich," to the user you want)

# Make sure the variable exists and is reset
$FaceName = "null"
$user0 = "null"
$user1 = "null"

# Make an endless loop
:loop

# Wait until the variable changes
WaitForChange($FaceName)

# Check facename isn't blank
IF ($FaceName != "")
  $user1 = $user0
  $user0 = $FaceName

  # Check detected face is different from previously detected
  IF($user0 != $user1)
    # Script commands to be run on change of facename variable
    Say("Hello " + $user0)
  EndIf

EndIf

# Enable Speech Recognition control only if user is Rich
IF ($user0 != "Rich,")
  # If user isn't Rich pause the speech recognition
  ControlCommand("Speech Recognition", PauseOn)
Else
  # Otherwise unpause speech recognition
  ControlCommand("Speech Recognition", PauseOff)
EndIf

# Loop back to the start
Goto(loop)

When I have more time I will do a few more based on ideas I have. If you have ideas but no clue on how to make it work post the idea and I'll see what I can do:)

#80  

So I am not able to get this working on my end. Not quite sure what I'm missing. I pressed the start button and nothing is happening? Im sure there is more to this than what I am doing. Anyone willing to take a stab at directing me? Thanks Chris