D.cochran's Teddy

StorySkip to comments

I am going to try to document this project because I think its pretty cool so far. It is also the beginning of a much larger project.

For the hardware right now, I am using a
Teddy Ruxpin bought on ebay
EZB V4
V4 Camera
2 rotational HD servos for shoulders
1 Rotational servo for neck
1 micro servo for mouth
amplified speaker (used for phones)
Scary Terry Audio Servo Driver Board

For software
EZ-Builder
MSSQL
EZ-DBMulti
rssGetter
RoboRealm

Project Description
I wanted to design a robot for my grandchildren but also use this as a platform to test some things that I want to do with my InMoov project. The robot needs to be able to identify the user and then customize itself to what that user would use/need. For example, if I was using the robot, I want the robot to be able to read me the traffic, weather and news from RSS Feeds. If my granddaughter is using the robot, I want it to be able to tell nursery rhymes and sing songs that are for her age level. Because my granddaughter is a primary focus for this robot, I wanted the jaw to move reliably when the robot is speaking.

ROBOREALM CONFIGURATION
To accomplish this, I had to have a way to identify which user is using the robot. RoboRealm has the ability to recognize faces and to be able to pass variables back to EZ-Builder. It also has the ability to use a feed from an http server for its camera source.

To make RoboRealm work, I added three modules to it. These are ReadHTTP, AVM Navigator and EZRobot_Variables. ReadHTTP is configured to access http://127.0.0.1:8010/CameraImage.jpg?c=Camera. I will get to the configuration of the webserver within EZ-Builder in a few moments. AVM Navigator is setup in Object Recognition mode with my face (and the other members of my family's face) trained. EZRobot_Variables is configured to pass the NV_ARR_OBJ_NAME variable back to EZ-Builder on host 127.0.0.1 port 6666 with a prefix of RR_.

EZ-BUILDER
HTTP Custom Server
The HTTP Custom Server is configured to run on TCP Port 8010. This is because I am already using 80 and 8080 for other development projects.

EZ-BUILDER
Startup script

Code:


$rsstext = ""
$rssLocation = ""
$feedDescription = ""
ControlCommand("HTTP Custom Server", StartServer)
Sleep(2000)
ControlCommand("Script Manager", ScriptStop, "Monitor")
ControlCommand("Script Manager", ScriptStart, "Monitor")
ControlCommand("Sound Servo", PauseOn)
Exec("c:\RoboRealm\RoboRealm.bat")
ControlCommand("Script Manager", ScriptStart, "WhoAmI")
Sleep(5000)
ControlCommand("Script Manager", ScriptStart, "GetDatabaseInfo")



This script prepares a couple of variables for later use, then starts the HTTP Custom Server so that RoboRealm can use the V4 camera feed through the HTTP Custom Server. It then starts a monitor script and then pauses the Sound Servo control. It then launches RoboRealm and runs a WhoAmI script before going to the database server to retrieve information for the user that is using the robot.

MONITOR SCRIPT

Code:


:MonitorFunction
$connected = IsConnected( 0 )
if($connected = 0)
ControlCommand("Script Manager", ScriptStart, "CheckConnection")
endif
$CPUTemp = GetCPUTemp()
if($CPUTemp > 30)
ControlCommand("Sound Servo", PauseOn)
sayezb("The temp of the process has exceeded safe parameters")
endif
$CurrentVoltage = GetVoltage()
if ($CurrentVoltage < 6.6)
ControlCommand("Sound Servo", PauseOn)
sayezb("You will have to charge me soon")
endif
sleep(5000)
Goto(MonitorFunction)



This script first checks to see if there is a connection to the EZ-B on the robot. If one doesn't exist, it then runs a script called CheckConnection. It then checks the temp of the CPU in the robot and warns if there appears to be a pending issue. It then checks the voltage of the battery and warns the user if they will need to charge the battery soon. It then waits 5 seconds and the loops to the start of the script again. This monitor keeps running while EZ-Builder is connected to the robot.

CHECKCONNECTION

Code:


$connected = IsConnected( 0 )
REPEATUNTIL($connected = 1)
If($connected = 0)
$Connection=0
Else
$Connection = 1
EndIf
if($connection = 0)
ControlCommand("Connection", Connect0)
sleep(3000)
$connected = IsConnected( 0 )
endif
EndRepeatUntil



This script is called by the monitor script and first checks to see if the connection is made to the robot. If there is not a connection, it tries to establish the connection and waits 3 seconds for the connection to be established. If it isn't established, the script tries again.

ROBOREALM.BAT

Code:


D:
CD\Program Files (x86)\RoboRealm
RoboRealm


This just simply executes the RoboRealm application.

WhoAmI

Code:


SayEZB("Let me have a good look at you.")
controlcommand("Script Manager", scriptStart, "GreetuserRich")



This is a very simple script that just says "Let me have a good look at you." and then launches the GreetUserRich. This could be combined with the GreetUserRich script and this one could be eliminated completely.

GREETUSERRICH

Code:


$user0 = "null"
IF ( $RR_NV_ARR_OBJ_NAME != "" )
$user0 = $RR_NV_ARR_OBJ_NAME
SayEZB("You are " + $user0 + ". It is great to see you!")
EndIf



This script was greatly reduced from what I had before. RoboRealm is very fast and very good at facial recognition. Because of this, I was able to reduce the amount of code here for checking things. If it becomes unreliable with many faces learned in RoboRealm, I will start adding in more checking. When RoboRealm is called, and a face is recognized from the HTTP camera feed from EZ-Robot, the $RR_NV_ARR_OBJ_NAME is passed back to EZ-Builder as a variable. This script takes the value of this variable and assigns it to $User0 and then says "You are . It is great to see you!"

GETDATABASEINFO

Code:


#Starting variables for EZ-DB
$UserName = ""
$UserName = $user0
$SqlReturn = " "
$SqlCommand = ""
$FatherName = ""
$MotherName = ""
$GrandmaName = ""
$GrandpaName = ""
$NumberOfVariablesToStore = 2
$RobotName = "Teddy"
$RobotID = ""
ControlCommand("Script Manager", ScriptStart, "GetRobotIDFromDatabase")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetGrandpaFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetGrandmaFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetFatherFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetMotherFromDB")
Sleep(1000)



A lot of variables are initialized here, and the $user0 variable value is then assigned to $UserName. The $RobotName variable is also set. This script then calls 5 other scripts that are all similar. All of these connect to a database through EZ-DBMulti to pull specific data back with 4 of them pulling back specific data about the current user based on the facial recognition that has taken place.

GETROBOTIDFROMDATABASE

Code:


$SqlCommand = "EXEC usp_GetRobotIDFromName '"+$RobotName+"'"

FileWrite("C:\EZ-DBMulti\Query.txt",$SqlCommand)
FileWrite("C:\EZ-DBMulti\Robot.txt",$RobotName)

EXEC("C:\EZ-DBMulti\EZ-DBMulti.exe", "In")
Sleep(1000)
$RobotID = $SQLReturn


This script prepares a sql command which executes a stored procedure on a MSSQL database called usp_GetRobotIDFromName and passes in the $RobotName variable value into the stored procedure as a parameter. It stores this information in 2 files. The first of these is called Query.Text (containing the command that is going to execute. The second is called Robot and contains the $RobotName value (really not used by this stored procedure, but EZ-DBMulti's database houses information for multiple robots.) It then calls EZ-DBMulti passing in whether EZ-DBMulti is supposed to retrieve data from the database (In) or pass data to be stored in the database (Out). After the query executes, the result from the execution of the stored procedure in the database is stored in the $SQLReturn variable in EZ-Builder so I then assign the $RobotID variable value to the value of the $SQLReturn variable.

SQL Server
usp_GetRobotIDFromName

Code:


USE
GO
/****** Object: StoredProcedure . Script Date: 1/26/2015 8:00:51 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2015
-- Description: Get Robot ID from its name
-- =============================================
ALTER PROCEDURE .
-- Add the parameters for the stored procedure here
(@RobotName VARCHAR(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;

SELECT
RobotID
FROM
Robots
WHERE
RobotName = @RobotName
END



This one is really unspectatular, but I wanted to show what the USP is doing. It simply is looking into the Robots table, and is returning the RobotID where the RobotName matches the $RobotName variable value from EZ-Builder (it was passed in when the USP was called through EZ-DBMulti.)

SQL SERVER
USP_GRANDPADATA

Code:


USE
GO
/****** Object: StoredProcedure . Script Date: 1/26/2015 8:03:57 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2014
-- Description: Returns Grandfathers data for a named user
-- =============================================
ALTER PROCEDURE .
(@username Varchar(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
GParentDetails.
FROM
.
INNER JOIN . ON ..UserID = UserInfo.UserID
INNER JOIN . ON ..UserID = ..
INNER JOIN . ON .. = ..
INNER JOIN . as Parents on
.. = Parents.FamilyID
AND FamilyTree.Parent = Parents.UserID
INNER JOIN . as GParents on
.. = GParents.FamilyID
AND Parents.Parent = GParents.UserID
INNER JOIN dbo.Users as GParentDetails ON
GParents.UserID = GParentDetails.UserID
INNER JOIN dbo.UserInfo AS GParentDetail ON
GParents.UserID = GParentDetail.UserID
AND GParentDetail.Gender = 'M'
WHERE dbo.Users.Username = @username
Group By
GParentDetails.
END


This USP is a bit more complex. Gets the UserFamilyName for the male grandparent of the user. The joins are used to map the relationship from the user up to the parent and then the grandparent of the user.

SQL SERVER
USP_GETMOTHERDATA

Code:


USE
GO
/****** Object: StoredProcedure . Script Date: 1/26/2015 8:07:11 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2014
-- Description: Returns Mother data for a named user
-- =============================================
ALTER PROCEDURE .
(@username Varchar(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
CASE when DATEDIFF(YYYY,..,GETDATE()) < 6
THEN
ParentDetails.
ELSE
ParentDetails.
END AS ParentName
FROM
.
INNER JOIN . ON ..UserID = UserInfo.UserID
INNER JOIN . ON ..UserID = ..
INNER JOIN . ON .. = ..
INNER JOIN . as Parents on
.. = Parents.FamilyID
AND FamilyTree.Parent = Parents.UserID
INNER JOIN dbo.Users as ParentDetails ON
Parents.UserID = ParentDetails.UserID
INNER JOIN dbo.UserInfo AS ParentDetail ON
Parents.UserID = ParentDetail.UserID
AND ParentDetail.Gender = 'F'
WHERE dbo.Users.Username = @username

END



This one is similar to the previous one in that it identifies the female parent of the current user. It also uses a CASE statement that checks to see what the age of the current user is. If the current user is less than 6 years old, it returns a family name (such as Mommy) for this parent. If the user is older than 6, it grabs the username for the mother. This can be extended much further than I currently have, but currently, when the user of this robot reaches age 6, the value returned changes and the robot automatically changes because of the change in age of the user.


I will be adding additional things like getting the birthday of the user or anniversary, or holidays such as Christmas, Easter and many more so that different phrases can be said based on the age of the user and the day of the year. I also plan to add things like the last time that something happened (for example a nursery rhyme was said) for a user and limit the robots activity based on this. It could be a song was played, or news was read, or whatever. I might also have a game for my granddaughter and knowing what questions were asked last time or previously, I could then ask other questions that were not asked yet or have been asked less frequently. Anyway, I will be posting the project file on the cloud if anyone wants to use anything that I have done.

There is more in there, I am just worn out on typing this right now. I will probably add more and a video later, but I wanted to start documenting. If you are still reading this and have any questions, please let me know.

sweetn8
Commented July 2015
@d.cochran Thank you so much for sharing such detailed instructions. I too am working on a Teddy Ruxpin and am planning on following your instructions from the beginning so I can use the same settings, etc.
CochranRobotics
Commented July 2015
If you are going to use EZ-AI, I would be happy to help you out. It is a lengthy install. It has also come a long way since these posts.
sweetn8
Commented July 2015
Yes, I am planning on using your EZ-AI. I am excited about all it can do. I know several children who will want to play with Teddy and having it remember them is amazing. Thank you for your offer to help! I will take you up on that! I was following the thread for that as well.
reaktor
Commented January 2016
Hello i'm interested by EZ-DBMulti (not EZ-AI for the moment) but i can't find it and this link is dead : http://cochranrobotics.com/Portals/0/EZ-DB%20Multi%20Source.zip
Can you help me?
Thanks
CochranRobotics
Commented January 2016
Hello wofty.

This was rolled into EZ-AI and is one of the core pieces that made that version of EZ-AI work. Because of this, and because there really were not many downloads of this app, I ended up killing it off in favor of people using EZ-AI.

In order to try to know how to help you, can you tell me what you are looking for? Also, can you tell me if you have any programming experience with C#? This should give me the information that I need to try to figure out the best way to assist.

If you have experience with C#, I would be happy to let you know what this did and how it did it. It would be possible to develop a plugin that does what EZ-DBMulti did but may take some time for me due to other obligations. If I were rewriting it with what is now available, I would definitely make it a plugin.

Thanks
David
Robot
AvatarCochranRobotics
Published on Monday, January 26, 2015