D.cochran's Teddy

 
#1

I am going to try to document this project because I think its pretty cool so far. It is also the beginning of a much larger project.

For the hardware right now, I am using a
Teddy Ruxpin bought on ebay
EZB V4
V4 Camera
2 rotational HD servos for shoulders
1 Rotational servo for neck
1 micro servo for mouth
amplified speaker (used for phones)
Scary Terry Audio Servo Driver Board

For software
EZ-Builder
MSSQL
EZ-DBMulti
rssGetter
RoboRealm

Project Description
I wanted to design a robot for my grandchildren but also use this as a platform to test some things that I want to do with my InMoov project. The robot needs to be able to identify the user and then customize itself to what that user would use/need. For example, if I was using the robot, I want the robot to be able to read me the traffic, weather and news from RSS Feeds. If my granddaughter is using the robot, I want it to be able to tell nursery rhymes and sing songs that are for her age level. Because my granddaughter is a primary focus for this robot, I wanted the jaw to move reliably when the robot is speaking.

ROBOREALM CONFIGURATION
To accomplish this, I had to have a way to identify which user is using the robot. RoboRealm has the ability to recognize faces and to be able to pass variables back to EZ-Builder. It also has the ability to use a feed from an http server for its camera source.

To make RoboRealm work, I added three modules to it. These are ReadHTTP, AVM Navigator and EZRobot_Variables. ReadHTTP is configured to access http://127.0.0.1:8010/CameraImage.jpg?c=Camera. I will get to the configuration of the webserver within EZ-Builder in a few moments. AVM Navigator is setup in Object Recognition mode with my face (and the other members of my family's face) trained. EZRobot_Variables is configured to pass the NV_ARR_OBJ_NAME variable back to EZ-Builder on host 127.0.0.1 port 6666 with a prefix of RR_.

EZ-BUILDER
HTTP Custom Server
The HTTP Custom Server is configured to run on TCP Port 8010. This is because I am already using 80 and 8080 for other development projects.

EZ-BUILDER
Startup script

Code:


$rsstext = ""
$rssLocation = ""
$feedDescription = ""
ControlCommand("HTTP Custom Server", StartServer)
Sleep(2000)
ControlCommand("Script Manager", ScriptStop, "Monitor")
ControlCommand("Script Manager", ScriptStart, "Monitor")
ControlCommand("Sound Servo", PauseOn)
Exec("c:\RoboRealm\RoboRealm.bat")
ControlCommand("Script Manager", ScriptStart, "WhoAmI")
Sleep(5000)
ControlCommand("Script Manager", ScriptStart, "GetDatabaseInfo")



This script prepares a couple of variables for later use, then starts the HTTP Custom Server so that RoboRealm can use the V4 camera feed through the HTTP Custom Server. It then starts a monitor script and then pauses the Sound Servo control. It then launches RoboRealm and runs a WhoAmI script before going to the database server to retrieve information for the user that is using the robot.

MONITOR SCRIPT

Code:


:MonitorFunction
$connected = IsConnected( 0 )
if($connected = 0)
ControlCommand("Script Manager", ScriptStart, "CheckConnection")
endif
$CPUTemp = GetCPUTemp()
if($CPUTemp > 30)
ControlCommand("Sound Servo", PauseOn)
sayezb("The temp of the process has exceeded safe parameters")
endif
$CurrentVoltage = GetVoltage()
if ($CurrentVoltage < 6.6)
ControlCommand("Sound Servo", PauseOn)
sayezb("You will have to charge me soon")
endif
sleep(5000)
Goto(MonitorFunction)



This script first checks to see if there is a connection to the EZ-B on the robot. If one doesn't exist, it then runs a script called CheckConnection. It then checks the temp of the CPU in the robot and warns if there appears to be a pending issue. It then checks the voltage of the battery and warns the user if they will need to charge the battery soon. It then waits 5 seconds and the loops to the start of the script again. This monitor keeps running while EZ-Builder is connected to the robot.

CHECKCONNECTION

Code:


$connected = IsConnected( 0 )
REPEATUNTIL($connected = 1)
If($connected = 0)
$Connection=0
Else
$Connection = 1
EndIf
if($connection = 0)
ControlCommand("Connection", Connect0)
sleep(3000)
$connected = IsConnected( 0 )
endif
EndRepeatUntil



This script is called by the monitor script and first checks to see if the connection is made to the robot. If there is not a connection, it tries to establish the connection and waits 3 seconds for the connection to be established. If it isn't established, the script tries again.

ROBOREALM.BAT

Code:


D:
CD\Program Files (x86)\RoboRealm
RoboRealm


This just simply executes the RoboRealm application.

WhoAmI

Code:


SayEZB("Let me have a good look at you.")
controlcommand("Script Manager", scriptStart, "GreetuserRich")



This is a very simple script that just says "Let me have a good look at you." and then launches the GreetUserRich. This could be combined with the GreetUserRich script and this one could be eliminated completely.

GREETUSERRICH

Code:


$user0 = "null"
IF ( $RR_NV_ARR_OBJ_NAME[0] != "" )
$user0 = $RR_NV_ARR_OBJ_NAME[0]
SayEZB("You are " + $user0 + ". It is great to see you!")
EndIf



This script was greatly reduced from what I had before. RoboRealm is very fast and very good at facial recognition. Because of this, I was able to reduce the amount of code here for checking things. If it becomes unreliable with many faces learned in RoboRealm, I will start adding in more checking. When RoboRealm is called, and a face is recognized from the HTTP camera feed from EZ-Robot, the $RR_NV_ARR_OBJ_NAME[0] is passed back to EZ-Builder as a variable. This script takes the value of this variable and assigns it to $User0 and then says "You are [the persons name]. It is great to see you!"

GETDATABASEINFO

Code:


#Starting variables for EZ-DB
$UserName = ""
$UserName = $user0
$SqlReturn = " "
$SqlCommand = ""
$FatherName = ""
$MotherName = ""
$GrandmaName = ""
$GrandpaName = ""
$NumberOfVariablesToStore = 2
$RobotName = "Teddy"
$RobotID = ""
ControlCommand("Script Manager", ScriptStart, "GetRobotIDFromDatabase")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetGrandpaFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetGrandmaFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetFatherFromDB")
Sleep(1000)
ControlCommand("Script Manager", ScriptStart, "GetMotherFromDB")
Sleep(1000)



A lot of variables are initialized here, and the $user0 variable value is then assigned to $UserName. The $RobotName variable is also set. This script then calls 5 other scripts that are all similar. All of these connect to a database through EZ-DBMulti to pull specific data back with 4 of them pulling back specific data about the current user based on the facial recognition that has taken place.

GETROBOTIDFROMDATABASE

Code:


$SqlCommand = "EXEC usp_GetRobotIDFromName '"+$RobotName+"'"

FileWrite("C:\EZ-DBMulti\Query.txt",$SqlCommand)
FileWrite("C:\EZ-DBMulti\Robot.txt",$RobotName)

EXEC("C:\EZ-DBMulti\EZ-DBMulti.exe", "In")
Sleep(1000)
$RobotID = $SQLReturn


This script prepares a sql command which executes a stored procedure on a MSSQL database called usp_GetRobotIDFromName and passes in the $RobotName variable value into the stored procedure as a parameter. It stores this information in 2 files. The first of these is called Query.Text (containing the command that is going to execute. The second is called Robot and contains the $RobotName value (really not used by this stored procedure, but EZ-DBMulti's database houses information for multiple robots.) It then calls EZ-DBMulti passing in whether EZ-DBMulti is supposed to retrieve data from the database (In) or pass data to be stored in the database (Out). After the query executes, the result from the execution of the stored procedure in the database is stored in the $SQLReturn variable in EZ-Builder so I then assign the $RobotID variable value to the value of the $SQLReturn variable.

SQL Server
usp_GetRobotIDFromName

Code:


USE [robot]
GO
/****** Object: StoredProcedure [dbo].[usp_GetRobotIDFromName] Script Date: 1/26/2015 8:00:51 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2015
-- Description: Get Robot ID from its name
-- =============================================
ALTER PROCEDURE [dbo].[usp_GetRobotIDFromName]
-- Add the parameters for the stored procedure here
(@RobotName VARCHAR(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;

SELECT
RobotID
FROM
Robots
WHERE
RobotName = @RobotName
END



This one is really unspectatular, but I wanted to show what the USP is doing. It simply is looking into the Robots table, and is returning the RobotID where the RobotName matches the $RobotName variable value from EZ-Builder (it was passed in when the USP was called through EZ-DBMulti.)

SQL SERVER
USP_GRANDPADATA

Code:


USE [robot]
GO
/****** Object: StoredProcedure [dbo].[usp_GrandpaData] Script Date: 1/26/2015 8:03:57 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2014
-- Description: Returns Grandfathers data for a named user
-- =============================================
ALTER PROCEDURE [dbo].[usp_GrandpaData]
(@username Varchar(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
GParentDetails.[UserFamilyName]
FROM
[dbo].[Users]
INNER JOIN [dbo].[UserInfo] ON [dbo].[Users].UserID = UserInfo.UserID
INNER JOIN [dbo].[FamilyTree] ON [dbo].[Users].UserID = [dbo].[FamilyTree].[UserID]
INNER JOIN [dbo].[Family] ON [dbo].[FamilyTree].[FamilyID] = [dbo].[Family].[FamilyID]
INNER JOIN [dbo].[FamilyTree] as Parents on
[dbo].[FamilyTree].[FamilyID] = Parents.FamilyID
AND FamilyTree.Parent = Parents.UserID
INNER JOIN [dbo].[FamilyTree] as GParents on
[dbo].[FamilyTree].[FamilyID] = GParents.FamilyID
AND Parents.Parent = GParents.UserID
INNER JOIN dbo.Users as GParentDetails ON
GParents.UserID = GParentDetails.UserID
INNER JOIN dbo.UserInfo AS GParentDetail ON
GParents.UserID = GParentDetail.UserID
AND GParentDetail.Gender = 'M'
WHERE dbo.Users.Username = @username
Group By
GParentDetails.[UserFamilyName]
END


This USP is a bit more complex. Gets the UserFamilyName for the male grandparent of the user. The joins are used to map the relationship from the user up to the parent and then the grandparent of the user.

SQL SERVER
USP_GETMOTHERDATA

Code:


USE [robot]
GO
/****** Object: StoredProcedure [dbo].[usp_MotherData] Script Date: 1/26/2015 8:07:11 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author: David Cochran
-- Create date: 01/25/2014
-- Description: Returns Mother data for a named user
-- =============================================
ALTER PROCEDURE [dbo].[usp_MotherData]
(@username Varchar(50))
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
CASE when DATEDIFF(YYYY,[dbo].[UserInfo].[Birthday],GETDATE()) < 6
THEN
ParentDetails.[UserFamilyName]
ELSE
ParentDetails.[UserName]
END AS ParentName
FROM
[dbo].[Users]
INNER JOIN [dbo].[UserInfo] ON [dbo].[Users].UserID = UserInfo.UserID
INNER JOIN [dbo].[FamilyTree] ON [dbo].[Users].UserID = [dbo].[FamilyTree].[UserID]
INNER JOIN [dbo].[Family] ON [dbo].[FamilyTree].[FamilyID] = [dbo].[Family].[FamilyID]
INNER JOIN [dbo].[FamilyTree] as Parents on
[dbo].[FamilyTree].[FamilyID] = Parents.FamilyID
AND FamilyTree.Parent = Parents.UserID
INNER JOIN dbo.Users as ParentDetails ON
Parents.UserID = ParentDetails.UserID
INNER JOIN dbo.UserInfo AS ParentDetail ON
Parents.UserID = ParentDetail.UserID
AND ParentDetail.Gender = 'F'
WHERE dbo.Users.Username = @username

END



This one is similar to the previous one in that it identifies the female parent of the current user. It also uses a CASE statement that checks to see what the age of the current user is. If the current user is less than 6 years old, it returns a family name (such as Mommy) for this parent. If the user is older than 6, it grabs the username for the mother. This can be extended much further than I currently have, but currently, when the user of this robot reaches age 6, the value returned changes and the robot automatically changes because of the change in age of the user.


I will be adding additional things like getting the birthday of the user or anniversary, or holidays such as Christmas, Easter and many more so that different phrases can be said based on the age of the user and the day of the year. I also plan to add things like the last time that something happened (for example a nursery rhyme was said) for a user and limit the robots activity based on this. It could be a song was played, or news was read, or whatever. I might also have a game for my granddaughter and knowing what questions were asked last time or previously, I could then ask other questions that were not asked yet or have been asked less frequently. Anyway, I will be posting the project file on the cloud if anyone wants to use anything that I have done.

There is more in there, I am just worn out on typing this right now. I will probably add more and a video later, but I wanted to start documenting. If you are still reading this and have any questions, please let me know.

#2

I'm gonna read all your posts. All I want to do is possibly mount a talking head on top of a Roomba with an attitude.

#3

Here is a video of the progress so far.

#4

Very impressive. Is Roborealm's recognition significantly better than what DJ built into object recognition in the camera control (which I realize is still Beta)? EZ-B seems kind if hit and miss with object recognition. Rather than different people, I am trying to recognize the cat, and some of her toys and react accordingly (if cat, say her name, turn on laser mounted on the camera pan/tilt and start a "play pattern", if toy, pick it up and either throw it if cat recently seen, put in storage to bring to me if cat not recently seen). I can recognize the cat if her face is in just the right positions, and toy recognition is barely working. If Roborealm would do better, it might be worth the investment rather than waiting for DJ to improve built in recognition.
(I still need to upgrade my server from 2003 to 2008 and get SQL Server installed too, but one thing at a time).

Alan

#5

Very nice David.... Wish I had your Windows programing skills dude... Anyway, I was just wondering if there is any chance you can do a tutorial of some sort on how to pass simple values or information back and forth from Roborealm and ez builder?... I mean how the roborealm works and how it and ez builder would interact with each other....

I have been tempted in the past to buy roborealm but because of not quite understanding how it worked and how to interface it with my robotics projects kept my credit card in my pocket....

#6

RoboRealm is impressively fast at object recognition. The issue that I see you coming into is that the cat can be in a lot of shapes (curled up, sitting, lying down, about to pounce, standing, just the face, just the back end) so you would have to train it in for a lot of situations. RoboRealm is really good though and I have been very impressed. You will also have to get the AVM module which allows the object recognition I use, but its really good.

My logic is to use the best thing that I can find for a job. Its the reason that I had a cable company for TV and internet and a phone company for my phones. Its becoming more blurred in that area, but It still works in robotics. The EZ-B and EZ-Builder is the best I could find. RoboRealm (now that it has a module to pass parameters to and from EZ-Builder) is the best I could find. The sound servo board is the best I could find for the size I needed (I don't have it yet, but reviews and videos are really good). There is also the comfort factor as to already knowing C# and MSSQL so it is the best for me.

One other thing that I do is try to section my code out. One script, usp, function does one thing. I find it easier to test and know it is good instead of writing long scripts or code that does many things, and then you change one thing that you think won't affect other things, but because it is in one large script, you don't know or forgot you used that variable or parameter somewhere else, and the script starts failing.

Back to the question, yes, I believe RoboRealm is worth the cost and it can be pretty seamlessly integrated into EZ-Builder. Its really good.

#7

I will do it richard and it will be very short because it is very easy.

#8

Ok, awesome David.... Smile

#9

Video is uploading. I threw myself for a loop at the end of the video and don't feel like editing it, but it will be up after it loads and is processed. I will do another one later showing me adding the components in RoboRealm.

I just had to chuckle. Hopefully you will too Smile

#10



I will make a new thread with this when I have a video that is good enough to post as a tutorial Smile