Rubik's Cube Solving Robot

bhouston

Canada
Skip to comments

I found this robot on Thingiverse; www.thingiverse.com/thing:2471044 and thought it would be a fun project. Check it out they've really created a great robot. Printing it takes about 70 hrs. and then you need to go shopping for baking supplies (Raspberry Pi's and such). So I thought why not make it run with EZ-Robot hardware (most people here have lots of that) and software. I modified the camera holder to accept an EZ-Robot camera and the arms to accept EZ-Robot HDD servo horns and connected everything to an IoTiny. Building it was pretty straight forward.
Programing it to solve the cube was another matter, so I got ahold of forum member ptp and asked if he would be interested in helping out and he was. He doesn't have a 3D printer however so I built him the robot and sent it to him. He has been busy working on an EZ-Robot plug-in to solve the cube as well as calibrate the arms and grippers.
We are hoping to have the Plug-in ready to share by the end of the month, so start printing. This would be a fun project for both kids and adults.
We'll keep you posted.

By — Last update
PRO
USA
#16  
@DJ,
The detection is done for each square.

I'm using HSV colors:

User-inserted image


User-inserted image


Hue is 360/2 = 0..179 (byte)

Challenges:
1) detect red-orange-yellow: They are very close e.g. The distance between Orange and Red and Yellow is only 7.5 points.

2) White
There is no white on the wheel, so need to look for saturation and value too, that is the reason why Blue is being recognized as White.

3) Middle cube with the brand sticker.
Is not only a color issue but a contour issue too... What a pin in the neck.

I didn't tried combining the HSL model:
User-inserted image


Regarding the Angles:
Assuming 0 is the middle, i need to detect the perspective, the problem is left or right, i need to review the calculation, I'm using an average angle, but after a specific rotation (cube corners (TL-BR) swap direction) and the global perspective is affected.

Quote:


The good news is that the camera and cube would always be the same angle in the real version - unlike holding it with your hand at random angles. So lighting would be more consistent.


Correct I'm trying the worst scenario.

Regarding the colors the problem is not finding 6 different colors the main problem is calling them a name.

I saw some internet rubik's cube videos, and they scan the 6 faces and the color clustering is done after, so is more easy to divide to color 1-6 (color distance) and present a BGR square color in the screen versus identifying the color.

I believe i'll handle both approaches:

A) Cube Robot: Color Distances (A-B-C-D-E-F) for solving.

B) Free movements, Color Recognition. If i can't find an automatic model, the user will need to adjust the intervals.

This is like magic... If you solve the cube without naming the colors, you have the idea the Robot recognizes identifies the color.
PRO
USA
#17  
@Andy,

The Inmoov project is almost a lifetime project, I still have some questions:
1) Finger control & usability
2) Legs are for walking ?
#18  
I know this is a challenging project. Way over my head if I had tried to conceive it. However I looked at your coding to try to learn something. You make it look so easy. What a wonderful process. :)
PRO
USA
#19  
@Dave,

Thanks for the words.

The idea is to make others challenges more EZ:)

I believe your are talking about the EZ-script:

Code:


:loop
WaitForChange($CubeVisionSequenceCounter)
SayWait("I see ")
REPEAT($x,0,8,1)
$cell=$CubeVisionColors[$x]

if ($cell=1)
$color = "Blue"
ElseIf ($cell=2)
$color = "Red"
ElseIf ($cell=3)
$color = "Orange"
ElseIf ($cell=4)
$color="Yellow"
ElseIf ($cell=5)
$color="Green"
ElseIf ($cell=6)
$color="White"
endif

SayWait($color)
ENDREPEAT
goto(loop)


i used the KISS principle (https://en.wikipedia.org/wiki/KISS_principle):)

The $CubeVisionSequenceCounter is incremented every time a complete different sequence is found.

So the idea is to monitor that variable and do something after.

@DJ:
I couldn't find a blocky equivalent for WaitForChange
PRO
USA
#20  
I always tell my friends Robots is more software than hardware.

There are so many sensors, but what makes them unique is the software (Firmware , API, Framework or Program).

One good example is the 3d camera:
1) PrimeSense/ Microsoft Kinect1/ Asus (1st generation) (Gone)
2) Microsoft Kinect2 (Will be gone after 2018)
3) Intel Realsense Cameras
4) Orbbec Cameras
5) Google Tango

All of them have more or less the same sensors, but... only 1 st generation and Microsoft Kinect V2 handles skeleton tracking.

So the software is the real thing.
#21  
KISS, yes. This's a concept I struggle with. Most of the time I overbuild and over think. It always works out better when I later reverse engineer and remove all the complexity I worked so hard implementing. LOL *eek*

EZ Robot and the rest of us are fortunate the group of guys like you have chosen to stick around here and show us amateurs the path. ;)

Respect. ;)
PRO
Synthiam
#22  
Use either one of these, and they can include white. They have a histogram option, which may provide more consistency in varying lighting conditions.

1) Simply create a profile for each color, adjust the sliders to isolate the color and give it a name.
2) The camera control can be configured to specify a maximum number of objects to detect (in this case 9)
3) Your plugin can watch the variables and see the location and color of each variable.
4) Sort the variables by X ascending and Y ascending using linq

User-inserted image
PRO
Synthiam
#23  
Voila, here you go. Once you create a profile for each color and give them a name, this will sort them for you...

Code:


class Square {

public enum ColorListEnum {
Red,
Green,
Blue,
White
}

public ColorListEnum Color;
public int X;
public int Y;

public Square(string color, int x, int y) {

if (color == "red")
Color = ColorListEnum.Red;
else if (color == "green")
Color = ColorListEnum.Green;
else if (color == "blue")
Color = ColorListEnum.Blue;
else if (color == "white")
Color = ColorListEnum.White;
else
throw new Exception(string.Format("Unknown color {0}", color));

X = x;
Y = y;
}
}

Square[] GetCubeColors() {

if (EZ_Builder.Scripting.VariableManager.GetVariable("$CameraIsTracking") != "1")
throw new Exception("Camera not detected any objects");

int count = Convert.ToInt16(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectCount"));

if (count != 9)
throw new Exception(string.Format("Camera only detected {0}/9 squares", count));

// Get all the variables and put them into our class of colors with X and Y
Square[] squareList = new Square[9];
for (int i = 0; i < count; i++) {

string colorName;
int x;
int y;

if (i == 0) {

colorName = EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectColor");
x = Convert.ToInt16(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectX"));
y = Convert.ToInt16(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectY"));
} else {

colorName = EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectColor_" + i.ToString());
x = Convert.ToInt16(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectX_" + i.ToString()));
y = Convert.ToInt16(EZ_Builder.Scripting.VariableManager.GetVariable("$CameraObjectY_" + i.ToString()));
}

squareList[i] = new Square(colorName, x, y);
}

// Sort the list of colors by X ascending and Y ascendinng
var sorted = from o in squareList orderby o.X ascending, o.Y ascending select o;

return sorted.ToArray();
}
PRO
Synthiam
#24  
One last thing, you can also do this entirely inside the plugin, rather than depending on people creating Color Profiles. If you'd like to see how, let me know and i can show you the EZ_B.CameraDetection namespace.

Simply do something like this...

Code:


EZ_Builder.UCForms.FormCameraDevice _cameraControl;

void detach() {

if (_cameraControl != null) {

EZ_Builder.EZBManager.Log("Detaching from {0}", _cameraControl.Text);

_cameraControl.Camera.OnNewFrame -= Camera_OnNewFrame;

_cameraControl = null;
}

Invokers.SetEnabled(button3, true);
Invokers.SetText(button3, "Attach Camera");
}

void attach() {

detach();

Control[] cameras = EZ_Builder.EZBManager.FormMain.GetControlByType(typeof(EZ_Builder.UCForms.FormCameraDevice));

if (cameras.Length == 0) {

MessageBox.Show("There are no camera controls in this project.");

return;
}

foreach (EZ_Builder.UCForms.FormCameraDevice camera in cameras)
if (camera.Camera.IsActive) {

_cameraControl = camera;

_cameraControl.Camera.OnNewFrame += Camera_OnNewFrame;

EZ_Builder.EZBManager.Log("Attached to: {0}", _cameraControl.Text);

Invokers.SetEnabled(button3, true);

Invokers.SetText(button3, "Detach Camera");

return;
}

MessageBox.Show("There are no active cameras in this project. This control will connect to the first active camera that it detects in the project");
}

class Square {

public enum ColorListEnum {
Red,
Green,
Blue,
White
}

public ColorListEnum Color;
public int X;
public int Y;

public Square(string color, int x, int y) {

if (color == "red")
Color = ColorListEnum.Red;
else if (color == "green")
Color = ColorListEnum.Green;
else if (color == "blue")
Color = ColorListEnum.Blue;
else if (color == "white")
Color = ColorListEnum.White;
else
throw new Exception(string.Format("Unknown color {0}", color));

X = x;
Y = y;
}
}

Square [] CubeColors;

// Set up your own color definition profiles or let the user adjust them
EZ_B.Classes.CustomColorConfig [] _colorConfigs = new EZ_B.Classes.CustomColorConfig[] {
new EZ_B.Classes.CustomColorConfig("red",true, 10, 10, 50, 0.2f, 0.2f, 0.1f, 0.9f),
new EZ_B.Classes.CustomColorConfig("green",true, 10, 10, 50, 0.2f, 0.2f, 0.1f, 0.9f),
new EZ_B.Classes.CustomColorConfig("white",true, 10, 10, 50, 0.2f, 0.2f, 0.1f, 0.9f),
new EZ_B.Classes.CustomColorConfig("blue",true, 10, 10, 50, 0.2f, 0.2f, 0.1f, 0.9f)
};

void Camera_OnNewFrame() {

if (_isClosing)
return;

List<EZ_B.ObjectLocation> detectedObjects = new List<EZ_B.ObjectLocation>();

foreach (var colorConfig in _colorConfigs)
detectedObjects.AddRange(_cameraControl.Camera.CameraCustomColorDetection.GetObjectLocationByColor(true, colorConfig));

if (detectedObjects.Count != 9)
throw new Exception(string.Format("Camera only detected {0}/9 squares", detectedObjects.Count));

// Get all the variables and put them into our class of colors with X and Y
Square[] squareList = new Square[9];
for (int i = 0; i < detectedObjects.Count; i++) {

var detectedObject = detectedObjects[i];

squareList[i] = new Square(detectedObject.ColorName, detectedObject.CenterX, detectedObject.CenterY);
}

// Sort the list of colors by X ascending and Y ascending and assign to a global variable for using elsewhere
CubeColors = (from o in squareList orderby o.X ascending, o.Y ascending select o).ToArray();
}
PRO
USA
#25  
@DJ,

Thanks, the information is useful.

I didn't want to reinvent the wheel... I had idea to explore the color profiles so we are on the same page:)

btw I'm already using the camera guidelines to reduce the ROI area.
PRO
Synthiam
#26  
Oh, almost forgot... to edit the color filters for each color, this is how you do it...

Code:


EZ_B.Classes.CustomColorConfig GetColor(EZ_B.Classes.CustomColorConfig color) {

string colorName = color.ColorName;

using (EZ_Builder.UCForms.FormCameraCustomColor fc = new EZ_Builder.UCForms.FormCameraCustomColor(_cameraControl.Camera)) {

fc.Configuration = color;

if (fc.ShowDialog() == DialogResult.OK) {

// set the color name to the original incase the user changed it
fc.Configuration.ColorName = colorName;

return fc.Configuration;
}

return color;
}
}
PRO
USA
#27  
Guys,

The plugin is 99% done.

Demo:


PS: Sound level is low.
PRO
Synthiam
#28  
Woooo! Now THAT is a plugin. Maaaaan what a great implantation. You have blown me away man
#29  
@ptp,

Great Job As Always...., You are unbelievable, and you have surpassed yourself, AGAIN....

Ron
#30  
Ya, looks great, Pedro. I'll be playing with it today.
PRO
USA
#31  
@DJ/All,
Thanks for the incentive.

The EZ-Robot HDD servos are a must for this project.

Initially i used only 4, after a few minutes i changed my mind .. and i borrowed (no way back) the other 4 from another project. The robot/machine is near my monitor and i do the code during the nights, i could not concentrate with the analog servos buzzing my ears.

All,
Thanks for watching, It's a cool robot to demo...

I'm involved in a few meetups so is a good opportunity to talk about robots and discuss bored stuff e.g. C#, C++, OpenCV.

is amazing what you can build with a 3D Printer, 8 servos, camera and wifi controller and ARC!
#32  
ptp, I can't expand the control window for the plugin to see all of the controls. Any thoughts?
User-inserted image
PRO
USA
#33  
Last minute change ... Fixed! please update the plugin.
I'm on Skype if you need help.
PRO
Canada
#34  
Wow! This is amazing. Great work PTP