Avatar JD is a Virtual 3D version of JD Humanoid. You can move this Avatar JD to move your real JD Robot. Also, when you move your real robot Avatar JD will move as well. Lastly you can use Avatar JD with the EZ-B Emulator
How to add the Avatar JD robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Remote Control category tab.
- Press the Avatar JD icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Avatar JD robot skill.
How to use the Avatar JD robot skill
Avatar JD is a Virtual 3D version of JD the Humanoid Robot. You can move this Avatar JD to move your real JD Robot. Also, when you move your real robot Avatar JD will move as well(Version 9: Internal coding changes: Removed timer and used VariableManager and On servo Movement event to get servo Values and Scripting variables.)
(Version 4: Used WPF Element host for smoother animation. Added $SIM_ global variables so you can write scripts to move Avatar JD and interact with the simulation. Added a tiled Floor for him to walk on. You can now move and rotate Avatar JD or you can rotate the world he is in as well as zoom in and out.)
I posted a challenge at the bottom of these instructions for folks who would like to help write some JavaScript to allow Avatar JD walk around and keep track of his location in real space. The initial code I provided allows him to walk in a straight line but needs to be enhanced to allow him to turn and stop.
Note the video below was made with version 3 so does not explain/demo some of the new version 4 features.
Welcome to the Avatar JD Plugin
When you connect the real JD robot then Avatar JD’s Servos will all be set to 90. If you don’t have a JD see info lower down on how to connect to the EZ-B emulator.
Moving the servos:
You can move this 3D Avatar JD by clicking on a 3D body part servo and dragging the mouse to move your real JD Robot. Drag the body part to the left to decrease the servo value and to the right to increase the servo value.
When in real time update mode your Avatar JD will also move as you use the Auto Position control or run other scripts to move your real JD robot.
You can also use the keyboard to enter in the servo positions or use the + and - key. The enter key is used to save the value.
Rotate, Move or Zoom:
Click on the JD’s Chest and drag the mouse left or right to rotate, or move Avatar JD around within his 3D world. You can also rotate the 3D world to view Avatar JD from different angles. Click on the 2 sets of radio buttons ([X, Y, Z axis] and [Move, Rotate, Rotate World, Zoom]) to determine what happens when you drag on his chest. Or you can click on the specific field that lines up with both radio buttons. For example in the screen shot if you clicked on the field with -11 you are selecting Rotate World in the Z axis.
If you don’t want to use the mouse you can just type in the value in the field, or use the + / - keys to increase or decrease the value and followed by enter to set the desired position or rotation. Use Shift minus (underscore character) or * (asterix) to toggle the minus sign on and off. A decimal point can be used to fine tune the value for example if you wanted to move Avatar JD 2.5cm in the X direction.
You can show / hide the tiled floor by clicking on the Show Floor checkbox.
Real time update mode:
If you want to move multiple servo's at the same time uncheck real time update mode. Then move or set the desired servos and click on the set all Servos button.
Create action sequences:
You can also create Action movement sequences by typing in an action name. You can also optionally adjust the servo speed, number of steps and the delay before moving. You then move your Avatar JD and when in you have the desired position click on Save Frame to Action. You can also click on the Pause checkbox and it will use the delay setting to pause the robot.
To play back the action go into the Auto Position control. If the action is not listed, then click on the . . . to open the Auto Position settings and click on save. This will force the Auto Position to update the list of actions.
Using Avatar JD with the emulator:
if you don't have a JD Humanoid or your JD is charging you can open up the EZ-B Emulator and set it to EZB_v4_OS_With_Comm_1 and check the Listening checkbox.
Then in ARC connect to 127.0.0.1:23
This allows you to simulate JD the Humanoid using the EZ-B emulator. You will be able to play action sequences from the Auto Position control or create and run scripts designed for JD the Humanoid.
Keep in mind the 3D Avatar moves like you are holding your real JD Humanoid in the air by the chest. So when you walk JD Humanoid forward the 3D Avatar will not be pushing off the ground to actually move forward. It will just move the servos. Be careful when moving the legs with Avatar JD so that your real JD does not fall over.
The 3D Avatar JD is great to visualize simple movements like moving the head, neck, arms or hands. You may need to click on Rotate World Z axis field and then click on and drag JD’s chest to see all the servo positions from different angles to get a better idea of the pose or to be able to click on servos that are hidden from view. You can also hide the floor if JD’s feet are disappearing from view.
Interacting with the simulation using scripts:
With version 4 you can now interact with the simulation using scripts to modify the following global variables.
$SIM_AvJD_PositionX, Y, Z - This determines Avatar JD’s location in the simulation and are the same as the move X, Y, Z values. The values line up with the center of JD’s chest.
$SIM_AvJD_RotationX, Y, Z These variables allow you to rotate Avatar JD in the X, Y, and Z axis.
$SIM_World_RotationX, Y, Z, - Allows you to view Avatar JD from different angles, it rotates the floor as well.
$SIM_ShowFloor can be set to 0 hide the floor, or 1 show the floor.
$SIM_Zoom Allows you to zoom in and out. 50 represents 50% of the original size, 200 would be 200% original size.
$SIM_RealTimeUpdate Turns real time update mode on or off.
$SIM_RightUpperLegLoadOffset, $SIM_LeftUpperLegLoadOffset - how much the force of gravity will offset the angle from his upper leg servo. Use when both feet or legs are not level on the floor. It allows you to move the upper leg servo from the weight of JD without changing the upper leg servo values.
$SIM_RightFootLoadOffset, $SIM_LeftFootLoadOffset how much the force of gravity adjusts the foot servo angles by.
You can use these global variables to write a script to make Avatar JD walk or move similar to how he would in the real world.
Here is an example JavaScript code to have him walk in a straight line.
Code:
// Walk_Avatar_JD:
// By keeping track of what frame JD is starting and and what frame he is moving to we can calculate how he moves
// The frameTransition amounts are changes to the existing values except for the FootLoadOffset which are absolute amounts
// If you rotate world in the X axis in the Avatar JD simulation to 8.46 or -8.46 you can see the floor as a horizontal line
// When standing with both legs straight the center of JD's body is 17.5 cm above the floor.
// JD rotates around the center of his body so for every 1 degree you rotate his body in the Y axis you
// move sine(1)*17.5 = 0.305417 cm in the X axis
// and cosine(1)*17.5 = 0.0026653 cm in the Z axis
// Each tile on the floor is 10 cm by 10 cm. At x=0, y=0 JD is centered in a tile.
// When in a particular pose you can see where JD's feet are in each square then after you change the pose
// say from Stop to Walk 1 you can then determine which foot JD would be standing on use that Foot's Load
// Offset to level that foot on the floor (Unless both feet are on an angle and raising JD upwards)
// Then move JD back in the X axis to place that foot in the same location and move JD up or down in the
// Z axis so that same foot is level on the floor. Using this approach you can determine how much to rotate
// his body and how much to move him to keep the foot he is balancing in place and how much the force of gravity
// will offset the angle of each foot servo or the angle from his upper leg servo.
// The Movement of Avatar JD looks a little funny since we are not taking into account each servo transition
// to get to the ToFrame. We and are just moving him based on the frame servo position end state.
// Currently the code just handles JD walking forward, however, FromFrame, ToFrame Transitions could be added to handle
// turning left, right, backwards as well as stop from any Walk 1 to Walk 8 postions.
// This would keep track of JD's X, Y, Z position while he moves around in the simulation or the real world.
var frameTransition = [
// FromFrame, ToFrame, RotateX, RotateY, RotateZ, MoveX, MoveY, MoveZ, RightUpperLegLoadOffset, LeftUpperLegLoadOffset, RightFootLoadOffset, LeftFootLoadOffset
["Stop", "Walk 1", 0, -10, 0 , -3 , 0 , 0.1 , 0 , 0, 5, 0],
["Walk 1", "Walk 2",-4, 0 , 0 , 0 ,-1.3 ,-0.2 , 4 , 0, 5, 0],
["Walk 2", "Walk 3", 6, 10 , -5, 3 , -1.5, -0.4,-11,-6, 0, 1],
["Walk 3", "Walk 4", 0, 10 , 0 , 2.8,-1.1 , 0.5 ,-2 ,0 , 0,-4],
["Walk 4", "Walk 5", 0, -3 , 0, -1 ,-1 , 0 ,0 ,0 , 0,-8],
["Walk 5", "Walk 6", 0, 0 , 0 , 0.2,-1.1 ,-0.6 ,0 ,-3, 0,-8],
["Walk 6", "Walk 7",-8, -2 , 3 ,-0.7,-0.4 ,-0.1 ,8 ,5 , 4, 4],
["Walk 7", "Walk 8", 6, -15, 1 ,-4.3,-0.8 , 0.8 ,-5 ,5 , 5,-1],
["Walk 8", "Walk 1", 0, 0 , 1 ,0 ,-0.8 , 0 ,0 ,0 , 5, 0]
];
var len = frameTransition.length;
// Constant to convert from degress to radians to be used to calculate the x and y distance
// depending on what angle $SIM_AvJD_RotationZ is at.
var degreesToRadians = Math.PI / 180;
// Get the current frame
var fromFrame = getVar("$AutoPositionFrame");
var workToDo = true;
var toFrame = "";
// Loop until the "STAND" frame, or the users stops the script, and check the from and to Frame combinations
// Once a match is found rotate and move Avitar JD
while (workToDo) {
// Get the current frame
toFrame = getVar("$AutoPositionFrame");
// If the Frame changed
if (fromFrame != toFrame) {
print("Frame changed from " + fromFrame + " to " + toFrame);
//Find Matching From and To Frame
for (var i = 0; i < len; i++) {
if (frameTransition[ i ][0] == fromFrame && frameTransition[ i ][1] == toFrame) {
// Rotate X
setVar("$SIM_AvJD_RotationX",getVar("$SIM_AvJD_RotationX") + frameTransition[ i ][2]);
// Rotate Y
setVar("$SIM_AvJD_RotationY",getVar("$SIM_AvJD_RotationY") + frameTransition[ i ][3]);
// Rotate Z
setVar("$SIM_AvJD_RotationZ",getVar("$SIM_AvJD_RotationZ") + frameTransition[ i ][4]);
// Move X
setVar("$SIM_AvJD_PositionX",getVar("$SIM_AvJD_PositionX") +
frameTransition[ i ][5] * Math.cos(degreesToRadians * getVar("$SIM_AvJD_RotationZ")));
setVar("$SIM_AvJD_PositionY",getVar("$SIM_AvJD_PositionY") +
frameTransition[ i ][5] * Math.sin(degreesToRadians * getVar("$SIM_AvJD_RotationZ")));
// Move Y
setVar("$SIM_AvJD_PositionY",getVar("$SIM_AvJD_PositionY") +
frameTransition[ i ][6] * Math.cos(degreesToRadians * getVar("$SIM_AvJD_RotationZ")));
setVar("$SIM_AvJD_PositionX",getVar("$SIM_AvJD_PositionX") -
frameTransition[ i ][6] * Math.sin(degreesToRadians * getVar("$SIM_AvJD_RotationZ")));
// Move Z - Assumes JD is walking on a flat level surface.
setVar("$SIM_AvJD_PositionZ",getVar("$SIM_AvJD_PositionZ") + frameTransition[i][7]);
// Load Offset values are absolute and not relative changes.
// Right Uppler Leg Load Offset
setVar("$SIM_RightUpperLegLoadOffset",frameTransition[ i ][8]);
// Left Uppler Leg Load Offset
setVar("$SIM_LeftUpperLegLoadOffset",frameTransition[ i ][9]);
// Right Foot Load Offset
setVar("$SIM_RightFootLoadOffset",frameTransition[ i ][10]);
// Left Foot Load Offset
setVar("$SIM_LeftFootLoadOffset",frameTransition[ i ][11]);
break;
}
}
// Finished processing the changed frame, and set the fromFrame as the toFrame
fromFrame = toFrame;
}
// stop looking if STAND frame is set. You can also just stop the script.
if (toFrame == "STAND") {
workToDo = false;
}
} // End While loop
Challenge:
Can you update the code to allow him to turn, walk backwards or stop? Hint: look at the first action frame for turn left and then assume you could be on any of the walk1 to walk8 or stop frames to start.
Once complete this code could also be used to keep track of the X, Y coordinates of where the real JD is when he walks around.
Please paste your FrameTransition solutions in the comments below so you can share with others.
---------------------------
Error loading plugin
---------------------------
Does this plugin require an update? Ensure this plugin is updated from the Synthiam Skill Store and try again.
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.TypeLoadException: Could not load type 'ARC.UCForms.FormMasterTemplate' from assembly 'ARC, Version=2015.8.9.0, Culture=neutral, PublicKeyToken=c3a3457c97d352d9'.
at csEZBuilderTest.MainForm.InitializeComponent()
at csEZBuilderTest.MainForm..ctor()
--- End of inner exception stack trace ---
at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck)
at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
at System.Activator.CreateInstance(Type type, Boolean nonPublic)
at System.Activator.CreateInstance(Type type)
at ARC.FormAddControl.d__7.GhSyH8BPDkFj0VChHSD7(Type )
at ARC.FormAddControl.d__7.MoveNext()
---------------------------
OK
---------------------------
Any help is very much appreciated
Thank you,
When I close the program it says:
Version: 2020.11.15.00
System.Exception: Variable not defined: $SIM_World_RotationX
at ARC.Scripting.VariableManager.GetVariable(String variableName)
at csEZBuilderTest.MainForm.GetVariables() in C:\My Documents\SVN\Developer - Controls\By Others\JD Simulator\csEZBuilderTest\MainForm.cs:line 192
at csEZBuilderTest.MainForm.TimerTick(Object sender, EventArgs e) in C:\My Documents\SVN\Developer - Controls\By Others\JD Simulator\csEZBuilderTest\MainForm.cs:line 113
at System.Windows.Threading.DispatcherTimer.FireTick(Object unused)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.DispatcherOperation.InvokeImpl()
at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(Object state)
at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Threading.DispatcherOperation.Invoke()
at System.Windows.Threading.Dispatcher.ProcessQueue()
at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at ARC.FormMain.JjuMfGuRXnlKuY1TVOMu()
at ARC.FormMain.UpdateVirtualDesktopShots()
at ARC.FormMain.svmMDy0sx5()
at ARC.FormMain.NewProject()
at ARC.FormMain.FPju88nbV2(Object )
at ARC.UC.UCRibbonMenuItem.YurlHFuP36KlyOoaaaCb(Object , Object )
at ARC.UC.UCRibbonMenuItem.e8Xg27DVXU(Object , EventArgs )
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
Specifically, rather than using a timer - the render is updated when servos are moved. There is an event which you can bind to that will be called when servos move. I removed the timer and put the code in that event.
I also use the form closing event to remove assets and clean up event assignments, etc..
Lastly, I check if the plugin is closing with (IsClosing) and prevent stuff from running while the plugin is being removed or ARC is shutting down, etc.
1) Remove the timer
2) Add a Form Load, because that's a safer place to put the things. Make sure the form onload event is set to this
Code:
3) You'll need a form closing, so stop doing the things that it was doing. Again make sure the main form on closing event is set to call this.
Code:
4) Now your timer event can be in the On servo Move Detail event
Code:
I found a way to make it work by using the Invoke delegate. I moved the rest of the code into another function GetServoAndVariables() just to make it easier to read.
I used the tbRotateWorldX form radio button, but I could have used any of the other form field controls. Then all code in the delegate { } runs in the UI thread.Code:
I can now make the code a bit smarter since I know what servo changed or if I add ARC.Scripting.VariableManager.OnVariableChanged I can also update things when one of the variables are changed from an ARC script. eg: if you set $SIM_ShowFloor to 0 to hide the floor from an ARC script.
I will update the plug-in once I make those changes.
Code:
also - if you’re unsure if invoke is required, there’s InvokeRequired to check
If you look at the java script code in the how to use above it shows how you simulate JD to walk in a straight line. So you would have to add to the frameTransition array to show how JD would move when you transition from one frame movement to another frame movement taking gravity, friction, pushing off the floor and other things into account. You could then build in other sequences like complex dance moves. You would then have to compare what actually happens with a real JD and adjust things. Hope that helps. Happy coding if you decide to take on the challenge.