BELA
its my current work... cant tell you more than that right now ;)
BEhavior is a foreign LAnguage for robots
Environment Map
Autonomous Policy Execution
Left: something was wrong with this model. I forget. Even so, the grasping of the broom is very hard since it is balanced on a point. So the gripper must be very precise to succeed.
Right: The model I trained can generalize to the task. With greater than 90% success rate.
Right: The model I trained can generalize to the task. With greater than 90% success rate.
Please shoot an email if you have any beans that you need swept up or if you would like this system deployed in your home.
Hand Teleoperation and 2D Tracking
Left: Hand teleoperation using RUKA hand. This was my first attempt so I had to do some tricks to get the tracking to translate well to the robot coordinate space.
Right: 3D tracking of hand keypoints using HAMER (displayed in 2D). This is used as input to the teleoperation system.
Right: 3D tracking of hand keypoints using HAMER (displayed in 2D). This is used as input to the teleoperation system.
RUKA
RUKA hand calibration. TODO cite (you can google it for now)
APRIL
This is 30mm april grid.
[1].
more calibration and tracking
using point clouds!
aparently roboreg has a better method of extrinsic calibration than using april tags… i am trying that since ive wasted lots of time trying to get the right samples for eye-in-hand (EIH) nad eye-on-base (EOB) calibration.
I can verify that the calibration is decent because the rendered virtual robot aligns well with the RGB image.
It could still use some tuning, but is broadly correct in all positions.
