Bimanual Teleoperation
PhD project - University of California, Los Angeles
Project Description
The Office of Naval Research funded a multi-institutional collaboration between UCLA, Case Western Reserve University, and Cleveland State University. The goal of the project and collaboration is to create a teleoperated bimanual robotic system which can be operated by a user in virtual reality and across large physical distances. The system (seen in CAD to the right) consists of a dominant arm - a 7-DOF Franka Emika FR3 manipulator outfitted with a multi-digit LEAP hand - and a non dominant 6-DOF Universal Robots UR5e manipulator outfitted with a Robotiq 3-finger gripper.
As the Controls Team Lead, I lead the development and testing of closed-loop robot controllers that accept human inputs from a VR headset, command robot joint velocities, and provide multimodal feedback, including the sense of touch.
Design Requirements
The controller had to meet a number of requirements, including:
- Avoid kinematic singularities
- Maximize manipulability of the end effector
- Be capable of "palm up" and "palm down" hand configurations
- Be fast enough to keep up with human movements
The controller I developed is an optimization-based controller that maximizes manipulability and minimizes joint velocities within the kinematic limits of the arm.
Take a look at the demo below at 2X speed! The video on the left is an external video feed of the robotic system, the top right video is the operator view in VR and the bottom left video is an external view of the operator performing a task with the system.