Robot Pen Thief
ProjectAs a part of my orientation before beginning classes at Northwestern, I particpated in some coding challenges. The goal of this specific project was to use a PincherX 100 robot to automatically grab a pen. To do this, I used sensor data from an Intel Realsense D435i camera and processed it in OpenCV. Since the pen I was using was purple, I performed color masking on the RGB data to create a contour of the pen. Once I located the center of the pen, I examined the depth map data from the Realsense to determine the pen's physical location in the camera frame.
My pen tracking algorithm relies on color masking in OpenCV
Once the location of the pen had been confirmed, the next step was to move the robot to that location and grab it. The first step in this procedure was to apply a transformation between the camera frame and the robot frame, which I achieved via a calibration run for a set camera and robot position. I then used the InterbotixManipulatorXS Python package to command the arm to the desired end effector position of the pen location. The gripper will grasp the pen, return to the home position, and drop it.
