Equipment for Object Recognition and Robotic Manipulation

We propose to purchase equipment for integrated perception and robotic manipulation of objects, with one ABB YuMi dual-arm robot with integrated vision sensors, one Fetch mobile manipulator robot, and one OptiTrack motion capture system. The equipment will be used in NSF-funded research experiences for undergraduates (REU), capstone robotics courses for undergraduate students, research training for graduate students, recruitment of minority students, and outreach to K-12 students. The proposed focus of our efforts is integrated object recognition and manipulation capabilities in environments that are semi-structured and cluttered, with significant object occlusion, and even dangerous and unstable, such as in search and rescue environments after disasters (e.g., an explosion). Current research in perception and robotic manipulation has been mainly focused on isolated objects. Visual perception is usually focused on object recognition only and assumes unobstructed views of an object. Work on robotic manipulation is often focused on the target object with the underlying assumptions that it can be perceived without occlusion, and is relatively isolated from the rest of the environment. Prior research that considered cluttered environments focused on moving away obstacles to isolate the target object so that the above assumptions would still hold before the target object was manipulated. To overcome these unrealistic assumptions in real world scenarios, we propose to study object perception and manipulation where the target objects are an integral part of the environment that cannot be fully isolated in views and in physical arrangements prior to manipulation. We also propose to track humans to automatically recognize their actions, thus enabling robots to predict the intents of the humans and automatically respond with collaborative actions in these and other tasks. The proposed instrument can significantly leverage all levels of computer vision and robotics research and education as well as general STEM education at UNC Charlotte, which is a minority-serving institution. Currently we do not have the proposed state-of-the-art equipment at UNC Charlotte. The equipment will be necessary for training Masters and PhD students in the proposed research areas. The proposed research and development can only be achieved on real robots with real sensors. The equipment will be used in NSF-funded research experience for undergraduates (REU), such as at the Socially Relevant Computing Research Experiences for Undergraduates Site, a summer REU program funded by the NSFat UNC Charlotte since 2006, as well as our capstone robotics courses for undergraduate students. The equipment will also leverage our site of the NSF Industry/University Cooperative Research Center (I/UCRC) on Robots and Sensors for the Human Well-being (ROSE-HUB) and enable us to conduct related research of relevance to government and industry. The center will also enable us to seek research experience supplements from NSF for undergraduates, teachers, and veterans. We will actively cooperate with the STARS Computing Corps (STARS) at UNC Charlotte to attract students, especially those from underrepresented groups, to STEM fields. We will leverage the CCI Women in Computing Initiative to recruit female students in research training with the proposed equipment. We will also take advantage of the UNC Charlotte Center for STEM Education to reach out to PreK-16 students more broadly. Research demos on the proposed mobile robot manipulators can also be used for outreach to STEM students in our university and K-12 students. The Robotics lab of the PIs has regularly hosted visits from the CCI STARS students and K-12 students, especially high-school summer interns and high-school seniors in all Explore UNC Charlotte events. It has been a major attraction to those students. The lab has also given several presentations at a local elementary school.