Tian Tan, SEAS Masters in Electrical Engineering ’25, China
This summer, I had the opportunity to work as a research assistant in ModLab under the guidance of Professor Mark Yim. My work primarily focused on the SMORES-EP V2 project, which is centered around enabling reconfigurable robots to autonomously reconfigure themselves using vision-based methods without relying on external localization systems. SMORES-EP is a modular robot designed and built at the University of Pennsylvania, and used by researchers at Penn and Cornell. SMORES stands for Self-Assembling MOdular Robot for Extreme Shapeshifting, and EP refers to the Electro-Permanent magnets the modules use to connect.
To equip the robot with perception capabilities essential for its vision-based reconfiguration, it was necessary to integrate cameras into the system. Given the compact size of the SMORES units, I had to utilize very small cameras. I developed an ESP32-CAM ROS package to evaluate the performance of two specific cameras, the OV2640 and OV5640, in terms of frame rate, resolution, and latency when streaming video over Wi-Fi. Through comprehensive testing, I was able to determine the optimal resolution and image compression settings that would maximize frame rate while maintaining an acceptable level of visual detail. Additionally, I contributed to the project by assisting in the design of the power supply PCB and creating the CAD model for the camera mount, ensuring that the cameras were securely and efficiently integrated into the SMORES units.
Another part of my work also involved developing algorithms for vision-based control that would enable the SMORES units to autonomously reconfigure. Initially, I tested my approach to AprilTag distance detection. Given that the focal length of the calibrated camera and the actual size of the AprilTag were known, I was able to accurately determine the distance between the AprilTag and the camera. Following these tests, I proceeded by attaching AprilTags, each with a unique ID, to the various faces of the SMORES units. I then wrote the Apriltag_detection ROS package, which accurately detects the 3D coordinates and orientation of the AprilTags relative to the camera. This method allows the control algorithm to identify each SMORES unit by its unique ID and differentiate between the faces of each unit. Using this approach, I successfully achieved the connection between two SMORES units.
In addition to my technical contributions, I also gained exposure to other fascinating research areas during our weekly lab meetings. These areas included magnetorheological fluids, micro-robots, and Variable Topology Trusses (VTT), all of which broadened my understanding of cutting-edge robotics research. My interactions with the research team were incredibly enriching, both intellectually and personally. I not only formed meaningful friendships but also acquired practical skills, such as assembling a 2S lithium battery and constructing an Electro-Permanent Magnet.
Although the project is still ongoing, I will continue working on it next semester and plan to implement vision-based multi-unit reconfiguration with integrated cameras. Reflecting on this internship, I have significantly enhanced my research skills in both software and hardware, and I gained insights into cutting-edge research challenges in modular robotics. This experience has solidified my determination to pursue a career in research. I am deeply grateful to Career Services for funding my summer research and to Professor Mark Yim for offering me the opportunity to contribute to this exciting project.
This is part of a series of posts by recipients of the 2024 GAPSA Summer Internship Funding Program that is coordinated by Penn Career Services. We’ve asked funding recipients to reflect on their summer experiences and talk about the industries in which they spent their summer. You can read the entire series here.
