In Pittsburgh, an innovative endeavor is reshaping the landscape of independence for individuals with disabilities. Spearheading this transformation is the Human Engineering Research Laboratories (HERL) at the University of Pittsburgh. With a substantial $41.5 million in backing from the Advanced Research Projects Agency for Health (ARPA-H), HERL is establishing a next-level robotic platform named RAMMP to enhance assistive mobility through advanced technology.
The RAMMP (Robotic Assistive Mobility and Manipulation Platform) project seeks to revolutionize existing mobility solutions. Focused on bridging the gap between technology and real-world applications, the initiative is incorporating cutting-edge AI alongside robotics and user-centered design. This drive is crucial for the 5.5 million wheelchair users in the U.S., who face over 100,000 injuries annually due to inadequate mobility solutions. The integration of Meta’s AI models, including DINO and SAM, plays a crucial role in addressing these challenges.
These AI models, DINO and SAM, are already recognized for versatility across various domains. DINO, a self-supervised vision transformer, skillfully learns from unlabelled data, while SAM excels in precise object recognition. This capability is harnessed in RAMMP to ensure that assistive devices can navigate real-world challenges reliably and safely.
HERL’s endeavor extends to engineering solutions for deploying these AI models on compact edge devices. By processing data on-device, these assistive technologies ensure immediate responsiveness to environmental stimuli, crucial for real-world navigation. Engineers face the challenge of optimizing these models for operation on limited hardware, balancing performance with factors like battery life and real-time interaction.
The RAMMP prototype already demonstrates success in real-world settings by leveraging DINO to assist users in identifying objects like cups and navigating obstacles. This enhances direct interaction through voice and touch inputs, further reducing cognitive load for users. RAMMP employs RF-DETR, a light detection model refined with DINOv2 embeddings, showcasing efficiency in auto-labeling training data, thus optimizing for a wide range of environments and conditions.
The partnership between HERL and ATDev exemplifies collaboration in innovation. HERL contributes its expertise in engineering and user-focused research, while ATDev translates these insights into practical devices. Their efforts engage a broad consortium that includes esteemed institutions like Carnegie Mellon, Northeastern, and industry partners, all marching toward a shared goal of advanced mobility solutions.
ARPA-H’s vision through RAMMP goes beyond individual mobility improvements; it aspires to set a precedent in U.S. health innovation. This project not only elevates the mobility of millions but also instigates economic opportunities within Pittsburgh and Pennsylvania, exemplifying technology’s role in societal advancement.
Looking forward, the RAMMP team is committed to refining AI perception models for enhanced environmental interaction. New iterations like SAM 3.1 and DINOv3 will focus on adaptive learning tailored to each user, ensuring a future where technology facilitates complete and independent participation in life.
You can read the original article here: https://ai.meta.com/blog/assistive-robotics-university-of-pittsburgh-sam-dino/
