Add to Chrome
Add to Firefox
Add to EdgeHuaishu Peng
SHARE: Single-view Human Adversarial REconstruction
Dec 30, 2023
The accuracy of 3D Human Pose and Shape reconstruction (HPS) from an image is progressively improving. Yet, no known method is robust across all image distortion. To address issues due to variations of camera poses, we introduce SHARE, a novel fine-tuning method that utilizes adversarial data augmentation to enhance the robustness of existing HPS techniques. We perform a comprehensive analysis on the impact of camera poses on HPS reconstruction outcomes. We first generated large-scale image datasets captured systematically from diverse camera perspectives. We then established a mapping between camera poses and reconstruction errors as a continuous function that characterizes the relationship between camera poses and HPS quality. Leveraging this representation, we introduce RoME (Regions of Maximal Error), a novel sampling technique for our adversarial fine-tuning method. The SHARE framework is generalizable across various single-view HPS methods and we demonstrate its performance on HMR, SPIN, PARE, CLIFF and ExPose. Our results illustrate a reduction in mean joint errors across single-view HPS techniques, for images captured from multiple camera positions without compromising their baseline performance. In many challenging cases, our method surpasses the performance of existing models, highlighting its practical significance for diverse real-world applications.
Toucha11y: Making Inaccessible Public Touchscreens Accessible
May 06, 2023
Despite their growing popularity, many public kiosks with touchscreens are inaccessible to blind people. Toucha11y is a working prototype that allows blind users to use existing inaccessible touchscreen kiosks independently and with little effort. Toucha11y consists of a mechanical bot that can be instrumented to an arbitrary touchscreen kiosk by a blind user and a companion app on their smartphone. The bot, once attached to a touchscreen, will recognize its content, retrieve the corresponding information from a database, and render it on the user's smartphone. As a result, a blind person can use the smartphone's built-in accessibility features to access content and make selections. The mechanical bot will detect and activate the corresponding touchscreen interface. We present the system design of Toucha11y along with a series of technical evaluations. Through a user study, we found out that Toucha11y could help blind users operate inaccessible touchscreen devices.
Calico: Relocatable On-cloth Wearables with Fast, Reliable, and Precise Locomotion
Aug 17, 2022
We explore Calico, a miniature relocatable wearable system with fast and precise locomotion for on-body interaction, actuation and sensing. Calico consists of a two-wheel robot and an on-cloth track mechanism or "railway," on which the robot travels. The robot is self-contained, small in size, and has additional sensor expansion options. The track system allows the robot to move along the user's body and reach any predetermined location. It also includes rotational switches to enable complex routing options when diverging tracks are presented. We report the design and implementation of Calico with a series of technical evaluations for system performance. We then present a few application scenarios, and user studies to understand the potential of Calico as a dance trainer and also explore the qualitative perception of our scenarios to inform future research in this space.
Metal Blossom: Laser Forming Complex and Freeform Metal Structures Imitating Flower Blooming
Nov 30, 2021
For centuries, human civilizations devised metal forming techniques to make tools and items; yet, customized metal forming remains costly and intricate. Laser-forming origami} (lasergami) is a metal forming process where a laser beam cuts and folds a planar metal sheet to form a three-dimensional (3D) shape. Designing foldable structures formable by lasers, however, has long been a trial-and-error practice that requires significant mental effort and hinders the possibility of creating practical structures. This work demonstrates for the first time that lasergami can form a freeform set of metallic structures previously believed to have been impossible to be laser-formed. This technological breakthrough is enabled by new computational origami methods that imitate flower blooming and optimize laser folding instructions. Combined with new ideas that address laser line of sight and minimize fabrication energy, we report a low-cost manufacturing framework that can be readily adopted by hobbyists and professionals alike.