Description
With the technological advancements enabled by AI, the vision of generally capable robots is now within reach. In this dissertation, I discuss my work on leveraging data-driven learning approaches for real-world robotic systems, centering on trajectory data—the complex, multi-modal, time-series information that serves as the core unit of data in robotics. The data sources in robotics are complex, potentially coming from multiple sources with varying quality. Additionally, collecting real-world robot data can be expensive and time-consuming, making efficient use of each data point essential.
I aim to address these challenges through three key research directions: trajectory representation learning, high-quality data collection, and sample-efficient policy learning. First, we explore how to learn effective representations from trajectory data, using both reconstruction-based and contrastive learning methods, and demonstrate how these representations enhance a variety of downstream robotics tasks. Next, we examine practical methods that can be used with real-world robotic systems to collect high-quality trajectory data and subsequently utilize that data to learn new skills. Finally, we investigate how to compose these robot skills with high-level language models to imbue robots with stronger reasoning and planning capabilities.
Together, these contributions advance the development of general-purpose robots capable of operating in complex, unstructured environments.