When attempting to accomplish unfamiliar tasks, people often look for tutorials to follow instructions. While it is easy to access online instructions shared by domain experts, navigating step-by-step guidance using existing tools remains inefficient. In addition, producing high-quality instructions that are easy to follow requires authoring expertise and a significant time investment in editing. This dissertation introduces video-based recording, editing, and playback tools optimized for creating and consuming tutorials from author demonstrations. Our interactive systems capture videos and high-level events that are important to a learner. Using video and audio analysis techniques, we develop algorithms that automatically produce high-quality instructions, which dramatically reduce the effort required for amateur creators. By introducing novel tutorial formats combined with video content, these designs in turn improve viewers’ learning experience.
We present a series of authoring tools that enable amateur authors to create effective tutorials: 1) MixT is a system that automatically generates step-by-step mixed media tutorials from software demonstrations. 2) DemoWiz is a tool that provides an increased awareness of upcoming events in a software demonstration video. 3) DemoCut is a semi-automatic video editing tool for physical tasks. 4) Kinectograph is a recording device that automatically follows an instructor for filming a physical demonstration. 5) DemoDraw is a multi-modal system to generate step-by-step motion illustrations from author’s body movements. Current authoring practices from professionals are encoded into automatic algorithms and interactive techniques. These systems are evaluated through a series of studies, which demonstrate that users can efficiently create and follow concise instructions using our tools.
Designing Video-Based Interactive Instructions
Researchers may make free and open use of the UC Berkeley Library’s digitized public domain materials. However, some materials in our online collections may be protected by U.S. copyright law (Title 17, U.S.C.). Use or reproduction of materials protected by copyright beyond that allowed by fair use (Title 17, U.S.C. § 107) requires permission from the copyright owners. The use or reproduction of some materials may also be restricted by terms of University of California gift or purchase agreements, privacy and publicity rights, or trademark law. Responsibility for determining rights status and permissibility of any use or reproduction rests exclusively with the researcher. To learn more or make inquiries, please see our permissions policies (https://www.lib.berkeley.edu/about/permissions-policies).