Typeface design education from Cooper Union
Ai mocap landscape

AI Motion Capture for Designers
with Richard Yee

Motion capture is a technique that records the movement of humans or objects and translates it into digital data. It is widely used in the entertainment industry for creating realistic animations for movies, games, and virtual reality. However, traditional motion capture systems require expensive hardware and specialized environments, which limit their accessibility and creativity. What if you could use artificial intelligence to generate motion capture animation without any hardware or motion data? In this course, you will learn how to do just that using MediaPipe and Tensorflow.js, two powerful frameworks for building machine learning pipelines for processing video and images

You will learn how to create real-time video effects using face, hand, and pose detection. Although this course will be taught using web technologies, you are welcome to explore other languages since the frameworks we will be covering are cross-platform. By the end of this course, you will learn how to use motion capture to design your own interactive experiences.


Basic web development knowledge (HTML, CSS, JavaScript)

Node.js and NPM knowledge (preferred)

Familiarity with Machine Learning/AI (optional)
Registration is now closed

Tue, June 6 – Tue, July 11, 2023

Where: Online

About Richard Yee

Richard yee
Richard Yee is a product designer and creative technologist on Google's emoji team. He graduated Cooper Union with a BFA in 2018 and works on innovative projects that combine design, technology, and user experience. He is passionate about creating immersive and interactive experiences that push the boundaries of what is possible on the web. He has received design awards from Awwwards, Css Design Awards, and the Webbys, for his indie web apps. In 2022, he co-authored a research paper on full body motion capture with Google AI and demoed the new model at the Conference on Computer Vision and Pattern Recognition. He currently works with the MediaPipe team to help open source their motion capture models and make them accessible to developers and designers.