Tue., Oct. 17 – Tue., Nov. 14, 2023
6:30PM – 9:30PM (EDT) Convert to your local time zone location: Online
Motion capture is a technique that records the movement of humans or objects and translates it into digital data. It is widely used in the entertainment industry for creating realistic animations for movies, games, and virtual reality. However, traditional motion capture systems require expensive hardware and specialized environments, which limit their accessibility and creativity. What if you could use artificial intelligence to generate motion capture animation without any hardware or motion data? In this course, you will learn how to do just that using MediaPipe and Tensorflow.js, two powerful frameworks for building machine learning pipelines for processing video and images
You will learn how to create real-time video effects using face, hand, and pose detection. Although this course will be taught using web technologies, you are welcome to explore other languages since the frameworks we will be covering are cross-platform. By the end of this course, you will learn how to use motion capture to design your own interactive experiences.
Prerequistes:
-Basic web development knowledge (HTML, CSS, JavaScript)
-Node.js and NPM knowledge (preferred)
-Familiarity with Machine Learning/AI (optional)
Required Materials
-A computer with a webcam and a microphone
-A web browser that supports WebGL (Chrome, Firefox, Safari, etc.)
-A code editor (VS Code, Sublime Text, etc.)
-A GitHub account or Glitch account
The class size will be limited to 16 students to allow for plenty of individual feedback. Class meets for 5 sessions. Times are listed as Eastern US time zone.
Registration for this class opens on September 1, 2023. Please join our mailing list to receive announcements and reminders about class offerings and free lectures.
You will learn how to create real-time video effects using face, hand, and pose detection. Although this course will be taught using web technologies, you are welcome to explore other languages since the frameworks we will be covering are cross-platform. By the end of this course, you will learn how to use motion capture to design your own interactive experiences.
Prerequistes:
-Basic web development knowledge (HTML, CSS, JavaScript)
-Node.js and NPM knowledge (preferred)
-Familiarity with Machine Learning/AI (optional)
Required Materials
-A computer with a webcam and a microphone
-A web browser that supports WebGL (Chrome, Firefox, Safari, etc.)
-A code editor (VS Code, Sublime Text, etc.)
-A GitHub account or Glitch account
The class size will be limited to 16 students to allow for plenty of individual feedback. Class meets for 5 sessions. Times are listed as Eastern US time zone.
Registration for this class opens on September 1, 2023. Please join our mailing list to receive announcements and reminders about class offerings and free lectures.
Instructor:
Richard Yee
Richard Yee is a product designer and creative technologist on Google's emoji team. He graduated Cooper Union with a BFA in 2018 and works on innovative projects that combine design, technology, and user experience. He is passionate about creating immersive and interactive experiences that push the boundaries of what is possible on the web. He has received design awards from Awwwards, Css Design Awards, and the Webbys, for his indie web apps. In 2022, he co-authored a research paper on full body motion capture with Google AI and demoed the new model at the Conference on Computer Vision and Pattern Recognition. He currently works with the MediaPipe team to help open source their motion capture models and make them accessible to developers and designers.