In early 2024, I was included in an amazingly talented team to work on The Lucky AI Experience for EDM superstar musician and producer Zedd. The track, Lucky, was from the amazing Telos album and featured (then) rising star Remi Wolf.
Working early on with the incredibly talented team at Mayda, we began a complicated process of creating what can best be described as an interactive music video experience for Zedd, UMG, and Intel to showcase the high-powered laptops.
The experience became an expression of many hidden technologies: a lot of 3D, vtuber style face and expression tracking, an ethical AI generative image system, and so many custom shaders!
The site experience itself is built in my favourite stack (for the era): Next.js, TypeScript, React, Sanity, Three.js, Drei, Fiber, Tailwind, MediaPipe, and Zustand, to name a few of the technologies. It took about 4 months to develop and build the ideas out from our early Proofs of Concept, supplied TouchDesigner demos, and agency supplied direction and turn them into a functioning and responsive concept.
I named the main character of the video the Queen of Stained Glass; she is a head-tracked puppet; she’s structured internally to use ARKit naming conventions with MediaPipe’s Face Landmarker, with MediaPipe controlling the tracking. Additionally, audio drives the depth of the peripheral “plates” of the Queen’s model.
The project was honoured with a Silver Emerging Technology Clio and a Bronze Experience/Activation Clio.
