Cinemorpheque AI: Art Series
XR Opening Video for Film Festival Ceremony, Interactive Installation Art Tour
2022 / Hong Kong, Taiwan, Beijing & Shenzhen
"Cinemorpheque" has been chosen to feature in the promotional video for the 46th HK International Film Festival. The installation's seamless depiction of the changing seasons through immersive scenes provides a striking parallel to the evolving history of Hong Kong's film industry.
"Cinemorpheque" is an immersive interactive installation inspired by classic Hong Kong cinema. The installation uses machine learning algorithms to recreate the four seasons from classic Hong Kong cinema, paying homage to the rich heritage of the city's film industry.
To achieve this, the technical workflow required translating historical cinematic source material into real-time interactive 3D assets, combining machine learning data processing with custom particle systems, hardware tracking integration, and virtual production rendering pipelines.
Generative AI Integration & Niagara Sampling
Custom Generative Adversarial Networks (GANs) were engineered and trained specifically for targeted image morphing, directly converting historical film archives and vintage celluloid stills into high-resolution, temporally coherent spatial textures. These trained visual datasets were integrated directly into Unreal Engine, where scripted Niagara particle systems utilized custom mesh arrangements to sample the visual data on a precise per-pixel basis.
This data-driven sampling method mapped color values and luminance maps directly to particle attributes, dynamically driving real-time environmental deformation and seasonal color grading transitions without relying on pre-baked animation sequences.
nDisplay Architecture in LED Volume & Cross-Disciplinary Workflow
Virtual production shooting for the project took place inside a XR stage and LED volume—the core nDisplay cluster configurations were established within Unreal Engine to synchronise real-time rendering outputs across LED wall panels and camera tracking frustums, maintaining stable frame rates and correct perspective alignment during in-camera visual effects shooting.
Throughout production, structured storyboard-to-runtime pipelines were maintained to translate director requirements into optimized real-time assets while coordinating directly with lighting technicians and stage builders to resolve physical-to-virtual constraints and manage on-set asset handoffs.
Exhibition Tour - Interactive Art Experience For the Public
Following the completion of the virtual production shooting phase on the XR stage, the project shifted into a public-facing interactive art installation format designed for multi-venue exhibition tours. While the initial stage production focused on virtual production, the public exhibition configuration required a hardware-to-software interactive pipeline to translate physical user input into real-time visual changes.
This transition involved integrating sensor hardware into a designed enclosure to stream visitor movements directly into Unreal Engine, enabling users to control the generative seasonal simulations on site.
The interactive installation relied on physical-to-virtual sensor synchronization, utilizing custom Arduino hardware and embedded Leap Motion sensor housed within a physical metal ring structure. Unreal Blueprints were authored to ingest real-time serial data from the Arduino microcontrollers, mapping incoming sensor parameters directly to Niagara particle system variables.
This concept enabled users inside the exhibition space to manipulate simulation density, wind forces, and seasonal morphing states using precise hand gestures.
PARTNERS
Hong Kong International Film Festival
Taichung Museum
Beijing Museum
Shenzhen Museum
MY ROLES
Lead Artist — XR Design & Art
XR Developer — Unreal-to-Disguise Development
TEAMS
Creative Director Chris Cheung
Technical Director Jeff Wong
Project Manager Daisy Lee
Lead Video Editor Kit Leung
Project Asistant August Leung
Project Asistant Mavis Fung
Engineer Wai Choy
XR System / Hardware Jason Chan, Chris Mak
Participatory AI Art / Touch Designer
Cityscape Projection Mapping / Blender & Touch Designer