KAIQING HAUNG

           Works                    Bio/CV                   Curios               

Dialogue of Self - Relationships


Performance & Augmented Reality
July 2024


Director, Visual Artist, Technologist: Kaiqing Huang

Choreographer, Music: Qiao Xiao

Choreographer, Editing: Xiaoqin Zhou

Performer: Duoduo Liang

Tutor: Tongzhou Yu, Yongru Lu, Jiaming Wang

Credit: Holokit

INTRODUCTION

“Dialogue of Self-Relationships” is a real-time performance combining Augmented Reality, dance, and human-computer interaction. We explore the relationships between real dancer, virtual dancer, and the audience. In this context, we investigate the possibilities of identity and perception among them. All these relationships involve the blending and conflict between the real world and the virtual world. Furthermore, this serves as a reflection and speculation on potential issues that may arise in a future digitized society. In addition, the project experimented with choreographing the dance accompanied by the AR production, aiming to explore the most effective workflow.

Artwork video [please select at least 1080p quality]


"I am not merely present in my body as a pilot is present in a ship, but that I am most tightly joined and, as it were, mingled together with it, so much so that I and the body constitute one single thing."

"What then am I? A thing that thinks. What is that? A thing that doubts, understands, affirms, denies, wills, refuses, and that also imagines and senses."

— René Descartes, Meditations on First Philosophy, Meditation II, 1641






CONCEPT
The work aims to explore the complex and unique relationships within the individual, shaped by diverse angles, attributes, and personalities, and how we engage with these virtual, implicit, and integral aspects of ourselves. 
In the expansive realm of immersive technology, the inner self is magnified as the digital, virtual self gradually emerges, navigating between individuals both physically and philosophically. 
This raises the question: how do we encounter this virtual self, another version of us? As the potential for virtual self-realization grows, we are prompted to reflect on how we perceive, interact with, and coexist with our digital selves, and the impact this dialogue may have on our body and mental perceptions.
This work explores these concepts through live performance, where the interplay between the self and AR brings the virtual self into being. We invite the audience to immerse themselves in this hybrid environment via AR headsets, empowering them to manipulate live music, becoming active participants in this immersive, simulated world.



The performance displayed on the mobile phone


AR PRODUCTIONConsidering appearance of virtual dancer, we carried out a variety of testing. Firstly, we used bare bone model, VFX and Shader graph to create the first version (1). After that, the second version (2) was created by the smoke plugin named Zibra Smoke & Fire.However, after testing those effects of the build, it was felt that an image closer to the real performer herself would be more tensional and also better reflect the subtle relationship between the virtual and the real. Therefore, we used Kinect to capture the appearance of dancer in advance.
(1) Particle Morphology
(2) Smoke Morphology
(3) Human Morphology
First, create a curve model in Blender and import it into Unity. Use Shader Graph to develop a dynamic flow effect along the curve. Then, create a linear VFX effect, linking the start and end points to the hands of both a real person and a virtual character. 
MOTION CAPTUREBased on the visual animation captured by Kinect with Depthkit technology, a second capture using Xsens Motion Capture  is required. It is essential to ensure that the movements in both captures are consistent. After using Xsens Motion Capture, all skeletal information of the dancer in motion can be obtained in Unity.CHOREOGRAPHYWe segmented the timeline, assigning keywords representing emotions and motivations for each segment. One set guides the virtual dancer’s movements, while the other directs the real dancer’s performance.These two sets align seamlessly, creating a fluid connection between the virtual and real, and enhancing the coherence of the overall performance.


Timeline driven by keywords shaping dancers' emotions and movements

Four main parts
We asked the dancer to create a body map after the performance to visualize her sensations before, during, and after.

Before: Focus of dancer is placed on bodily control, with attention primarily on the body and less on the environment. During: Sensory perceptions such as touch, vision, and hearing gradually expand, extending beyond the self.

After: The dancer’s focus shifts from the virtual dancer and the environment back to her hands and feet.



MUSICWe set up a music station beneath the base soundtrack, inviting the audience to modify the music and sound effects in real time. As they adjusted the sounds, they became part of the evolving interaction between the "two" performers, blending into this mixed-reality experience. We utilize software MAX to let participants alter the sounds. Considering the complexity of the use, we have incorporated a MIDI keyboard and simplified the MAX interface. 
 
 
The participant modifies the sounds in real time
RELATIONSHIPSReal dancer with virtual dancer:
The virtual dancer represents a recording of a real dancer from a past moment, creating a dialogue between two versions of the same identity across different temporal spaces during the final performance. The real dancer constantly recalls and envisions the virtual dancer's movements, which influences her own performance.
As the performance unfolds, she imagines the virtual dancer not as a mere digital entity, but as a real partner interacting with her. This mental shift subtly alters the virtual dancer’s identity in her mind, blurring the boundaries between the two versions of the dancer.


Audience with “dancers”:
As the audience steps into this immersive environment and gains the power to shape the music in response to the performance, they inevitably become entwined in the dynamic between the real dancer and the virtual one.
In doing so, they assume a role that carries with it a sense of responsibility, a responsibility not only to witness but to influence the unfolding narrative.
 
MUTUAL INFLUENCEThe creation of AR and choreography is a dynamic, mutually influential relationship. When a real dancer performs alongside a virtual counterpart, unexpected effects can emerge. Thus, choreography must anticipate this, allowing sufficient freedom for the dancer to interact with the virtual element in unpredictable and spontaneous ways. Our paper explores this in detail. https://bit.ly/3UkYKJFVISION FOR THE FUTUREIn future work, we aim to explore more possibilities in these relationships. For example, we could create dozens or even more virtual counterparts of the real dancer performing in the same space, or alter the basic physical laws of the virtual world, allowing virtual dancers to defy gravity or perform inverted in midair.Another possibility could involve presenting different versions of oneself, such as variations in gender, age, or personality, etc. These different iterations are likely to emerge in the digital societies of the future, making the exploration of these relationships particularly valuable.