The Notion of Dialogue in the Interactive Dance
Introduction, background and the main objective
Just as other art forms, dance has been significantly influenced by the development of digital technology over the last three decades (Mullis, 2013). Electronic or digital interactive systems have been experimented within the dance domain (Bevilacqua, et al., 2011). ‘Full-body interaction with electronic or digital media has undergone experimentation for more than fifty years both in arts and science communities’ (Alaoui, et al., 2013, p. 358).
As a sound artist/engineer, I have previously participated in the creation of an interactive dance performance that involved a dancer and two sound artists/engineers, interacting with each other through a feedback loop. The dancer was the initiator of the performance and the sounds created by her movements were received by the sound artists. They manipulated the sounds and played them back to the dancer, which provided her the sonic material to dance to. In this interaction, the interesting point was that the dancer found it hard to define her active role/part in this interaction. In other words, it was challenging for her to recognize if she had more influence on the sound artists - by the creation of the sounds - or she was more under the influence of the sound artists. This presents the effort of each opponent in the interaction, to possess their part in it.
Considering Mullis’s argument in his article, Dance, Interactive Technology, and the Device Paradigm, I am referring to the non-human part of the interaction as the device (Mullis, 2013). Since the participants in the interaction are the dancer and the device, it requires the device, as the non-human part of the interaction, to take certain measures to be able to convey a sense of an active opponent for the dancer. That means in this common interaction, the device may become able to possess its part in the interaction, in a dynamic manner. In other words, in the interactive dance, the device and dancer are the participants and possess their part in the interaction. I refer to this ongoing struggle as dialogue.
Research Questions
The research question here is: ‘How and to what degree can an interactive dance performance with a Hip-Hop dancer using absolute orientation sensors and auditory input, convey a human-computer interactive dialogue through surprise and movement-sound mappings?’
Theoretical framework
The logic here is to create the sense of dialogue between the participants in the interactive dance system (i.e., the dancer and the device), a sense of surprise needs to be created by the device since surprise disrupts the dominance of the dancer in the interaction and allows the device to take its part in it. According to Huran, human beings are wired for expectation (Huron, 2006). Therefore, by disrupting this expectation we can achieve a sense of surprise in the interaction and by disrupting that we can disrupt the possibility of the dancer to be in full control of the device in the interaction, and let the device e possess its parts in it. In order to do that I utilized to strategies, randomness and dependent mapping system. Randomness refers to random acts from the device, they are not necessarily being affected by the dancer. Dependent mapping system refers to different responses, to the same movement inputs from the dancer, based on different temporal and spatial circumstances that the dancer is executing those movements. These strategies were used in practical work to evaluate the mentioned argument. The design strategy included an initial phase that was handled by the researcher alone, and then the iteration phase, which included Abdullah Ghazanfar, as the dancer/co-designer in the procedure. The iterative design happened in four iterations. However, the presence of Abdullah was not consistent because of the Coronavirus pandemic regulations, and the second iteration was done considering the feedback received by an evaluator.
Implementation and design
In order to fortitude the theoretical argument presented, regarding the research objective, a system was designed and implemented. This interactive system focused on Hip-Hop dance, and its subject was about the issue of sea-level rise. The device’s output was an audio-visual one. Regarding the inputs of the device, it captured the dancer’s movements by two sensors (BITaline R-iOT sensors), tied to the dancer’s wrists. Also, it captured impulsive sounds (i.e., clapping and foot-banging) by a contact microphone and a condenser one, in order to trigger events on and off. The data received by those sensors were used to some extent, to control the device’s audio/visual output. The device’s output consisted of a video projection and speakers. I used MAX/MSP as the programming environment considering its efficiency for working with both audio and video materials and also its suitable and convenient user interface.
Initial design
In this phase, the device was mostly responsive to the dancer. Although it was designer, so it could be a platform that could be developed further, by the help of co-designer. Regarding the visuals, a video, representing the matter of the sea-level rise was projected on the walls. It was being flipped randomly on the walls. The visuals included footage of water surface, half below and a half above the water, representing the sea-level rise. The rising started from one side of the projection and continued till it reached to the other end (filled out the projection).
Regarding the audio outputs of the device, it consisted of a lifeline and also Piano and Drum synthesization. The synthesization was done via sonification of the dancer movement’s data (accelerometer data). Also this data was used for implementing effects (delay) on the created drum rhythm. Therefore, he had control over the whole process (via the sensors). By Lifeline I am referring to a sound or combination of several sounds, with any possible nature (mostly ambient) that is present during the entire time, in a performance. This lifeline was controlled by the device and was linked to the video output. It consisted of an underwater sound and a ambient sea sound that was filtered out, in sync with the water level rise in the visuals. The water level rise in the visuals was tracked by color-tracking and was linked to a high pass filter which filtered out higher frequency areas of the ambient sea sound spectrum, by rising the sea level, making the sound darker and less bright as it rose. Also, an underwater sound was played back, which its level had a direct relationship with the high pass filter on the ambient sea sound and as the filtering amount rose, the level of underwater sound rose as well. In other words, with raising the water level, more ambient sea sound was filtered and the level of underwater sound was raised and it became more audible and dominant. This procedure’s sample and samples of the mentioned significations are presented below:
Also, here is an audio example, presenting the audio output of the initial phase:
Unfortunately, there is no video recorded from the initial phase.
First iteration
Moving on to the iteration phase, I shall start by elaborating on the first iteration. Suggested by the co-designer, the Drum sonification was replaces by played back beats (randomly chosen, in selection and playback time) for the dancer to dance to them, and the delay effect was replaced with a reverb effect which affected the beats. Also, the dancer could turn events on and off by using the acceleration values, accelerating above a certain threshold. Also, the functionality of impulsive sounds was changed and the use of Euler angles data, received from the sensors, was added in this iteration as well. The mappings in this iteration are as follows:
- Clapping:
- Starting the beats playback and Piano instrument
- Changing the beat manually
- Piano engagement (on and off)
- Right-hand movements:
- Roll: Piano pitch
- Yaw: Piano velocity (up to zero)
- Acceleration (X-axis): Piano engagement (triggering on and off)
- Left-hand movements:
- Roll: reverb and video delay mix
- Acceleration (X-axis): reverb engagement and video delay (on and off) / reverb limitation (dependent mapping system)
- Foot-banging:
- 1st bang: Starting the visuals and lifeline playback
- 2nd bang: Stops the system sound completely
Also, the following video presents the functionality of the system in the this phase
Second iteration
After evaluating the system with one user, there were some changes made in the design. Therefore, in the second iteration, the synthesization process was removed and a lecture playback system was implemented in the patch. It included several lectures regarding the subject of the performance, being randomly chosen for playback. Also, the reverb affected the lectures, instead of the beats. Following the implemented changes in the second iteration phase can be observed:
Also here are the mappings for this iteration, which remain consistent during the next ones as well:
- Clapping:
- Starting the beats and lectures playback
- Changing the beat manually
- Lectures engagement (on and off)
- Right-hand movements:
- Roll: Lectures speed
- Yaw: Lectures level
- Acceleration (X-axis): Lectures engagement (triggering on and off) / distortion amount (adding up during time)
- Left-hand movements:
- Roll: reverb and video delay mix
- Acceleration (X-axis): reverb engagement and video delay (on and off) / reverb limitation (dependent mapping system)
- Foot-banging:
- 1st bang: Starting the visuals and lifeline playback
- 2nd bang: Stops the system sound completely
Third iteration
The third iteration was done by the help of Abdullah and he raised a point regarding the engagement of the dancer. He mentioned that: ‘in order to become surprised by the changes in the device’s behavior, first you have to be engaged with it’. Thus, regarding that matter, he suggested increasing the number of beats and choosing various beats, from various Hip-Hop sub-genres to make the device’s output more comprehensive and appealing for a wider variety of Hip-Hop dancers. Also, he mentioned that it may increase the chance of the dancer to become surprised by hearing a contrast between the beats the device is playing back to him/her. Following the video recording from Abdullah, trying to interact with the system in this iteration phase, can be found. However, due to a technical problem (i.e., computer overheating), I had to turn off the video pixelation process:
Fourth (final) iteration
Fourth (final) iteration: In the final iteration, the number of beats was increased to five, and the lectures to six. That provided a wider verity in the device output, for the dancer to engage with it, and Abdullah admitted that matter. It could be observed that implementing a wider variety of beats in the patch, increased the chance of him becoming engaged with the device and also, made it more possible for him to become surprised, by hearing a contrast between the randomly played back beats. He also tried to interact with the visuals and lectures and adjust his movement regarding them. Following, the video recording of this session is provided below. Again, due to computer overheating, I had to turn off the video pixelation process.
Evaluation of the practical work
This cannot be denied that the more feedback the system receives, the more promising the evaluation results would be and the process of future development may be more prospective. But, considering the Coronavirus pandemic, this did not become possible. Although there was access to one evaluator and the feedback received from her (which could be positively biased, based on her, being a relative to the researcher), plus having dialogues with the co-designer and observing his state during the iteration process, in the interaction with the device, a rough evaluation became possible. Based on those, on a limited scale, it can be stated that by using a balanced amount of randomness and increase of the dependent mapping system, the device can become able to affect the dancer’s perception of its behavior and create a sense of surprise, and followed by that, a sense of dialogue in the interaction, for the dancer. However, this evaluation is certainly not optimal based on the aforementioned reasons.
Future possibilities:
One of the main suggestions regarding the future possible works is the use of a multi-channel playback system and randomizing the sound output, between various speakers. This possibility will expand the ability of the device to act towards the dancer and surprise him and reduce his possession of parts, to create a sense of dialogue in the aesthetic relationship between them. The other suggestion is to enhance the dependent mapping system, both in-depth and expansion.
Conclusion
In this thesis, the research objective was to define how and to what degree can an interactive dance performance with a Hip-Hop dancer, using absolute orientation sensors and auditory input, convey a human-computer interactive dialogue through surprise and movement-sound mappings. The answer to that question, a theoretical framework was defined, and based on those, a design strategy was conducted to create execute the mentioned proposal. This design strategy was put into practice, by the help of a dancer/co-designer, in order to be tested and fortitude the theoretical argument. However, the practical work needed to be used by various dancers, so they could evaluate the prominence of the presented argument in practice. But, considering the Coronavirus pandemic, this did not become possible. Although there was access to one evaluator and the feedback received from her (which could be positively biased, based on her, being a relative to the researcher), plus having dialogues with the designer and observing his state during the iteration process, in the interaction with the device, a rough evaluation became possible.
Based on those, on a limited scale, it can be stated that by using a balanced amount of randomness and increase of the dependent mapping system, the device can become able to affect the dancer’s perception of its behavior and create a sense of surprise, and followed by that, a sense of dialogue in the interaction, for the dancer. However, this evaluation is certainly not optimal based on the aforementioned reasons.
References
Alaoui, S. F., Bevilacqua, F., Pascual, B. B. & Jacquemin, C., 2013. Dance interaction with physical model visuals based on movement qualities. International Journal of Arts and Technology, 6(4), pp. 357 - 387.
Bevilacqua, F., Schnell, N. & Fdili Alaoui, S., 2011. Gesture Capture: Paradigms in Interactive Music/ Dance Systems. In: G. Klein & S. Noeth, eds. Emerging Bodies: The Performance of Worldmaking in Dance and Choreography. Hamburg: Department of Human Movement Science / University of Hamborg, p. 183–194.
Huron, D., 2006. Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, London: The MIT Press.
Mullis, E., 2013. Dance, Interactive Technology, and the Device Paradigm. Dance Research Journal, 45(3), pp. 111-124.