ARTV3034 Visual Art Diary
This page contains some of the tests, explorations and reflections performed in the second semester of 2021 for the course ARTV3034.
I am more interested in our relationship to technology than the tools and politics that are used to create the works which explore these things. Because my focus is on the communication (like the communication cycle, encoding and decoding information) between humans and their computers in an environment which provides the least amount of friction, the tool that I have decided to focus on for this task is Epic Games’ Unreal Engine 4, using the xbox kinect for input. In an ideal situation, I would somehow like to construct lights or objects in the real world to also react/integrate with the digital world to further create a sense of oneness between the two.
After further reflection, I have realised that what I hope to create is a kind of simulation which is real enough to enable the audience to effectively suspend their disbelief but unreal enough to speak to the post-body world which we will inhabit after being uploaded, thereby challenging the audience’s relationship with technology. A key to this simulation will be the simple inputs. Unlike a flight simulator which requires a knowledge of the function for hundreds of individual buttons. I hope to have my simulation require simple body movements.
I managed to find the required adapter cable for the Kinect to plug into my PC. Using this cable and a 30-day free trial of the iPi motion capture software, I wanted to see how difficult it would be to create motion capture setup, controlling a character's movements in Unreal Engine through an Xbox 360 Kinect. The first ‘shot’ of the previous video demonstrates the computer’s ‘vision.’ The left side shows an object’s relative distance to the camera using colour. A photo of the clean background must first be taken so that the system is able to tell what the background is and presumes the new thing in the frame that isn’t in the clean photo is a person. As we can see on the right side of the slide, this blob of new information then has a digital skeleton bound to it which is the same skeleton the character in the game engine is rigged with. This movement is then directly translated to the character in Unreal, as seen in the second ‘shot’ on the following slide.
While trying to solve all of these technical issues, I didn't ignore developing and exploring the theory of the work I was creating. While reading Nick Bostrom’s The History of Transhumanist Thought, I found a quote from Immanuel Kant who said in his essay “What Is Enlightenment?”
Enlightenment is man’s leaving his self-caused immaturity. Immaturity is the incapacity to use one’s own understanding without the guidance of another. Such immaturity is self-caused if its cause is not lack of intelligence, but by lack of determination and courage to use one’s intelligence without being guided by another. The motto of enlightenment is therefore: Sapere aude! Have courage to use your own intelligence!
I found this quote really resonated with me and made me think about my own “self-caused immaturity.” In the context of Bostrom’s writing it also helped me to further understand why I find Transhumanism such a captivating field of thinking and discussion. The movement’s viewpoint is about a person or people overcoming natural obstacles through their own power and will (and a whole lot more about protecting people’s rights and freedom of choice etc). But it all very closely relates to how I try and live my life and view my place in the world.
Two quotes from Dominic M. McIver Lopes’ The Ontology of Interactive Art that I found particularly foundational for my investigations this semester were:
-“One paradigm of "strong interactivity" is a game”
-“that the structure itself is shaped in part by the interactor's choices. Thus strongly interactive artworks are those whose structural properties are partly determined by the interactor's actions”
The first quote was particularly helpful for me to read early on because up until this stage, I felt like the line between a ‘game’ and ‘art’ was somewhat blurry and that games were a taboo in academia which led me to actively avoid any sorts of writing or design theory around game development. I never intended, nor feel as though I have constructed a game, but to not be afraid of exploring themes around games was freeing and did lead to some fruitful research such as Jenova Chen’s application of Mihaly Csikszentmihalyi’s theory of ‘Flow’ in his works flOw, Flower and Journey.
I see Char Davies Osmose and my work as opposite sides of the same coin. Davie’s work is concerned with reconnecting its viewer with nature by challenging our relationship to it and technology while working in an immersive virtual space. I seek to challenge our relationship and preconceptions relating to technology by manipulating the datasets that construct the fabricated environment. I’m not sure Davie’s would necessarily agree or appreciate being associated with a “techno fantasies common in VR”1 but while our implementation of immersive technologies are in a sense antithetical, the goal of a visceral response and challenging of the technology align quite nicely. Davies often speaks of the intimacy between the ‘immersant’ and the work. This intimacy is a huge inspiration for the instillation plans of my own work.
During my research I found David Rokeby’s Very Nervous System and Myron Krueger’s series Responsive Environments particularly interesting instances of interactive work.
Krueger, who began as a computer scientist started his practice after a relatable feeling of frustration with the technology, specifically the modes of interaction and how he had to adapt to it, rather than it being an intuitive experience. In an interview with Krueger he speaks of a “very natural desire to identify with the image on the screen.”2 His work made me think more about the kinds of influence that the user might have in the world that I construct. By empowering the audience to exercise a level of creativity within the work, they would create a much deeper and personal connection to it. For instance, if I were able to allow the user to shape the environment around them and populate it with objects that they saw fit, they would no longer be participants of an interactive experience but take on the role of co-creator with the technology as their partner.
Many of Krueger’s more developed environments such as Psychic Space and Videoplace seem to anticipate the users interactions with the work and have predetermined responses to these interactions. Given that this series is relatively old and that technology has come a long way, I wonder if a more procedural system would now be a more elegant approach, allowing for a greater breadth of interactions to be effectively responded too. Although Krueger seems to be generally more concerned with manufacturing practical tools which help to create a more intuitive communication with technology rather than the implications of this interaction, I still find his concerns and manner of addressing them quite relatable to my own practice.
Rokeby initially seems to be very closely aligned with my own work, his primary concern being constructing a visceral relationship between the body and technology. He goes about this by using sound which demonstrated to me both the importance and potential of sound in my own work. While this endeavour clearly mirrors my own practice, his reasons for creating his Very Nervous System are apparently to act against what he saw as the strongly biased medium of the computer in an act of contrarianism. Stating “[b]ecause the computer is objective and disinterested, the experience should be intimate.”3
In contrast, I tend to have a more futurist/posthumanist view of technology, that perhaps we should experience ‘intimate’ interactions because as technology evolves and develops they will no longer be objective and disinterested but perhaps beings worthy (as all sentient creatures are) of intimacy and love.
1. Char Davies, "Landscape, Earth, Body, Being, Space, And Time In The Immersive Virtual Environments Osmose And Ephémère", in Women, Art, And Technology (repr., Cambridge, MA and London, England: MIT Press, 2003), 322-337.
2. Myron Krueger, Myron Krueger - Videoplace, Responsive Environment, 1972-1990S, video, 2008.
3. David Rokeby, Very Nervous System (1982-1991), video, 2009.
The previous images demonstrates the constructed digital space in which I performed my IWP presentation. I thought it would be fun to re-create a digital version of the space that the presentations would have been taking place in if it wasn’t for the lockdown. Using photos I had previously taken in the curved wall room, I tried to measure the size of tables, room, etc. and create their digital versions. During the presentation I embodied a digital avatar and showed my presentation slides on the digital TV behind it. This exercise helped me to consolidate the knowledge I had gained in the tests as well as push me to learn some new things that would be very useful in the production of my final work such as the capabilities and limitations of lighting a real-time artificial environment.
This diagram was scanned from a textbook I used during my business communications course last semester and has been repurposed to describe reciprocal the communication process between the human and the computer.
Independent Work Proposal
This semester, I hope to continue where I left off in my exploration of transhumanism using a digital medium. Transhumanists believe that humans at the present moment are a steppingstone in the evolution of our species and that we will eventually transcend our bodies with the integration of technology and augment our intelligence by amalgamating with artificial intelligence.[1] Doing this will enable humanity to design their evolution and make death optional. This carries with it many ethical and pragmatic issues, not least of which is that such a vision of the future would enable a form of eugenics.[2]
To reflect the growing reciprocal relationship between humans and technology, I hope to create a dialogue between the viewer and my digital artwork. This dialogue will be achieved using an Xbox 360 Kinect, motion capture software, Unreal Engine 4, a projector, and speakers. As the transhumanist believes we will become one with technology, I hope to create a feeling of oneness between the user and the artwork. I will attempt to achieve this by utilizing Jenova Chen’s application of Mihaly Csikszentmihalyi’s theory of ‘Flow’.[3] During this flow state, users lose awareness of themselves, actions become almost automatic and time feels distorted.[4] Taking inspiration from Char Davies’ Osmose, I hope to create a state of complete immersion with the interaction being “based on the intuitive, instinctual, visceral processes”[5] and by doing so, users in a sense become one with the computer, emulating the goal of transhumanism. Once this dialogue has been constructed, I hope to manipulate and distort the digital world the user interacts with, in an attempt to discover the more immediate “cultural impact of software and digital image technologies”[6] as well as the post-body world that the transhumanist dream strives for. I expect one of the biggest challenges I will face during my investigations will be to create an environment in which the user may achieve a ‘flow’ with the technology, a state which inherently involves a task to complete, while simultaneously creating an experience which challenges the relationship between the user and the technology they’re interacting with.
My work will aim to demonstrate the relationships capable of existing with present day technologies and then distort these relationships to challenge the viewer to think more critically about them. The Law of Accelerating Returns, written by engineer and inventor Ray Kurzweil identifies the exponential rate at which technology is being developed. Kurzweil writes “we won't experience 100 years of progress in the 21st century—it will be more like 20,000 years of progress (at today's rate)”.[7] This rapid development demonstrates that technology is only going to become more integrated with us and our everyday lives. As we become more reliant on technology, I believe we as a species need to be more thoughtful and critical about the decisions we are making for the development of technology. The transhumanists believe that we will be able to one day design and manufacture our own evolution. If this is true, then the decisions we make now will impact humanity for the rest of time.
[1] Nick Bostrom, The Transhumanist FAQ (Los Angeles: World Transhumanist Association, 2003), 5.
[2] Kate Levchuk, "How Transhumanism Will Get Us Through The Third Millennium", in The Transhumanism Handbook (Gewerbestrasse: Springer Nature Switzerland, 2019), 85-86.
[3] Jenova Chen, "Flow In Games" (MFA Thesis, University of Southern California, 2006).
[4] Milhaly Csikszentmihalyi, Flow: The Psychology Of Optimal Experience (New York: Harper Perennial, 1990), 49-67.
[5] Char Davies, "OSMOSE: Notes On Being In Immersive Virtual Space", Digital Creativity 9, no. 2 (1998): 65-74.
[6] Alan Warbuton, RGBFAQ, video, 2020, https://alanwarburton.co.uk/rgbfaq.
[7] Ray Kurzweil, "The Law Of Accelerating Returns", Alan Turing: Life And Legacy Of A Great Thinker, 2004.
One of the many reasons I chose UE4 as the program to create the interactive experience in is due to its node based coding system. This system helps non-coders like myself more easily approach creating specific programs that I would otherwise require a knowledge of C++ to construct. Thanks to my experience in some other programs which utilize nodes, I already find their logic quite approachable.
This is the diagram I drew as I was developing the locomotion system to help me wrap my own head around it. Despite this being the method that “worked” I tried several similar systems for days which collectively fried my brain, requiring me to frequently refer to this page to help me follow through with coding it. Even though the logic is simple and in hindsight it all seems pretty obvious, not knowing what nodes existed or did led to a frustrating couple of days of trial and error.
After staying up all night writing an essay, I went for a walk and thought about the kind of things I could do to challenge and play with the aesthetics of my virtual world. I don’t think any of the ideas are perfect but they might be good jumping off points.
-
Importing alembic simulations into the world.
-
Creating cloth simulation on sphere that is imported and acts as the skydome for the space
-
Creating a simple destruction simulation in Houdini using Voronoi fracture etc and import that into UE4 but have it play in super slow motion and in reverse or something to create an obvious time distortion. Doing this might help you build off some of the time distortion effects seen in Dan Graham’s Present Continuous Past or Frank Gillette and Ira Schnider’s Wipe Cycle
-
Export megascans assets into maya, create simulations using them then reimport them into UE4 to create an unreal reality kind of aesthetic
-
Revisit the cascade particle sim
-
You spent a long time on getting the player to move in the world. Make it worth it.
-
Have collision boxes which change particular parameters in the space.
-
Manipulate the environment around the user based on relative distances using LOD’s. Further challenging the reality and malleability of the virtual world while incorporating themes of the digital sublime.
You need to give more thought with what you are going to do with your audio. It’s an important aspect of the work and so far has hardly been considered.
To make the viewer uneasy and untrusting of the work I hoped to make the horde of figures suddenly turn to face them as they walked between the groups. This idea was first brought to me during a crit session by one of my classmates who said that doing so might amplify the uncanny feeling as well as give the user a stronger initial understanding of their influence in the digital world. In order to achieve this effect, the individual ai had to be aware of the players global position as well as all turn when an invisible prism was intersected by the player. Communicating the intersection of this prism as well as the global position proved to be far more difficult than I anticipated.
The locomotion system I constructed fundamentally altered the way in which the user exists in the digital space (on a code level). Because of this, almost none of the ‘simple’ effects I attempt to achieve by following along with a tutorial are ever simple.
After finally getting the interaction to work, I realized that the method I had used to do it broke other aspects of the interaction and took another few hours to fix.
After everything was working and the AI turned with their body to face the user I tried to add to the effect by having the faces track the exact position of the player and the bodies face slightly off from direct (or potentially vice versa). In doing this I hoped create a greater sense of unnaturalness to the effect, deepening the uncanny valley. After spending 4 more frustrating hours trying to achieve this effect, I decided that it wasn’t worth spending any more time in pursuing.
Brief Reflections/Considerations
While there certainly has been extensive thought put into the semiotics of my work, some of the visuals used have been dictated by technical limitations. For instance, I didn’t choose the 5 figures in the introductory stage because I felt like they collectively represented the human race. Rather, they were 5 photorealistic free scanned figures who were rigged correctly and ready to be implemented into my scenes.
The inclusion of the cartesian plane in the experience was done for a variety of reasons. Last semester I found it difficult to move past a piece I made that was inspired by Decarte’s notion of “Cogito ergo sum.” This image was quite a stereotypical representation of the famous philosophical position. By placing the disembodied user in the cartesian plane, I am hoping to allude to Decarte and his famous quote but do so in a more nuanced way. This reference will hopefully encourage further philosophical reflection such as mind-body dualism, a hot topic between the transhumanists and post modernists. I personally agree with the transhumanist, Natasha Vita-More’s opinion which is that “we start our cognitive processes from the sensorial mix that we have in relationship to our environment” so to remove our bodies and its senses would be to remove an essential part of what makes humans who we are. In creating a highly constructed, unnatural and seemingly infinite space, I am attempting to make our relationship between our senses and environment more obvious to the user and encourage them to reflect on how their senses shape their identity. The sound of breathing in this space is key to this reflection, making the user conscious of their own breathing and by extension their body.
Somewhat anecdotally, the cartesian plane is seen as foundational in the development of perspective in western art. This link directly speaks to the importance of immersion and its development through history, eventually leading to VR and other immersive technologies such as those used for Life in Silicon. The grided plane is also the digital artists equivalent of a blank canvas, further linking it to my broader practice. So not only does the cartesian plane allude to some of the philosophical discussions among transhumanists but it also speaks directly to the technology and medium of the work itself.
There are so many wonderful and complex themes attached to this work such as the nature of being, truth and reality. Even surface level investigations of these themes have led me to uncover some of my own presuppositions when creating this experience. For instance, I was raised in a religious household which instilled in me a bias towards mind-body dualism. If given more time I would have liked to have further researched metaphysics, rationalism and stoicism. I’m disappointed for not having more time to research philosophical theory that quite obviously speak to the transhumanist movement and by extension, my work. I was considering auditing a course in philosophy this semester but expected that I wouldn’t have enough time for it (and I believe I was correct).
The belief that technology will lead to an enhanced human experience clearly carries social issues that I have hardly begun to investigate. Although transhumanists are advocates for individual freedom and the right to choose if and what enhancements a person may receive, if a rich person can afford to augment their intelligence and a poor person cannot, then the class divisions will grow faster than the technology develops. Given more time I would like to investigate these social and economic factors involved in the transhumanist movement and attempt to incorporate them into my practice.