Keywords: interaction design, interactive multimedia, physical computing, spatial interaction, gestural interaction, sonification, visualization, generative computing, wearable computing, computer vision, sensors, interactive art, site-specific installation, informative display

Friday, April 18


Interactivity - Zachary Liberman


Interactivity. A simple concept of involving, responding, engaging and taking part of something. Often a 2 directional process with no set limit on the amount of times it exchanges, something is presented, you interact, it reacts/responds/changes, which then follows with your own response and so it continues. Interactive art highlights and involves the audience and environment around it, bring them closer and deeper into the work. Sculptures, websites, installations, computers and some performances can all breach over into this field. Rather than be static and inanimate, interactive art presents a new way to be involved with a creative outlet of another.

Whilst there are many various artists in this field, their goals are individual and personal. Some may use interactive to present information, such as “The Secret Lives of Numbers” [1]. This project shows the popularity of a very large array of numbers based of user interaction. Others like “Skipping Digital” [2] are more of a fun based, ‘play’ piece that each person participates in the manner they choose, whether for entertainment, understanding or just for a giggle. One of these more ‘play’ based artists is Zachary Liberman.

Zachary Liberman describes himself as a “researcher as I feel that my artistic practice is a kind of research” [3] A researcher that’s work tries to develop and express various forms of what it is to be human. As he himself says, his “work takes a playful look at these essential parts of human existence and creates playful, organic myths that aim to shed some light on their nature and origins.” [4] Having displayed his work all over the world, in both interactive installations and performances, Zachary Liberman’s bring a joyfulness to some of the more ‘taken-for-granted’ ideals and interactions in everyday life.

To me, his installation/performance “Drawn” highlights this outlook clearly. “Drawn” is a work that takes physically drawn images and brings them to life, so to speak. Originally a performance piece exhibiting “painted forms ap
pear to come to life, rising themselves off the page and interacting with the outside world.” [5] The performance was also supported by Pardon Kimura to add an audio level to the piece. Kimura is a sound artist with various albums and musical credits to his name and he brings his experience from various fields into this piece. The performance was toured around Europe and Asia during 2005.

In 2006, “Drawn's” success and audience interest prompted an installation version was developed. “After every performance, crowds would gather to peer at the table, and often ask to try the software themselves… their delight was immediately evident. [6] Very similar to the performance version, anybody could come up, draw and then experiment. The drawings were collected and displayed across the walls surrounding the installation, “creating an increasingly dense tapestry of ink forms.” [5] Like the performance version it was developed from, Drawn (installation) was displayed in a variety of museums are Europe.

But how does Drawn actually work? As the artist draws onto a sheet of paper, a camera mounted above captures the drawn image and sends the data to a computer. This computer stores and transforms the image in its memory and turns the drawings into vector shapes. The artist then manipulates the shapes by various hand motions around them. Pushing, pulling and twisting are all possible. The shapes themselves become almost organic as they bounce and flex at the users ‘touch’. Pardon Kimura’s part comes in to play when an ink stroke is touched. Various sound effects are created in response, and reflect the movement and action of the ink stroke. For example, if a stroke is shaken side to side, the audio might be a swishing sound, tracking the movement of the ink. This also occurs in the installation version where movement from the ink sparks an audio response. This to me adds another dimension and mentally engaging level to the piece as it gives the lifeless ink strokes more life.

I find that Drawn is an excellent example of Zachary Liberman’s goal to create playful yet exploration artworks. (See [3]) By combing something primordial like drawing with modern day computer graphics and blending them together, Liberman succeeds in creating a successful interactive and engaging piece. I find the childish nature of the piece a better way to draw the audience’s focus, as we are all children at heart. From examples I have witness, the more simple and basic the drawings, often the more enjoyable and endless possible manipulations are possible. I see this piece as more ‘fun’ than mental stimulating, but at the same time, often this is the best way relate to a wider range of people. As Liberman said in response to the use of Drawn used in a workshop for kids, "There was a lot of laughter. "[6]

Another piece of Liberman that future develops on the idea of play to stimulate the mind is “Motionscapes”. Designed specifically for children with learning difficulties and special needs “an intuitive and fluid means of self-discovery and expression.” [7] This ideal of letting these children develop their sense of the world and surroundings in a playful manner future reflects Liberman’s goal to highlight human existence in various forms.


“Motionscapes” use a fairly basic hardware set-up
in its approach. A simple camera is placed in front the user, and using a projector for image display. As images are projected onto the screen, the users move in front of the camera. The camera takes these motions, computes them and then causes the projected image to respond to the motion. This allows children just simple movements to thoroughly engage with the projection. One thing Liberman has done by focusing on motion rather than the physical being, if the child has a carer, they can sit there being still and let the child engage on their own. This helps individualise the Childs exploration of the piece.

The piece has various ‘modes’ to create very different types of interactive and display. These modes are twirl, fluid, loops, bounce and stripes. Each mode presents a new way for the child to get involved, with their movements creating a variety of effects on the screen. Twirl involves a ball rotating around a centre point. The more the child moves, the bigger the ball gets. It also has an audio file where as the size increases, so does the pitch. Fluid is plasma like image of abstract colours and fluid forms. Movement creates new particles in the image. Movement also determines the direction of the fluid, with larger movements creating more of a ‘push’. Once again, audio is accompanied based on particle position and movement. Loops uses multiples of a single image. Movement causes the loops to start in a flowing order from top right to bottom left. Sound is again based of motion. Stripes uses movement to create coloured bands. Different ways of moving affect the colour of the bands. Sound is generated from the colour and position of the bands. Bounce is a collection of balls on the screen. Movement triggers the balls to move and react to the Childs gestures. They collide with the screen walls and each other. Sound is based of position and speed, including collisions.


When you compare Drawn and Motionscapes, you can clearly see similarities and what must ultimately be Zachary Liberman’s ‘style’. With both pieces there is a child like essence and playfulness. Simple movements create an action on the screen that a user can react and cause to react. There is no political stance or message, no attempt to force a user to think a certain way. Its an opportunity for a users self fulfilment, what they see and do is a reflection unto themselves. Liberman gives us a chance to be ourselves and change and manipulate objects and worlds we normally do not have access to.


The comparisons between informative interactive art and interactivity for the sake of the users enjoyment is a blurred line. One may argue that information presented in an interactive format is not ‘art’, just a prettier way to do it. At the same time, is an interactive art piece designed for fun and enjoyment anything more than a glorified technological toy? Each individual has their own stance, mine leans more to the enjoyment of toys, such as Zachary Liberman’s work. They do however present a user the ability to create their own information from the work, information only bound by their creativity. You could even say this information, in total could provide more information than a piece specifically designed to give you a set piece of data. Informative interactivity has its place however. It provides you with a thoroughly more engaging way of learning and exploring a particular field of data. It can also preset opportunities to look at the data and sort through it in a way that you previously had not thought of.

In conclusion there is a fine line between the set goals of informing with interactivity and providing a medium and creative outlet for a person via an interactive artwork, something I don’t feel justified to complete decide on. I do however feel that Zachary Liberman's work is an excellent example of a way an artist can present a variety of people a simple way to interact with totally individual and varying results and response, all in a fun and play focused way.



References:


[1] (2002: Golan Levin, Jonathan Feinberg, Shelly Wynecoop and Martin Wattenberg – The Secret Lives of Numbers - URL)

[2] (2007,ENESS – Skipping Digital – Melbournes QV Square - URL)

[3] (July 26, 2006: We Make Money Not Art - Zach Lieberman's talk at Futuresonic
By Regine - URL)

[4] (June, 2006: Herraiz Soto & Co - Zachary Lieberman: pintar la voz – Interview by San Cristóbal -
URL)

[5] (Zachary Libermans Website – Drawn
- URL)

[6] (Zachary Libermans Website – Drawn Installation - URL)

[7] (Zachary Libermans Website – Motionscapes - URL)

Friday, April 11

Task 3 – Short Research Report: Camille Utterback

Programmer and artist, Camille Utterback has developed a number of interactive installations that have been exhibited in museums across the globe and also in private spaces. Becoming a fan of computers and technology at an early age, Utterback taught herself to program in BASIC and even made programs to complete her maths homework at age ten. After losing interest in computers during school Utterback focused her attention on art and eventually “rediscovered how cool computers were” by accident [1]. Having already completed a BA in Art from Williams College, Massachusetts, Utterback’s rekindled interest in computers led her to complete a Masters degree from The Interactive Telecommunications Program at NYU, where she now also lectures as a professor. Utterback also works as an occasional professor at the Masters of Fine Arts in Design and Technology Program at Parsons School of Design as well as running her own business, Creative Nerve, which she founded as a means for developing future commercial installations [2].

One of Utterback’s oldest and most playful installations is Text Rain developed in partnership with Romy Achituv in 1999. Text Rain is an interactive installation that projects a mirrored video of participants on to a screen in black and white. Coloured letters, which together form a poem, fall from the top of the screen and can ‘land’ on participants just like real rain [3]. The participants movement is tracked by a video and enables them to catch, lift and play with each of the letters individually or as a cluster. The intelligence behind this effective installation is based on a simple tracking device – “falling text will ‘land’ on anything darker than a certain threshold, and ‘fall’ whenever that obstacle is removed” [4]. If participants are able to catch enough letters, words and phrases became apparent and participants and observers are able to see the relation and meaning of the poem. New phrases of the poem appear as the falling letters slowly fade enabling participants to delve further into the poem. “Reading the phrases in the Text Rain installation becomes a physical as well as cerebral endeavor” [4].

Text Rain has three permanent public installations (in Germany and USA) and has also been exhibited in Spain, Brazil, Taiwan, Poland and Japan to name a few [4]. Text Rain’s popularity and attractiveness is attributed to its accessibility by the everyday folk; its simplicity and the fact that the participant is not required to wear any expensive equipment encourages people of all ages and education levels to play with the installation. Several people are able to interact with Text Rain at any time, some of which may not even be aware of their interaction, as it is merely the person’s presence that affects the falling letters. This installation breaks down the barrier between real and virtual space because users are able to “simultaneously shift between the two, or to feel present in both simultaneously” [3].

Utterback’s Text Rain is one of her most well-received installations because of its playfulness. Participants are also able to directly see the effect of their involvement in the installation and do not have to think about any abstract themes that influence the installation. I believe it is also an inspirational piece of work as its simplicity enables novice artists to envisage other installations and thus encourages them to create installations of a high calibre while understanding that complexity does not necessarily add effectiveness.

“I am passionate about creating experiences that show people that their interactions with computers do not have to be frustrating, deadening, and potentially debilitating. Instead, we can imagine and create a world where this interaction is seamless, intuitive, playful and inspiring. By using video cameras to create physical-digital systems that engage people's bodies instead of just their fingers and eyes, I hope to refocus attention on the embodied self in an increasingly mediated culture.” - Utterback [3]

See/Saw developed by Utterback and her business associate Adam Chapman is an interactive installation that uses the movement of an actual see-saw to tell a story. As participants see-saw a word is projected onto a screen behind both participants; with each movement up and down the words on both screens change simultaneously. The words that appear on each screen are responsive to the angle of the see saw in that the person who is close to the ground will see a word associated with negativity while the person elevated will see a word associated with power. The installation also takes note of when the see-saw motion stops and a short phase is played to each of the participants that incorporates the word that is shown to that user. Each participant only hears one phrase when they come to a stop as the speakers are embedded within each end of the see-saw and thus the other participant does not hear the opposing phrase to their “point of view”. The words shown to each user do not follow a linear pattern but are completely jumbled to make no sense. “Its structure as well as its meaning derives from the temporal and spatial interactions with the see saw” [5].

The See/Saw installation is an effective piece of art due to its originality; few exhibits in any place over the world allow for such ‘hands-on’ interaction. The relationship between the angle of the see-saw and the words displayed to each of the participants conveys depth and meaning to a participant but also enables them to experience the installation. The sound that is played to the participant as they slow down the see-saw motion enables them to discover even further the level of interaction and it adds a drop of playfulness.

Utterback’s works thoroughly convey her interest in art while also displaying her technical capabilities. They encourage people to take a broader look at the world and consider the relationship between the virtual and physical worlds. I believe Utterback’s works are an inspiration to young aspiring designers and artists pushing them to develop further installations that are both appealing to the eye and show some level of abstract thinking, thus creating a number of completely revolutionary works.

“By refiguring the possibilities for interaction with digital media, I question and explore the space between the symbolic and the corporeal; between the virtual and the real. By creating poetic relationships between these spaces I hope to engage people both emotionally and viscerally.” - Utterback [4]

[1] Utterback, C., 1999, GirlGeek of the Week, (Accessed via Safari, 5th April 2008). (Online). http://www.girlgeeks.org/innergeek/gkwk/gkwk_utterback.shtml

[2] Creative Nerve, Inc. (Accessed via Safari, 4th April 2008). (Online). http://www.creativenerve.com/

[3] Utterback, C. Design Systems for Human Interaction Not Human-Computer Interaction, (Accessed via Safari, 4th April 2008). (Online). http://www.core77.com/reactor/utterback.html

[4] Utterback, C., 2008, (Accessed via Safari, 4th April 2008). (Online). http://www.camilleutterback.com/

[5] Utterback, C., 2004, Electronic Book Review, A Riposte to: Camille Utterback, (Accessed via Safari, 6th April 2008). (Online). http://www.electronicbookreview.com/thread/firstperson/tabular

Short Research Report - "Seen" by David Rokeby

Interactive art is artwork that not only engages the user, but the user is also an integral part of the artwork. The user can input data through many mediums including video, colour sensors, motion, heat and other input devices. Generative art, electronic art, andimmersive art all integrate the user as a vital towards the final outcome of the piece. Although some interactive art pieces are designed site specifically and therefore created to adapt to the users and environment that it is housed in, many installation pieces are created so that they can be replicated in many different gallery spaces and still achieve similar outcomes.

David Rokeby

[i]

David Rokeby is a sound and video installation artist born in Ontario, Canada. Since 198, he has been based in Toronto, Canada. He has designed many interactive installation pieces over the years which predominantly engage the human body and its interaction with an artificial system.

“These exhibitions serve as a public research laboratory where my ideas about interaction and experience are tested, affirmed, or shot down. This is a condensation of the results of my free-form research.” - Rokeby[2]

His exhibitions have been displayed all over the world, from The USA, Canada, Asia and Europe. Such venues include:

- The Venice Biennale in 1986

- Ars Electronica (Linz Austria) in 1991

- The Mediale (Hamburg Germany) in 1993

- The Kwangju Biennale (Korea) in 1995

- The Biennale di Firenze (Florence, Italy) in 1996

- Alien Intelligence (Kiasma, Helsinki) in 2000[3]

Some of his recent exhibitions and installations by David Rokeby include:

- Machine For Taking Time (Boul. Saint-Laurent) (2007)

- Cloud (2007)

- San Marco Flow (2005)

- Gathering (2004)

- Sorting Daemon (2003)

- Cheap Imitation (2002)

- Taken (2002)

- Seen (2002)

- inter / face (2002)

- Machine for Taking Time (2001)

- n-Cha(n)t (2001)

- Guardian Angel (2001)

- Shock Absorber (2001)

- Watched and Measured (2000)

- Universal Translator (1999)

- The Giver of Names (1991-)

- 60 (1995)

- Watch (1995)

- Silicon Remembers Carbon (1993-2000)

- Petite Terre (with Erik Samakh) (1992)

- Measure (1992-94)

- (Perception is) The Master of Space (1990)

- Liquid Language (1989)

- Echoing Narcissus (1987)

- Very Nervous System (1986-90)

- Body Language (1984-86)

- Reflexions (1983)[4]

Seen (2002)



[5]
One of David Rokeby’s most well known pieces is the video installation piece “Seen“ from 2002. The installation was commissioned for "Next Memory City", the Canadian Pavilion of the Venice Biennale of Architecture 2002. It is set in “Piazza San Marco“ in Venice, one of the most prominent tourist attractions in the world. The area is used as a source material for the actual installation. The source material consists of 4 video projections over the square, all from the same camera mounted in the same position. These projections are 30 minutes each and record the architecture visible from that point, as well as the people and pigeons who walk through the square. The movements of the people and pigeons are also recorded, and turn out to be an integral part of this installation piece.

Seen[6]


The first and fourth projections show the moving objects (the people and pigeons) and seperate them from the still objects (architecture and souvenir kiosks). The second and third projections then show different perspectives on the traffic flow patterns through the plaza. The second projection is a variation of the movement image on the first projection. It takes the first image and feeds it back on itself on a half second delay, turning all the moving subjects into a procession of themselves. This creates an interesting visual effect as areas with more human traffic flow would appear to be much more densely packed than areas which had less traffic flow. That showed a great overview of the areas within the plaza where more people would travel through, and areas which had a high probability of people passing through. It would also leave an interesting pattern where the infamous pigeons would move. The third projection then traces the trajectory of the occupants in “Piazza San Marco“ and creates a colour gradient relating to the direction that they are moving in.

As all the video projections were processed in the video’s full resolution, every person and pigeon left a mark. As pigeons flew across the plaza, the arcs on their flight left a trail; as did running padestrians leaving trails as they dodged and turned through the crowd. Interesting patterns were therefore created, especially amongst the flocks of pigeons and their footprints.

This installation has the capacity to amaze audience as it transforms regular movements by pedestrians and pigeons into an explosive mix of colour and abstraction over numerous screens within the same room. Within this, the audience can see different views of pedestrian movement in the same area silultaneously, making the movements of the people in the installation almost timeless and almost ghostly. The thought of social spaces is easily ignited amongst the viewers.

“This explains our attraction to optical illusions and mind-altering experiences (chemically-induced or not). Those moments of confusion, where identification and resolution aren't immediate, give us a flash of the raw experience of being. These moments of confusion are also the fulcra of paradigm shifts. It's only when our conventional way of dealing with things breaks down that we can adopt another model, another way of imagining and experiencing a scenario.” - Rokeby[7]

The installation “Seen” sparks many thoughts into the minds of the viewers, by far exceeding the great aesthetic awe. Viewers are set to think about aspects such as social presence, the tension between the architecture and the people who use it, and the use of space in general. It visualises life itself in a way that many people can imagine in their minds, but cannot express in any way to other people. It may be seen as an appreciation of a gathering of people in a well congested area, or it can be seen as a critique on the use or even miss-use of a public area.

The video footage in “Seen” is not live data, but rather a pre-recorded batch of video which has been processed by Rokeby. This would suggest that he is more concerned about the visual aesthetics and informational themes of the piece than the accuracy of the data. This has been how much new media has worked in the past decade, as artworks are beginning to have more of a public and social statement. This follows the mindset of social theorist and English professor Mark Hansen.

“the humanities must embrace technology and that humanists must enter full-scale into the informatics revolution by, for example, contesting the meaning and value of “information” and rethinking what it means to be human in a “realtime,” digitally-networked, global world in which we often cognize in concert with intelligent machines.” - Hansen[8]

The installation “Seen” is created in a style very typical of David Rokeby, as images of users are collected and therefore used as part of the artwork. A very similar installation to “Seen” is “Taken” which was created almost immediately after “Seen” and was displayed in The Art Gallery of Hamilton, Hamilton, Canada. Similarities include:

- Collecting images of the audience and relaying them back onto large screens

- Using multiple viewpoints of the users to collect different projections

- Morphing the different projections to create a sense of fusion in time

- “Freezing time” as a way of analysing social density in a public space

- Collecting and re-presenting interactive experiences of users within a set environment

[9]


[10]

“Seen” can be seen and interpreted in many ways. It can be interpreted as a piece of entertainment or a form of social commentary. These interpretations create a sense of the user’s experience, which is intended to be magical as it is not an experience that the user endures on a day to day basis.

“...as an artist, it’s my traditional right to use every trick in the book to create a magical experience” - Rokeby[11]

Fantasy and illusion can be created in the piece by real life images are presented to an audience, yet it is altered to create a sense of surrealism and enchantment. Rokeby achieves this sense of enchantment by freezing time and creating lapses of movement. Not only is this an intriguing piece to look at, but it simulates a response from the user in the form of a social realisation. A viewer in such an installation or artwork is seen as an integral element of the artwork, and they can single handedly control the outcome of how the visual piece looks. A person can also understand their role in the set environment, as they can see how the environment around them interacts. They are therefore a part of the artwork and as a result of this they are more engaged in the artwork.

“Because the computer removes you from your body, the body should be strongly engaged”-Rokeby[12]

The way that Rokeby has expressed the content is fairly subtle, there is no writing involved where the viewer is bombarded with and kind of propaganda or other written forms which can influence a user’s mind set. It is simply observing an area and the people who interact within it, and then presenting it back to the user in a way that they may or may not have previously realised or anticipated. Because there is no real meaning behind such abstract artworks, there are different interpretations of the same artwork, drawing many different responses from the audience.

David Rokeby has exceeded in interactive art installations as he has found a way to subtly engage his audience and make them feel like they are part of the artwork without making them feel like they are overly exposed to the public. As many of his pieces involve collection of images of large amounts of people, the users don’t feel like they are being targeted and they all serve an equal amount into the final product that the installation crates. Although Rokeby’s pieces aren’t directly live inputs like other artists have created, he leaves a lasting impression as he usually tries to create a social commentary for the user to think about – both regarding themselves and the environment that they interact in.


[2] Dodsworth, C (1998) Digital Illusion: Entertaining the Future with High Technology

[7] Rokeby, D (2002) The Construction of Experience

[8] Hansen, M Department of English, The University of Chicago

http://english.uchicago.edu/graduate/amer/m-hansen.html

[11] Dodsworth, C (1998) Digital Illusion: Entertaining the Future with High Technology

Assignment 2

KEN RINALDO-Standby Deliver

Interactive art is an art which involves the spectator in some way. Sculptures achieve this by allowing the observer to walk in, on and around the piece. Other works such as computers and sensors allow response from motion, heat and other types of input. There can be many types of interactive art, some being contextual theatre, installation art, computer-generated art, electronic art, interactive film, internet art, kinetic sculpture, new media art, performance art and video game art. Ken Rinaldo is an interactive artist like many other who makes their artwork have a message. This message is usually to emphasize the original problem in a more practical way than written information. The following information will be present in the following, an introduction to Ken Rinaldo, the problem which he wishes to address to people who view his artworks and information on his artwork “Standby Deliver”


Ken Rinaldo is an installation artist who focuses on the intersection between natural and technological systems. The combination of organic and electro-mechanical elements give a connection and a co-operate evolution between living and evolving technological materials. His artworks are mainly influenced by theories on living system, artificial life, the communication and interaction between species between and the beauty and fundamentals in the nature and organisation of energy, matter, data and information. His works emphasis on the concern for ecological issues, which are often not highly regarded or taken in account for within the domination of technological and cultural progress. Ken’s motivation in becoming an installation artist comes from the encouragement and self determination with his interest in relationship between humans, machines, nature and culture. [1] He believes that the relationship with a work of art can demonstrate and become both real living system as well as a metaphor for a co-evolved and symbiotic coupling that can exist between nature and culture. Ken Rinaldo work mainly focuses on interactive multimedia installations that blur the boundaries between the organic and inorganic. He has been working with the connection between art and biology for over two decades and his works include interactive robotics, biological art, artificial life, interspecies communication, rapid prototyping and digital imaging [1]


One of Ken Rinaldo works is “Stand By Deliver”. It is based on sugar tweaked cognition. Sugar tweaked cognition is the research of sugar consumption by humans. The increase of sugar intake in our daily lives has increased dramatically in the past decade. Even sugar consumption in the third world countries has increased dramatically. Sugar consumption has been rising at over 1.7 percent a year for the last decade, higher than the U.S. population growth rate of about 0.8 percent. [2] An average American would have the daily consumption of thirty teaspoons of sugar. This does not include natural sugar; it comes from cakes, lollies, chocolate and soft drinks. The increase of sugar consumption has increased due to the amount of added sugar products which have increased in our daily consumables. Examples of this is the spring water we buy can now be flavoured in different fruit flavours, soft drink are also adding extra flavour such as vanilla, lemon and lime and a lot of our cereals have increased in sugar intake. Our gum which was originally peppermint or spearmint has now increased to multi fruit flavours. Another reason for the increase of sugar consumption is that a lot of food is said to be “fat-free”. However foods which are fat free tend be high in sugar. Sugar over the past ten to twenty years has managed to gain a positive image. With the high demand for a healthy diet, many people are aiming towards the fat free products and hence thinking they are not consuming the kilojoules but in fact they are increasing their intake of sugar.
Sugar consumption has raised many issues. The two most common issues that are raised are health issues. The increase of heart disease and diabetes and increased dramatically due to the high in take of sugar. Another issue is has risen are environmental issues. Most of the sugar is grown in third world countries; sugar and salt cropping are the main source of income for people living in third world counties. However the price that the Pilipino farmers have to sell their sugar has lowered dramatically over the past thirty to forty years due to the ineffective world pricing controlling mechanism of the International Sugar Agreements (ISA). This has forced the farmers to leave their crop land and move to other places to which have cheaper rent or a smaller land to manage. This has caused the increase of deforestation and soil erosion. Furthermore, the expansions of human settlement, forests have been cleared to grow crops, wetlands have been drained, and grasslands have been irrigated. [3] The sugar product which has caused a great environmental issue is chewing gum. The littering of gum has caused great problems. Gum is very difficult to remove, the deposal of gum that we chew usually end up on our pavements, hair and the bottom of our shoes. Councils receive countless complaints about the amount of gum in the environment. Gum as well as being difficult to remove is also very expensive to remove. One piece of gum will cost around ten pounds to remove. [4] .

When gum ends up in our greenery, animals eat the gum, thinking they are food and they eventually choke on it as it does not digest. Although Ken’s “Stand by Deliver” is trying to send the message across about sugar consumption, it focuses highly on the damage chewing gum can do to our environment.


Stand by Deliver is a very interesting piece of interactive art. In comparison with his other art works which mainly focused on robotic interaction and artificial life. This took on a different approach.

For images click here

His interactive art piece which consists of steel plates facing each other and moving back and forth attached to activating motors. Under the steel plates is a lit glass sugar molecule. Spectators have the chance to chew gum, which then they can stick to the plates. The steel plates come together and come apart. The gum will stretch which will create long colourful stings of the sticky gum. After many cycles of the plates moving back and fourth, the glass sugar molecule is coated with colourful goo. Although the idea of placing chewed gum on a piece of steel may be quite intimidating, but the reason behind it is to let participants know what the consequences of dumping gum on a pavement can lead to. Ken got the inspiration to design this installation from his great love for sweets, especially chewing gum. He would go through four packs of gum within fifty minutes. He would start of with spearmint, peppermint, fruit and finish off with cinnamon. By the time he got to the cinnamon flavoured gum, he would only be able to taste the cinnamon flavoured the most. After chewing all this gum, he would place the gum on a piece of paper and he thought that that the colours and arrangement were really “pretty” hence he developed this machine which would create a picture from chewed up gum. He always thought that during the process of developing this collage or picture with the chewed up gum that maybe the sugar in the gum had got to him in his body and brain. He thought that maybe the mouth candy that was in his mouth had changed to eye candy hence that is why he found the collages so interesting and beautiful. His message that he wants to send to his participants is that the damage gum can do to our environment. [5]


This piece of installation art is very different to his other works. His other works focuses highly on robotic-human interaction. His method of sending his message through to his participants is quite subtle. You would not automatically think that it is to tell us about the damage of chewing gum. The message is not only towards chewing gum problems but all sugar related products. The packaging of lollies, chocolate and soft drinks have led to a huge problem cleaning our beaches, neighbourhood and coastline. The amount of littering in these areas is damaging our environment and killing our animals. However many people do not realise how damaging it can be because they can not see it before them. Some people have this theory in their head that if they cannot see it they will not believe it. His idea of creating these machines which creates pretty gooey pictures with chewed up gum is also trying to send through the message that although this picture is pretty, there is also a lot of damage this pretty picture can do to us. It is almost contradicting himself.
His artwork is very creative; it has a way of attracting spectators because of the uniqueness in this art piece. The idea of being able to stick your chewed up gum to some people may seem disgusting and unhygienic, but it is because of how odd this process is that it becomes interesting and spectators want to know more about it.

References
[1] Rinaldo K 2004, Emergent Systems: Artist Statement, viewed 3 April 2008 http://accad.osu.edu/~rinaldo/

[2] Lord R 1997, ‘US Sugar Consumption Continues to Grow’ in Agricultural Outlook March 1997 [pdf], viewed 9 April 2008 http://www.ers.usda.gov/publications/agoutlook/mar1997/ao238g.pdf

[3] TED Case Studies 1997, ‘Phillipine Sugar and Environment’, viewed 5 April 2008 http://www.american.edu/TED/philsug.htm

[4] Adams T 2005, ‘This is a stick up’ in The Observer, viewed 5 April 2008
http://www.guardian.co.uk/politics/2005/jul/03/localgovernment.uk

[5] Rinaldo K 2004, Emergent Systems: Stand By Deliver, viewed 3 April 2008 http://accad.osu.edu/~rinaldo/

Asg 3: Hyperinstrument Design

Tod Machover who was attempting to invent new technology for music launched Hyperinstruments in 1986. The concept he worked with was to augment musical instruments with the aid of digital technology to give the performers extra ability for expression and creativity. It wasn’t until after 1992 that this concept was expanded to build stylish interactive musical instruments, satisfying both entertainment and human performance.

What is a hyperinstrument?

Most classic hyperinstruments are built from traditional musical instruments. Having been a cellist and a piano player I know that sound is generated using structures called vibrators. To make a certain note, a vibrator must produce a stable pitch of constant frequency. The physical structures of vibrators are how they generate sound, and is also how the musical instruments are classified e.g. wind or strings. To go into more detail a string instrument produces sounds through the vibration of strings. Frequency and tone quality of the sound depends on varying factors such as strings’ length, thickness, linear density and tension.

Hyperinstruments, on the other hand, are divided into two types. There are those that imitate traditional musical instruments, usually combined with a computer to analyse collected data. These are referred to as hyperinstruments e.g. hypercello, hyperviolin. Others are built from scratch looking unique in their varying structures. These are also referred to as hyperinstruments, but do not have ‘Hyper’ as a prefix e.g. Musical Jacket, Beat Bugs [1].

Typical characteristics: hyperviolin

The main goal in designing hyperinstruments is to expand the power of traditional instruments, providing musicians with more ways to interpret and express music [2].
Looking at most Hyperinstruments their structure are alike traditional musical instruments, even though they are electric. This is because they are designed in mind to suit musicians, so that they do not have to learn new techniques, as original look and feel will certainly improve playability. However there are still variations between each generation.

Here is an example:

Figure 1: Old version of a hyperviolin, which is more or less similar to a traditional violin [3].

Figure 2: New version of the hyperviolin, with a more streamlined style it is certainly a break away from the traditional violin [4].

By expanding these instruments it also means putting in new functionalities, such as the addition of sensors and transmitters. One particular example of this is the hyperbow. The aim of its creation was to capture the most intricate aspects of violin bowing techniques. It has been generally agreed that string instruments are one of the most difficult instruments to play, and it is through experience in which I have to agree. What makes it so difficult is the need to master the coordination between the left hand, which controls the finger, and the right hand, controlling the movement of the bow. The bow is essential to the instrument. Like a singers voice, the bow is used to produce sound, in particularly variations of loudness, the duration of notes, and to distinguish different musical expressions.

What appears to be a simple task is actually quite complex. To create a ‘good tone’ a person must coordinate between the speed in which the bow is drawn, the force applied to the string and the distance between the bow and the bridge of the instrument. It is these interactions in which researchers are keen to investigate. To enable this study of violin playing techniques is a measurement system that can be used to measure parameters in real time, maintaining portability and playability. This system consists of a carbon fibre violin bow and electric violin, in which custom electronics were installed. The bow holds a light and compact electronics board marked by a set of gesture sensors, a small battery and a transmission device [5]. Combined with the audio data provided by the electric violin, the gesture data is collected from this system and recorded by a laptop computer.

Data from these studies can be analysed, and can be used to help understand distinctions between novices and experts as well as the differing bowing techniques between experts.

Example: Tod Machover's design of the hypercello

In 1986, Tod Machover and Joe Chung began the development of hyperinstruments at the M.I.T Media lab. The purpose was to enhance and expand the performance intelligence through technology. The aim was to develop techniques that would allow the performers normal playing technique and interpretive skills to control computer extensions to the instrument, combining the ‘personality’ of human performance with the accuracy and clarity of digital technology. The driving inspiration behind hyperinstruments was in fact Machover’s general musical philosophy to “convey complex experience in a simple and direct way” [6].

The hypercello was designed for the famous cellist Yo Yo Ma in 1990. Along with its development Machover also composed a 25-minute long solo for the hypercello- ‘Begin Again Again’, where the performer controlled a variety of sound producing and transforming devices, allowing of elaboration in new and unexpected ways. Thomas Levenson [7] provides a good description of the hypercello, where it serves as an input device to a stack of machines that process the data created by a performer and generates a variety of different types of responses.

As the cellist plays the cello, sensors pick up the angle of the cellists bow hand wrist forms, where the bow is on the cello’s strings, how much pressure the cellist is placing on the bow and where the cellist is fingering the strings on the finger board. A special circulatory hooked up to the cellist sensors then translate that information into digital information, transmitting the results to the systems main computer. This is synchronised with the electronic signal of the tune actually played, which is then fed directly to the synthesizer, which can either manipulate the data or pass it through as a reproduction of a conventional cello sound.

To accompany this is a software tool, also developed by Machover and Chung to control the flow of information from a performer to his machines. The bow setting vibrations on the string also acts as a conductor’s baton. Machover’s interest in timbre in particular, further led him to develop a technique, which authorizes the performer to add the pure cello sound from the strings to a sampled, or synthesized timbres stored as MIDI data’s in the computer. This allows the performer to decide precisely how much of their playing they want to hear. The computer then generates an accompaniment to the cellist’s solo, in which the performer can control pace and rhythm. As it carries out each function the main computer processes the data it receives, modifying or even creating new sounds, depending on the instructions given within the software.

Yo Yo Ma’s performance of ‘Begin Again Again’ illustrates the vastness of Machover’s hypercello and its software. The technique of Yo Yo Ma enables the computer to measure, evaluate and then respond to many aspects of the performance. These responses could be used in different ways and at different moments of the piece. While at times the cellist playing sounds electronic other times the interweaving of the sounds are indirect and mysterious, where “the entire sound world is conceived as an extension of the soloist…but as a new kind of instrument” [8].

Other interactive instruments

Discussed above was the first type of hyperinstruments mentioned in the introduction. These were the first generation of hyperinstruments, designed in mind for skilled, professional musicians, such as Yo Yo Ma. These hyperinstruments measured many shades of performance expression, and using this to enhance the instruments capabilities.

The Brain Opera: What is it about?

Starting in 1991, Tod Machover continued inventing hyperinstruments, but the purpose this time was directed for non-professional music lovers. Anyone who wanted to express ideas, experiences and feelings in music and sounds was able to create the Brain Opera, and participate in live performances. Each performance of the Brain Opera consisted of two parts. First was an introductory period where the audience could explore, experiment and play with a variety of Machover’s instruments, following was a 45-minute musical event orchestrated by three conductors. The music incorporated recordings made by the incoming audience, along with material and musical contributions from participants from the World Wide Web.

Breaking down the Brain Opera

The Brain opera is divided into three movements, similar to many musical compositions. These final performances are what make the Brain Opera unique. There have been many experiments in the past that have incorporated audience response or contribution to musical environments, but non-so far have tried to recreate it as a real ‘piece’. Machover has attempted this by creating a structure that lets people play with the individual elements of the music, get to know them, add to them, and then see the fragments fit together like a giant puzzle [9].

Although it is difficult to pick out, there is a lot of pre-composed music in the Brain Opera. The music for Melody Easel and Harmonic Driving (in the introductory period) is all composed, but modifiable by the audience. The Gesture Wall, Rhythm Tree and Singing Trees are more dependent on improvisation, where sounds and limits are set and the design and playability of the system, but do not determine the actual music that can be played. An example is the Speaking Trees, where the system sounds out pre-composed stimuli, response is recorded is completely up to the audience member.

In the performance, Movement 1 is improvised, where there is some music but plenty of room for new sounds, thoughts and texts. Movement 2 is completely composed, with room for interpretation. Movement 3 is a mix, Machover has composed basic elements, but has left plenty for the imagination, allowing for addition, modification, mixing and producing a very different version of the piece. Movement 3 also acts as a solo for participants from the World Wide Web. Using ‘The Palette’ the ‘musical instrument for the web’ [10], which is an advanced, multi-user Java applet, active only during Brain Opera performances. Participants are also urged to submit audio self-portraits and images to the Brain Opera, which will then incorporate this to each live performance. The more material the Brain Opera can work with the more exciting the piece will look and sound.

What does the Brain Opera achieve?

With a name such as the ‘Brain Opera’, it is interesting to try to understand what Tod Machover was trying to achieve. Machover designed Brain Opera as an experience to stimulate audiences to reflect on questions revealed by Marvin Minsky, a great inspirer of the idea, whom questioned in a philosophical like manner- why do we like music? Why does music make us feel? And think? Brain Opera also serves to see how independent fragments and layers of music come together to form a complex, yet integrated sonic images. The deepest hope for the Brain Opera is to encourage people to excite their own minds, and the desire to “look inside and hear what is going on” [11].

The title ‘Brain Opera’ is provocative. It is the audiences’ involvement, not the manipulation of the hyperinstruments, that makes the Brain Opera an ‘opera’. Unlike traditional operas with a set narrative, Brain Opera steers away from any narrative, however it does have lots of ‘voices’, both professional and amateur, singing, speaking, individual or communal, where the whole generated texture is very vocal. What is more significant is the “psychological journey with voices” [12]. Participants embark on their own voyage, navigating through the process of understanding the situation of each instrument, and seeing how these can evolve into full musical structures in the performance, creating a rich experience.

Machover combined music and interactive electronic technology to stimulate. The goal of the audience, whether young or old, is to get involved with the performance and the whole experience, not just to pay attention to the technology. Those with an open mind would have a more fulfilling experience. What Machover is doing is more than just piecing notes together to make music; instead he is trying to “touch people’s lives” [13]. What the Brain Opera is trying to achieve is to take the seriousness of how people perceive music and instead making them listen and enjoy the performance.

Reference:
[1], [2], [3], [4] Nguyen, T 2006, ‘Hyperinstruments’, retrieved 5 April 2008 from http://www.tml.tkk.fi/Opinnot/T-111.5080/2006/FPaperit/hyperinstruments_final.pdf
[5] Young, D 2006, ‘Studying Violin Bowing’, retrieved 5 April 2008 from http://www.acoustics.org/press/151st/Young.html
[6], [7] Machover, T 2000, ‘The Hyperstring Trilogy’, retrieved 6 April 2008 from http://brainop.media.mit.edu/hyperstring.html
[8] ‘Hypercello/ Yo Yo Ma’, 2000, retrieved 7 April 2008 from http://brainop.media.mit.edu/Archive/Hyperinstruments/hypercello.html
[9] ‘Welcome to the Brain Opera’, 2000, retrieved 8 April 2008 from http://brainop.media.mit.edu/indexold.html
[10] ‘Interact with the Brain Opera’, 2000, retrieved 9 April 2008 from http://brainop.media.mit.edu/online/net-music/main.html
[11] Machober, T 2000, ‘The Brain Opera and Active Music’, retrieved 8 April 2008 from http://brainop.media.mit.edu/Archive/ars-Electronica.html
[12], [13] Scientific America, 1996, Interview with Tod Machover, retrieved 9 April 2008 from http://www.sciam.com/article.cfm?id=0004C81C-0C01-1C75-9B81809EC588EF21


Art drives a car. Programming provides the fuel to get me there.
-David Rokeby [1]


A hundred rectangular plates, suspended in mid air, ripple about as if brushed by pulses of air from the Ontario Science Centre’s air conditioning. The vast network of floating plates rotates slowly, sending large pulses through the interlocking mass. This is not however, the world’s largest baby mobile, nor a chandelier on steroids. This is ‘Cloud’; the ambitious brainchild of Canadian installation artist David Rokeby. [2]



Rokeby describes ‘Cloud’ as a combination of three disparate states of matter; solid, liquid, gas, as well as the interrelated energy and space. Rokeby’s ‘Cloud’ throws into contrast the large and imposing space of the Ontario Science Centre’s Great Hall, with a work that he describes as ‘largely transport or diffuse’. As such, his work occupies a large volume of the room, but animates and interacts with the room, rather than obscuring and hindering it. [3]

The numerous rows of blue plates ostensibly represent the atoms within matter. Computer-controlled motors rotate the rows at different speeds, sending fluctuating waves throughout the work. These movements symbolize the energy and state transitions of matter, representing the different states of matter in an easy to digest manner. By allowing the user to look directly up into the heart of the ‘matter block’, Rokeby achieves his goal of placing the viewer inside the space of a wave or matter state. [4]

Rokeby, who has been producing similar scaled installation works since 1982, is a decorated installation artist, whose works have received international recognition. He has hosted exhibitions in numerous locations throughout Europe, Asia and North America. He has received considerable acclaim and recognition, receiving the BAFTA award, Governor General's award in Visual and Media Arts and the Prix Ars Electronica Golden Nica Award. [5]

Many of his works feature a prominent theme of the relation between human and computer-based intelligence, and how they interact with the physical environment. [6] They are interactive installations that directly engage the users in conjunction with computer-based perception systems, although unlike other installation artists, Rokeby provides no direct interface for the user to touch or manipulate. As he has put it, his intention was to help viewers learn about themselves, by contrasting people to the areas where computers fail imitate human behaviour. [7]

One of Rokeby’s most recognized and acclaimed work that exemplifies this concept of human/computer intelligence is his ‘n-Cha(n)t’ sonification work. For this work, Rokeby was awarded the 2002 ARS Electronica award. [8] An evolution upon a previous Rokeby exhibition, the ‘Giver of Names’, ‘n-Cha(n)t’ builds upon his previous sonification technologies, creating a network of computers that talk amongst one another in an attempt to reach a consensus concerning the naming of a given object. [9]

Left alone, the computers communicate amongst themselves in their own trademark, idiosyncratic way. However, users are also able to interact with the computers, ‘interrupting’ them in a way, by speaking into a microphone. This form of input allows the computer to add more information to the ongoing ‘discussion’ between the computers, as the agents attempt to make sense of and decipher the words provided to them. [10] Visually, the computers communicate their current level of receptiveness using images of an ear, representing various states such as listening, thinking and ignoring. [11]

While the work throws into light the contrast between human and computer based communication, these interlinked agents are not truly intelligent. As articulated by Rokeby himself, his intention was merely to create a group of intelligent agents, capable of the most rudimentary form of verbal communication, “granted a fraction of some freedom they are utterly incapable of desiring.” [11]

Over 20 years ago, Rokeby produced his first large scale installation. Over 20 years, Rokeby still endeavors to impart his message of human intelligence vs computer intelligence. Over 20 years, Rokeby has refined and received critical acclaim for his first and possibly most influential installation: ‘Very Nervous System.’ [12]











Very Nervous System is a large scale installation that utilizes cameras, image processors, synthesizers and computer processing to create a ‘space’ that captures the movements of the viewer’s body to create coherent music. [13] Motion capture and video software records not only the movements of the user, but the location and movements of various body parts, allowing to computer to interpret and translate the user’s interaction into a truly unique and intuitive interaction. [14] Utilizing top of the range motion tracking and video processing capabilities, VNS is an incredibly flexible system, allowing Rokeby to capitalize on the commercial potential presented by the installation. [15]

Rokeby understood that as the computer was a logical device, the use of a computer to interpret human based activity was biased. Rokeby understood that interaction should be intuitive and natural and strove to impart this upon his computer-controlled installation. [16] To achieve this end, Rokeby replaced the random number generation of computers with the ‘complexity of sentient human response’ to create his art. The result was an interactive sound based environment that also prevented the user from controlling the installation or producing predictable results. [17]

The interaction between Very Nervous System transcended the traditional human-computer interaction of other contemporary systems at the time of its conception. In essence, the environment is being adjusted to the user, rather than allowing digital information and technology to transcend the capabilities of the human body. [18] As Rokeby has described the relationship, it is an open ended feedback loop, where the computer and person exchange and respond the information the other presents; the viewer dances and moves while the computer interprets the movements and generates respective music for the user to dance to. [19] This form of mutual feedback between the computer and viewer was the foundation of Rokeby’s journey into the mechanisms of computer-based intelligence and it’s relation with human intelligence.

David Rokeby has spent over 20 years of his life working to define computer intelligence, and applying his knowledge to contrast human intelligence in an effort to make people more aware of themselves. He is both an inventor and an artist, and his works exemplify this combination of design and practicality to achieve an ends. [20] After 20 years, Rokeby perseveres, exemplifying the boundaries between human and digital interaction, and endeavors to teach something new to both computers and his audiences.


References:

[2]

http://www.youtube.com/watch?v=r1kYDwyu8pU

[3], [4]
http://homepage.mac.com/davidrokeby/cloud.html

[5]
http://homepage.mac.com/davidrokeby/cv.html
[7]
http://www.horizonzero.ca/textsite/invent.php?tlang=0&is=3&file=17

[1], [6], [20]
http://www.horizonzero.ca/textsite/invent.php?tlang=0&is=3&file=3

[8], [10], [11]
http://homepage.mac.com/davidrokeby/nchant.html
[9]
http://www.swr.de/swr2/audiohyperspace/engl_version/interview/rokeby.html
[11]
http://www.youtube.com/watch?v=FMIjxnNllMA

[17], [19]
http://www.medienkunstnetz.de/works/very-nervous-system/
[15], [18]
http://www.horizonzero.ca/textsite/invent.php?tlang=0&is=3&file=11
[16]
http://www.fondation-langlois.org/html/e/page.php?NumPage=80
[12], [14]
http://homepage.mac.com/davidrokeby/vns.html
[13]
http://www.wired.com/wired/archive/3.03/rokeby.html

All images obtained from:
http://homepage.mac.com/davidrokeby/installations.html



Assignment 2: Hyperinstruments Report

Introduction


The hyperinstruments project was started by the composer Tom Machover at the Massachusetts Institute of Technology Media Lab in 1987, continuing from work started in 1978 at IRCAM in Paris. Hyperinstruments aim to capture data regarding the performance of music, process the data using computer software and produce output in real-time (Machover 1992).

Early hyperinstruments were designed for use by expert musicians and the data captured focused on nuances in performance to alter the sonification. Later versions of the technology would extend the scope of the focus to help teach performance skills to novice musicians by stimulating imagination and providing precise feedback (Machover 2004; Grindlay 2007).

Today, aspects of hyperinstrument design have entered mainstream culture as interactive music artwork and games. Additionally, research into hyperinstruments takes place at a wide range of commercial and academic institutions (Blaine 2007). This report however will focus on the works of the MIT Hyperinstruments Group.

Early Hyperinstruments


The early work done by the Hyperinstruments Group at the MIT Media Lab, focussed on the creation of software capable of capturing the output parameters of an electronic instrument, processing this data in some way and producing an output in real-time. As their work became more progressed, the group started making traditional instruments augmented with sensors to measure subtle parameters of the musician’s performance. The following section details the work from some of the projects undertaken by the MIT Media Lab group.

Keyboard Based Hyperinstruments


For the project Fusion Figace, which premiered in 1982, the early hyperinstruments developed by Machover used electronic keyboards to report extremely detailed data to a computer on 32 parameters – information such as pitch, amplitude and phase. This data was interpreted by the computer to create an electronic orchestra influenced by the musician’s performance. The software for this system was written in FORTRAN - In Hyperinstruments: A Progress Report, Machover comments that this system gave much more information than later MIDI based systems, although the process of manipulating the data was far more complex – as unique code had to be written for each manipulation (Machover 1992).

MIDI Based hyperinstruments


In 1987 a new system based on the Musical Instrument Digital Interface (MIDI) standard was developed. In this system, data was captured directly from the MIDI data-stream giving simpler data which was far easier to process. The musical-input device remained an electronic keyboard, but percussion devices were added creating a richer audio-environment.

The captured data was then processed according to one of four rule-sets, and was finally outputted to a digital synthesiser for output. These systems were entirely electronic – from data capture to output. These techniques developed for Machover’s opera VALLIS illustrated major design issues for future hyperinstruments – primarily that the absolute control of the instrument be with the musician in an easily accessible way (Machover 1992).

“Too often in computer/musical systems, control over the computer is inadequate, forcing the human to severely compromise musicality for the lack of a responsive partner. In the worst case, a computer can be worse than a tape recording; while it may be equally deaf and uncontrollable, it may also be playing the wrong thing” (Machover 1992).

By 1990, the BUG-MUDRA project combined input from multiple instruments – MIDI electric and acoustic guitars, and for the first time gestural data from the conductor. This enabled the creation of more controlled and complex manipulations of real-time audio. The continued development further outlined design issues and started a shift away from MIDI based devices to interpreting data directly from acoustic instruments (Machover 1992).

Hypercello


The ongoing research lead to the development of the Hypercello in 1991 and the project Begin Again Again... For this project a cello was designed with augmentation of sensors to measure a suite of parameters such as finger position and pressure, bow position, wrist angle and the pitch and amplitude of each string; a computer would capture this data via the MIDI protocol and pass it into a specially designed software environment for processing; finally, the output would be passed to hardware synthesisers and signal-processing equipment for playback. The result of this processing created the expected sounds of a cello, accompanied by a complex, changing electronic sequence of notes.

“It starts out with the performer exerting careful control over the electronic extensions, each bow change, each accent chosen to elicit a specific response. Gradually, however, the computer part starts to develop on its own - as if a Pandora's Box had been opened - becoming denser and more complex than a single human could control.” (Machover 2004)

With the collaboration of cellist Yo-Yo Ma, Begin Again Again... was performed live on numerous occasions. The score written by Machover utilised the capabilities of the hypercello with some movements being largely unaltered cello sounds, and others with the computer generated sound leading the performance. This was achieved by the computer following a pre-programmed set of rules which change throughout the performance (Machover 1992).

More Recent Hyperinstruments


Following the success of the early projects, the hyperinstrument group expanded its focus to creating other forms of computer-mediated musical devices. These projects had a greater emphasis on interactivity for amateur musicians, expanding the use of hyperinstruments as learning tools. The hyperinstrument group achieved these goals by embedding digital sensors into objects, creating instruments with integrated sensors as learning tools, visualisation of music.

One project undertaken by the group was Brain Opera which ran from 1996 until 2000, project aims included the creation of inexpensive, fun, easy-to-learn hyperinstruments designed for children (Machover 2004). These instruments combine elements of tangible computing – by changing the control-interface to a tangible object, used by touching, squeezing and shaking.

Mainstream Hyperinstrument Works


The development of powerful, low-cost game consoles has allowed the development of commercially released music games which utilise concepts from the hyperinstruments project. The game Guitar Hero for Playstation 2 by Harmonix Music Systems (comprised of former students of Machover) allows players to play complex guitar parts through a simplified interface – the software automatically handles pitch, while the player focuses on rhythm and playing the correct sequence of notes. This system simplifies the creation of music by being limited to pre-composed elements, but still gives players the ability to personalise their performance (Blaine 2006).

Guitar Hero is designed purely for entertainment, but many of its elements could be used in a learning environment, the scoring system keeps track of timing accuracy, pitch accuracy and other parameters. These could be used to provide detailed input on where a musician needs to practice their skills and provide suggestions to do so (Blaine 2006).

Conclusion


Hyperinstruments in the hands of professional musicians are able to provide uniquely detailed insights into a performance through the tracking of additional parameters only subtly revealed in normal performance. By allowing these nuances in a musician’s performance to be captured and emphasised, the musician can create far more complex expressions of a musical work.

Conversely, in the hands of an amateur musician, hyperinstruments can provide additional feedback not present in traditional instruments while providing a fun, easy-to-play interactive experience. This opens new possibilities for the process of teaching music, and for allowing people to interact with music, rather than simply passively listening.

References


Blaine, Tina. “New Music for the Masses.” Adobe Corporation, 2006 Retrieved from: http://www.adobe.com/designcenter/thinktank/ttap_music/
Grindlay, Graham C. “The Impact of Haptic Guidance on Musical Motor Learning.” MIT, 2007.
Machover, Tod. “Hyperinstruments: A Progress Report.” Cambridge, MA, MIT, 1992.
Machover, Tod. “Shaping Minds Musically.” BT Technology Journal, London, Vol. 22, No. 4 2004
Rossing, Thomas D. “Flutes to hyperinstruments.” Nature, 18 August 1994: Pages 509-510.

Thursday, April 10

Short Research Topic – Nigel Johnson

Nigel Johnson is a British artist focused within the domain of interactive media and the development and production of computer-controlled, two and three dimensional, "real-time" interactive installations and digital artworks [1]. Johnson is currently a research professor at the School of Media Arts and Imaging at the University of Dundee.

This report will focus on a few examples of Johnson interactive Art installations including his internationally recognised ‘Fire-Fly’, and ‘A-Life’ - his large-scale adaptation of John Conway’s "Game of Life", incorporating elements of artificial intelligence, cellular automata, artificial life and gaming.

Fire-Fly:

“The inspiration for Fire-Fly came about from witnessing for the first time one early summer’s evening in New Jersey, USA, large points of luminous, glowing, green light, ‘dancing’ in the night air”[2]

Fire-Fly is a piece created by Johnson in 2002 and is comprised of a structural grid above the participants heads, LED’s, ambient sound, ultra sonic motion detectors and user movement to mimic the gleams and glows of a firefly. Obviously for any installation in which light is a key part the ambient lighting should be subdued in order to maximise the impact from the installation. Hence why Fire-Fly is displayed in almost total darkness to intensify the green light emitting from the LED’s. The artist explains that the ‘lightweight, thin, phosphor-luminescent discs [that hang from above] mimic the chemical [of a fire-fly] and produce a diffuse but bright green glow’ [3].

Interestingly Johnson’s experience with fire-flies are in contrast to my own experiences with fire-flies - in my experience the fireflies emit more of a sharp distinct piercing light rather then a ‘diffused glow’ that Johnson describes, so I’m not too sure why the disc are used if he’s trying to imitate the fire-fly’s chemical reaction. However there are supposedly over 2000 species of fire-flies and the ones I’ve had experiences with are from Queensland, Australia which most likely are a different species from the ones Johnson saw in the USA. [4]

In my opinion a better effect could be created if Johnson had made the installation look more natural; perhaps by entwining bush leaves through out the grid in which the led lights would reside. As participants look up at the light they would see flashes of light coming from inside a bush rather then a bare grid structure. This could easily be implemented and the only consideration would be to make sure the sight of the motion detectors was not obscured.


The ambient sounds of beetle wings flapping can be heard as participant navigate the installation. The sound is distributed around the space via a switching matrix connected to amplifiers and four loudspeakers positioned at the outer edges of the installation space and approximately 2 metres above floor level. [5]

The sound is really important in this installation. It would really help give meaning to the participant as without it a variety of interpretations could be derived from blinking green lights – Morse code signals, green eyes blinking etc. It’s a shame Johnson hasn’t manage to implement his idea of mounting smaller speakers spread out amongst the lights to enable a more dynamic movement of sound as participants move from one side to the other. The earlier idea of using branches and leaves to make a more natural look would be beneficial for this idea as smaller speakers could be mounted onto the grid structure itself and then purposely hid by the foliage.

Despite my critisim of the piece Fire-Fly has gained significance and international recognition through their inclusion at peer reviewed, festivals such as European Media Art Festival and the World Wide Video Festival.

Digital Gateway:


Digital Gateway was created in 2004 for the entrance to a new gallery called ‘Wired Worlds’ at The national Museum of Photography Film and Television in Bradford, UK. Johnson’s ‘Digital Gateway’ utilised multi-processing and l.e.d. technology, dealing with the transition from one state to another, from the physical analogue world into the digital domain. [6]


Its structure consisted of two opposing clear glass vertical walls embedded with 128 autonomous modular circuit-boards, consisting of 6,912 red light-emitting diodes, infra-red sensors, 128 micro-computers.[7] The idea would be for participants who were entering the exhibition to see themselves represented as blocks of light at the beginning of the gateway and as they progressed through these blocks would become less coarse; eventually being represented as binary i.e. 1’s and 0’s. Johnson decribes it as “[taking] the human form as the vehicle for its central theme and imparts the idea that this representation can be broken down into fundamental building blocks.”[8]

In terms of showing analogue to digital or real world to the digital world this concept is an extremely clever idea. After all, digitization consists of binary – 1’s and 0’s, which in computers constitutes On and Off, so it makes sense to use LED’s which also only have two states – they can either be on or off.

As it was an entrance piece it had to deal with fluctuations in traffic. In the building process Johnson identified several criteria that the installation had to meet – one of them being that visitor flow can vary from zero to highly active [9]. The installation also had to be ‘real-time’. If there had been a delay in displaying the visualization it wouldn’t work so well and there wouldn’t be that ‘seamless’ transition of which he speak about.

In its period of inactivity Johnson had programmed the installation to kind of experiment with itself by giving it the opportunity to ‘play’ with individual imagery.

To give a true sense of immersive-ness perhaps the use of 4 walls arranged into an enclosed corridor through which the participant walks through would give a greater effect. Being enclosed by thousands of LEDs all changing depending on your movement would give a heightened sense of what the Digital Gateway is trying to convey. But as I mentioned earlier about the criteria he identified before building the installation – one of them was budgetary constraints. If you had an extra two walls of LEDs the costs go up significantly; not just because of the extra walls but the extra processors needed to control them, and the use of strengthened glass so participants could walk along the bottom wall.

A-Life

“At one level, this installation is a simple metaphor encompassing the complexities of life, distilled down into the basic components of life, birth and death governed by the rules that control these events. At another level it is full of the subtleties and complexities of life itself.” [10]

A-Life is Johnson’s re-creation of the earlier work of John Conway’s “Game of Life” [http://www.tech.org/~stuart/life/rules.html] which is a simple yet fascinating algorithm with has fascinated numerous people of various studies for over 35 years.

Using equipment similar to the Digital gateway Johnson constructed a wall 3.4m wide x 2.9m high x 1.9m deep and a remote interface unit.

This is a large scale, real-time, interactive computer installation which Johnson describes as “incorporating elements of artificial intelligence, cellular automa, artificial life and gaming.” [11]

This work follows on from some of Johnson’s earlier research into artificial life and eco systems from the late 70’s which the artist says was partially inspired by Conways ‘Game of Life’. Johnson’s Walk II (1979) and Eco.II (1980-81) were his initial attempts at creating artificial life forms in computer software.

A-Life allows the participant via the remote interface unit to play the ‘Game of Life’. Consisting of three buttons and a joy stick allowing movement in 4 directions – up, down, left & right. The first button allow the participant to initially select a starting pattern of cells to be ‘live’ and the subsequent buttons allow the participant to pause and clear the current game.

Although many people have re-created the ‘Game of Life’ most of them have been on the small scale so its pleasing to see somebody bring this intriguing idea of bringing “simple rule-sets that could lead to the development of complex systems” [12] to a larger scale and implementing a wireless interaction between the participant and the installation. But in terms of originality this piece is lacking as the credit for the algorithm and initial implementation goes to Conway.

Nigel Johnson has been in the game of interactive installations for a long period of time and if you visit his site you can see a varied amount of interactive work.

References:

[1] Nigel Johnson: Research Summary
http://imaging.dundee.ac.uk/research/profiles/research-profiles/view/nigel-johnson/research-summary/; last accessed 10/04/2008
[2] Fire-Fly: Proposal
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/firefly/firefly.html; last accessed 10/04/2008
[3] Fire-fly: General Description
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/firefly/ff_gen.html; last accessed 10/04/2008
[4] Wikipedia: Fire-Fly
http://en.wikipedia.org/wiki/Firefly; last accessed 10/04/2008
[5] Fire-Fly: Installation
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/firefly/ff_inst.html; last accessed 10/04/2008
[6] [7] Research History : Digital Gateway (2004);
http://imaging.dundee.ac.uk/research/profiles/research-profiles/view/nigel-johnson/research-history/digital-gateway/; last accessed 10/04/2008
[8] [9]Digital Gateway: General Description;
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/nmpft/nmpft_gen.html; last accessed 10/04/2008
[10] Project – “A-Life”
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/alife/alife.html;
; last accessed 09/04/2008
[11] Prof. Nigel Johnson: Rearch Summary
http://imaging.dundee.ac.uk/research/profiles/research-profiles/view/nigel-johnson/research-summary/; last accessed 09/04/08
[12] A-Life – Context
http://imaging.dundee.ac.uk/people/njohnson/content/proposals/alife/al_context.html; last accessed 09/04/08

Assignment Task 2 - Short Research Report - Design Work by Karlheinz Essl





Note: I am not at all familiar with sound so I researched on someone who did know about it. Then I realized how little I know about sound and how very difficult it was to write about it. Forgive me if I missed out on things. (Edit: referencing didn't look like it worked with the blog).

Design Work by Karlheinz Essl

[i]

Karlheinz Essl is an Austrian top performer in Electronic music. He also does the impossible: composing unending electronic music pieces, writing music samples, creates software for Max/MSP, develops generative sound and video environments and has to power to improvise music over and over again. All this is featured in his music. Essl believes that ecstasy and trance’ always meant something for him. He tries to feel and share it through his music. To him, “composing is a pain”[ii]; he likes to improvise, for “things can happen that the composer never planned on having.” [iii]
The music is influenced from rock to jazz, and even music from the Renaissance period through to the sounds of computer –generated music. With all this impact, he breaks the normal barriers of a composer. He contradicts the meaning of a music play, what it is, how does it begin, how does it end, and how it must be structured. And amazingly, he does this through the use of MAX/MSP software.
The compositions from Essl are widespread and range from instrumental classical music, to mixed forms of live-electronic, to real-time composition and improvisation concepts to sound installations and internet projects. Essl uses m@ze°2 (Modular Algorithmic Zound Environment)[iv] - an electronic keyboard instrument he created that allows users to compose and improvise music. It is connected to his RTC-lib (Real-time Composition Library), a Max/MSP software library for his algorithmic compositions. It contains random, harmony, rhythm and structural generators. [v]
Most of times, Essl is a composer and performer in public. He believes that in today’s society that musicians are very limited to the composers of the music and the performer of the music. What makes the composer unique is that he should not only be able to compose for himself but also compose music in all categories and for all occasions. Essl has been studying closely to the principles of organization and drama through his music. Some of his works have been commented of being “constructive” from listeners, Essl says “[construction] only helps me get into areas where I would normally not be able to reach”. [vi]

Lexikon-Sonate


[vii]
Lexikon-Sonate is a program that composes an endless piano playing piece. Basically, there is no score for the computer to read from, and there is no pianist playing a piano, or a composer to direct it. The algorithm is comprised of "which key should be pressed how quickly and held down for how long"[viii] which then directs the piano into playing the note.
The goal was to create a program which can compose in real life – synchronous. The user constructs the sound from their own character personalities (personal taste), and henceforth becomes the composer him/herself.
He used music gestures from “Bach, Beethoven, Schönberg, Webern, Boulez, Stockhausen and Taylor” [ix] and placed them into ‘modules’. These modules are then mixed and a part will be extracted from the piece from different constraints placed on it. The technology he wrote this in was Max/MSP as seen in the image above. There are the structure generators seen on middle left hand side, that controls the melody, chord, texture, repetition and pauses.[x]
Lexikon Sonate is surprising it can create sound from nice soft tones to fast high pitches, varying in volume, pace and style. It is truly amazing that something can be created from a piece of software and sound system that would never repeat itself. It is very humane, yet contains none of the classical composition, for it can be relaxing at one point, but so intense the next. It would definitely have been more interesting if he not only used the piano, but other instruments. Yet even though there is still a limit in the sounds of a keyboard, the variables Essl’s program Lexikon Sonate can process, is unimaginable.
Sequitur
This new work comprises of performances using solo instruments including the flute, clarinet, trumpet, violin, cello, toy piano, e-guitar and voice mixed in with live electronics. The Sequitur program receives the input from the instruments, and differentiates it into eight parts. The random generator plays a different amount of parts each time, at various intervals all random. What is interesting about this is that there are inputs of music, but what comes out as the result is totally different to what was played. The processing is done in real time, with sound transformers altering the sonic shape. [xi]
The song Sequitur III for violin and live-electronics begins high pitched the setting for a film - a torturing scene. Then I feel the lost romance in the next tones. At the middle, it sounds like Chinese music for Peking Opera which elevates the mood. It becomes sad, and then the high keys of the piano makes it light again. That was the only instrument I can fully recognized, the process by which the violins and other music go through in Sequitur III it becomes intertwined and mysterious. What is different in their location to our context is that it all seems classical, yet modern without any harsh beats of a drum. Some parts make you tap your feet; other melodies make you concentrate more to the composition of the song.


Sonnez la cloche!

In 2003, he used church bells to create a live performance. This piece was performed in Tyrol, Schwaz and he firstly took a sample of the church bells, which then turned it into a never-ending piece. Meanwhile, he varied the sound using his Max/MSP program and waited till the real church bells rang and synchronized his music along with it. An audience member commented that the piece “changed the very Alpine air we breathe”. [xii]
The sound of Sonezz la cloche is so complex to figure out, and what Essl has created in his programs are more farfetched than you can imagine. Watch this video to see.







Other Works

Through the Curly Rain Taps is a 2005 generative visual sound installation composed by Karlheinz Essl in collaboration with Ramón Gonzalez-Arroyo. It contains the “collaboration, comination of image and sound, infinte flow, relationship to the work of the two different and similar painters”.[xiii] Essl provided the real time endless piece of music which synthesised beautifully with the images.

Panta Rhei is a 2006 generative sound environment for an art installation by Jürgen Messensee. The sounds depict water in its natural course – from destructive waves to the peace and soliloquy of the ocean. This piece is randomly generated with “the rising and falling of the sounds, the glissando movements and the spectral sound shifts are controlled by a time-variant wave model, the parameters of which are constantly changed by random operations.”[xiv] In Panta Rhei, everything is meant to be fluid.

From all his works, I realized how nothing is absolutely the same, how electronic sound can be generated, processed and portrayed as an art form.

[i] Lackinger,H., (2004) “Karlneinz Essl”, Image: http://www.sammlung-essl.at/shared/photoshop/KHE/lackinger/KHE-strudel.jpg (Accessed via Mozilla Firefox, 10th April 2008) (Online).
[ii] Transcript by KHE, “Karlheinz Essl interviewed by Bruce Duffie“ (1997) WNIB Broadcast, Chicago (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/bibliogr/essl-duffie.html (Online)
[iii] “Karlheinz Essl”, (2008), Myspace network (Accessed via Mozilla Firefox, 10th April 2008) http://www.myspace.com/karlheinzessl (and http://www.freetranslation.com/ for German translations) (Online)
[iv] Essl, K., (2007), “m@ze°2” (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/works/maze.html (Online)
[v] Essl, K., (2007), “Real Time Composition Library” (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/works/rtc.html (Online)
[vi] “Karlheinz Essl”, (2008), Myspace network (Accessed via Mozilla Firefox, 10th April 2008) http://www.myspace.com/karlheinzessl (and http://www.freetranslation.com/ for German translations) (Online)
[vii] “Lexiton-Sonate 3.2 User Interface” (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/works/lexson-modules/dep.html (Online)
[viii]
[ix] Essl, K., (2007), “Lexikon-Sonate” (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/works/Lexikon-Sonate.html Image: http://www.essl.at/pic/lexson/lexson32.png (Online)
[x] Essl, K., (2007), “Lexikon-Sonate: An Interactive Realtime Composition for Computer-Controlled Piano” http://www.essl.at/bibliogr/lexson-sbcm.html (Online)
[xi] Essl, K., (2008), “Lexikon-Sonate: An Interactive Realtime Composition for Computer-Controlled Piano” http://www.essl.at/works/sequitur.html (Online)
[xii] Ferguson, S., (2003) “I hear the Bells” (Accessed via Mozilla Firefox, 10th April 2008) http://www.essl.at/works/sonnez-la-cloche.html (Online)
[xiii] Essl, S., (2005) “Through the Curly Rain-Taps” (Accessed via Mozilla Firefox, 10th April 2008) http://sammlung-essl.at/deutsch/musik/archiv/klangwelten/rain-taps.html (Online)
[xiv] [xiv] Essl, K., (2008), “Panta Rhei” (Accessed via Mozilla Firefox, 10th April 2008). http://www.essl.at/works/panta-rhei.html (Online).

Short Research Report - Interactive Installation Art

Interactive installation art uses various materials and other media to change and/or enhance the way a particular space can be experienced by users. It is not necessarily limited to gallery spaces and can be installed anywhere in everyday public or private areas.
Almost any media can be used to create and interactive art experience in a particular environment. New media such as video, sound, performance, computers and the internet are often used, along with the conventional materials, ranging from natural to man-made. Though some installations are site-specific, in that they are designed to only exist in the space for which they were created, others are able to be adapted to other areas, thus creating a slightly unique and different experience each time.

Rafael Lozano-Hemmer is a Mexican-Canadian artist. Lozano-Hemmer is best known for his large-scale theatrical interactive installations displayed in spaces across Europe, Asia and America. Using his custom made interfaces as well as new technologies, such as: real-time computer graphics, film projections, positional sound, video and ultrasonic sensors, LED screens and other devices, his installations aim to provide "temporary antimonuments for alien agency" [1]. “I always use the word alien instead of the word new. The precedents for new are so large, but if you say alien, it is simply something that does not belong there” [2]. His works seek to break up the constant and ever increasingly uniform urban state that society is becoming by providing critical platforms for involvement and participation.


UNDER SCAN (above) is a large-scale public art project commissioned by the East Midlands Development Agency in England. Thousands of "video-portraits" taken in Derby, Leicester, Lincoln, Northampton and Nottingham will be projected onto the ground of the central squares and pedestrian streets of these cities [3].

The work uses very sophisticated, integrated technologies. The world's brightest projector (110,000 lumens of intensity) is installed to inundate the particular area with bright light, which would allow for the creation of the shadows of people, which is a vital part of the work, walking through it. A computerised surveillance camera will sense people as they walk though the vicinity, it predicts where the individual would go and sends the information to the 14 video servers and 14 robotic video projectors which can project anywhere within the site. Thousands of video portraits, which would have been previously recorded, of people from the local area, are selected at random by the servers. These will then be displayed in the viewers’ paths by the projectors and will come to life through interaction with pedestrians' shadows.
Though this is interactive art, the participants, the everyday members of society who view the work, do not actually directly manipulate the work themselves. They cannot actually have their own input and influence how the work is made. However, their physical presence is essential so that Lozano-Hemmer’s work is able to “come alive”. If nobody takes part in the installation, the work will not exist.

“In interactive art the viewer is an integral part of the artwork, so many artists attempt to foster intimacy by personalizing experiences, by establishing close relationships … I am interested in more collective and connective experiences that several people partake in.” [4]. As Under Scan has moved from country to country, the recording of the various locals of the area in Lozano-Hemmer’s work has already formed an “intimacy” with the participants of the installation, as they are all from the same place. They will feel some sort of familiar presence, as they are all locals and have the same sense of community. Furthermore, by allowing those that are filmed to express themselves as they like, Lozano-Hemmer has enabled a “relationship” to form between the participants and the video portraits. Due to the fact that those who are recorded are free to act and move as they like, the projected images will have their own individual traits and characteristics. As the video portraits would have their own personalities, viewers of the work would be intrigued by the individual reactions of the portraits and as a consequence, treat it as though they are meeting another person, thus forming the “relationship”. They would be able to interrelate with them, in that they are able to wave back if they were waved to, imitate the portraits’ actions or even just smile and watch what the portraits are doing.

Under Scan did not use sound projections at all. The portraits were mute. However, each of the portrait subjects was trying to do something to stimulate the public (talking, dancing, rowing etc.). In response, people were trying to communicate with the subjects of the video portraits. The non-verbal gesturing carried out by the portraits engaged the viewers even more as it caused them to focus even more on the message the portrait was trying to get across, even if the portraits did not want to say anything, the fact that there was no sound coming from the portraits, allowed the viewers to think about what was happening and interpret it in their own way.
More intimacy and connection is created as the images are projected from the participants own shadows. It is as though the viewer was able to bring forth the portrait thus causing more intrigue and curiosity. Those being recorded were asked to look at the camera at some point so that eye contact will be made, allowing the viewer to feel more involved and connected and not just a spectator, it would again; cause a bond between the viewer and projection.

“Call my work relationship-specific, not site-specific” [5]. Lozano-Hemmer, through Under Scan, and many of his other works, such as “Subtitled Public” and “Body Movies”, creates not only a relationship between the viewers and his artworks, but also between the viewers themselves. As a result of the fact that the computer system programmed by Lozano-Hemmer would point 14 robotically controlled projectors to interrupt the path of pedestrians, 14 individual portraits were simultaneously projected over the site; meaning 14 individuals would be able to interact at the same time. However, most of the time two or three people would gather around one portrait and start to understand what they were trying to tell them, thus creating more “relationships” between the viewers as they interacted with each other.

His other work, "Body Movies" (seen right), was also able to create various relationships between his viewers. This project, following a somewhat similar concept to “Under Scan”, transformed a public space with 400 to 1,800 square metres of interactive projections. Thousands of photo portraits, which were taken on the streets of the cities where the project is exhibited, are shown using robotically controlled projectors. However, the portraits only appear inside the projected shadows of local passers-by, whose silhouettes measure between 2 to 25 metres high, depending on how far people were from the powerful light sources placed on the floor of the square” [6]. Depending on how big images were, and the ability of the participants to “bring forth” the images using their shadows, it created a great contrast in size.
This created various reactions and relationships between the participants. “Every time we show this piece, the behaviours are totally different, ranging from playful parading to erotic performances to aggressive stances … In Rotterdam, after the work had been showing for a few days, participants started using props. Breakdancers appeared. People brought their pets. A man in a wheelchair projected his shadow 22 metres high and he seemed to derive a lot of pleasure from crushing everybody around him. It is a learning experience to see how people self-represent” [7]. Thus, all his works not only create a relationship between the user and the installation, it also creates for a means of communication between those involved in the work.

So that the "Under Scan" system was able to redistribute the portraits, the computerised severs needed to be stopped. During this time, the lights as well as the projectors were turned off. This period was called the Interlude. This occurred every 7.5 minutes, and in this time, the tracking mechanism was revealed to the public. It was displayed as a number of grids, as seen in the picture to the left. Though this was never meant to be an important piece of the installation, it was found that the Interlude provided a breather or “a moment when participants could suspend their disbelief or their act of faith” [8] and turned out to be quite popular and enjoyable. Audiences found it intriguing to be able to go behind the scenes of the installation to see the basis of how it worked [9]. This again allowed for more interactions between the viewers themselves, as they discussed what they were seeing, as well as interaction between the viewer and the installation, as they comprehend how they are being tracked within the installation site.
“I am interested in how local people use their public spaces and how they relate to each other in them. Under Scan will create an interactive shadow play where members of the public become actors” [10]. Lozano-Hemmer’s works not only requires the audience to interact with the installation, but also allows for interaction between each other, which ultimately makes the work more engaging. “Under Scan” was able to bring a different sense of interactivity, as it caused the participants in move around differently, as the way they moved in the space of the installation was different from the normal movement in public space. “This is perhaps due to the fact that the original concept behind the work was centered on the idea of the shadow, which is directly linked to body movement” [11].
The general nature of the installation called for interactivity as it brought out the curiosity within. Older people took a contemplative (observer) role, while younger adults and children were more playful and engaged with, and curious about, the piece. The Interlude, on the other hand, offered another type of experience. The participants’ behavior switched from a calm/contemplative attitude to a more active and intuitive way of behaving as they reacted to the realisation that they were actually being tracked [12]. Overall, Under Scan was able to allow Lozano-Hemmer to see how people do use their public space and how they do relate to each other as the nature of his work/s instigates and encourages interactivity from all those involved.

REFERENCES

[1] Rafael Lozano-Hemmer Website: http://www.lozano-hemmer.com/einfo.html
[2] [4] [5] [7] Glandman, R. (2006) “Rafael Lozanon-Hemmer at ARS Electronica”, Akrylic. http://www.akrylic.com/articles/34/1/Rafael-Lozanon-Hemmer-at-ARS-Electronica/Canadian-Art-Vol-19-No-4-Winter-2002.html
[3] [6]Rafael Lozano-Hemmer Website: http://www.lozano-hemmer.com/eproyecto.html
[8] [11] [12] Walker, S. et.al. (2007) “Ethnographic Interventions: A Strategy and Experiments in Mapping Sociospatial Practices” http://www.humantechnology.jyu.fi/articles/volume3/2007/mounajjed-peng-walker.pdf
[9] Audience interview: http://www.youtube.com/watch?v=Bfn14sLJmyU (7:10)
[10] “News from arts council England, east midlands” Issue 16. http://www.artscouncil.org.uk/documents/newsletters/phpuJBaWK.pdf

IMAGES

Under Scan Images:
Left:http://www.bbc.co.uk/northamptonshire/content/images/2006/01/30/under_scan_body_250x180_180x250.jpg
Right: http://www.threecitiescreate.org.uk/_images/1underscan074.jpg

Body Movies Image:
http://www.akrylic.com/articles/34/1/Rafael-Lozanon-Hemmer-at-ARS-Electronica/Canadian-Art-Vol-19-No-4-Winter-2002.html

Interlude Image:
Screenshot from YouTube video: http://www.youtube.com/watch?v=Bfn14sLJmyU

Short Research Report - Alex Davies

This short report reviews and discusses the interactive installations created by Australian artist Alex Davies, in particular four of his installations: Dislocation, Pugilist449, Drift and Reflux. 

Alex Davies studied a Bachelor of Fine Arts with Honours between 1995 and 2001, in 2006 he continued his research at The University of New South Wales.  His art practice involves sound and time-based image production, spanning a diverse range of media including film, network, real-time audio-visual manipulations and responsive installations.”[1]

Dislocation

What is Dislocation?

This installation combines a mixture of audio, visual deception and audience participation. The installation combines the virtual and physical realities together by ‘dislocating’ the participant’s location in the installation space and placing the participant in a series of pre-recorded video clips. 

How does the installation work?

The interactive installation provides the participant with 4 small holes that are cut into a wall.  Each hole displays a monitor that displays the room the participant is standing in from the rear end.  The participant can see the other three participants peering into their own small hole, at this point an illusion of mixed reality is synchronized behind the real time visual images displayed on the participant’s monitor.  Such illusions could include a person entering the room with a dog; initially the participant’s first reaction is to look behind them with an expectation to see a person and a dog. 

What sorts of computing technologies are used in the installation?

Alex Davies primarily used Max/MSP and a green screen environment.  The artist used Max/MSP to program the sequencing of virtual and physical realities, using CCTV cameras during the pre-recorded scenarios and during the installation.  Max/MSP was programmed so that it would choose different scenarios based on the audience’s location within the installation space.  For example, if three people were peering through three different holes in the wall, the program would track and determine that the fourth hole is not being used and so executes a scenario where a person enters the same installation space and peers into the fourth hole giving an illusion that another person is standing next to the participants.

Did the installation achieve its primary goals?

According to the artist, the interactive installation was a success because it allowed the participants to experience a virtual world where they became the subject.  The installation goals at first seemed easy to execute and perform, but the artist commented that trying to make the pre-recorded scenarios as ‘real’ looking as possible was quite difficult because many of the scenarios relied heavily on the participant’s location within the environment.  If a participant stood close to the rear of the room, they would collide with the virtual ‘phantoms’ (that is, the pre-recorded scenario).

I think the installation proved to be a success because participants enjoyed the experience and were compelled to introduce their friends and family to the installation, which signified that the integration of the virtual reality with the physical reality was very much ‘real’ enough that the participant was compelled to show others of their new found discoveries and experiences.

Pugilist449

What is Pugilist449?

Pugilist449 is a video production installation where the audience is taken into a boxer’s viewpoint during a one-on-one boxing fight.

How does the installation work?

The installation is screened on a white screen, with full surround sound speakers.  The audience sits in the middle of the installation similar to a movie cinema.  The artist wears a custom made boxing helmet with 3 CCTV cameras attached and various microphones placed in and around the helmet.  All the sound and visual information is captured onto high definition digital cameras and edited with time-distortion effects such as slow motion.

What area of interactive art is the artist exploring?

The installation explores the area of space within an environment; the artist is exploring the fusion that is created between the audience and the boxing opponent.  The artists’ aim is to immerse each member in the audience with a distinct feeling of space invasion from the boxing opponent.  For example: every human being has an imaginary space ‘bubble’, each person’s ‘bubble’ varies in size depending on their personalities, whether quiet and shy or outgoing and communicative.  If a person’s personality is quiet and shy their space ‘bubble’ will be very large as they avoid contact with other people’s ‘bubbles’, however a person that is very outgoing and communicative will alternatively have a smaller space ‘bubble’ as they can handle with close proximities with other people.  In the same way with Pugilist449, the artist creates a pre-determined space ‘bubble’ between the boxing opponent and the victim (audience’s point of view).  With constant physical and metaphorical punches towards the victim, the artist attempts to explore the way certain people react to this space ‘bubble’ intrusion through time edited audio and visual aids.

Drift

What is Drift?

The Drift interactive installation is an outdoors site-specific installation, the installation was exhibited during a cruise voyage, and the interactive object was disguised as a normal sightseeing telescope.  The enhanced interactive object allowed the viewer to view surrounding areas through a manipulation of time and area.

How does Drift work?

Alex Davies worked closely with another interactive installation artist Daniel Heckenberg.  The installation works by the viewer panning the telescope from the front of the boat to the rear of the boat.  When the viewer was looking towards the front of the boat they would see reality, in normal time and with no manipulations, as the viewer panned across the scenic landscape towards the rear of the boat, time distortions and images are manipulated to show the viewer past events during the cruise voyage.

What was the artist’s aim for this interactive installation?

Comments from Alex Davies’ website outline his aim for Drift to be a “fluid and elastic manipulation of time in a panoramic visual display [altering] individuals’ perceptions of the journey by being able to examine aspects of the past in significant detail” [2].

Did the viewer experience the artist’s intentional aims?

I think the viewer was able to fully experience the artist’s intentional aims, by allowing the viewer to control how they manipulated time by panning from the front to the rear of the boat whilst at the same time, being able to explore past events that they had never seen.  I think the telescope was a very good choice of objects, because when the viewer was looking through the telescope, they were limited by what they could see, when the viewer looked towards the rear of the boat, the telescope provided the viewer with events that had happened whilst the viewer was looking towards the front of the boat.

What areas does the artist explore in this interactive installation?

The artist explores areas on the “movement of both individuals and media [and how they] are mapped across time and space” [2].  The artist’s exploration and development of the telescopic time manipulating object stretches our mind and perceptions of the physical reality.  The installation is unique because the images and past events that are shown in the telescope are taken from the physical reality, rather than from a pre-made collection library of images and videos.  I enjoy this particular area of art through interactive installations because each time the viewer peers through the telescope they can experience something new and unique rather than a collection of scenarios, similar to the Dislocation installation.

Reflux

What is Reflux?

Reflux is an interactive installation where participants interact with a child’s doll.  Basically, the participant shares a memory with the doll, and the doll responds with a recorded memory that was recorded earlier by someone else, using time to distort and fragment the memories throughout the dolls’ life cycle (exhibition duration).

How does Reflux work?

Reflux works by installing a number of different sensory inputs into the interior of the doll, the artist installed 2 microphones, one in each ear of the doll to record the memories given by the participants, a mercury sensor was also installed to detect when a participant was interacting with the doll, which switched on the recording equipment.  Each time a participant recites a memory; the doll records the memory in a computer database and responds with a random memory that was recorded earlier by another participant, as the exhibition time progresses, the doll’s data memory begins to fragment, similar to the common medical condition amnesia.

What sorts of computing technologies are used in the installation?

Similar to Dislocation, Max/MSP is used to trigger and record the audio memories.  Using the installed mercury switch, Max/MSP is programmed to switch on recording software and log each incoming memory with a unique tag and store it in a database.  Alex Davies also used Max/MSP to control the choice of memories from the database that the doll would reply.

How did physical time affect the doll’s responses (audio recordings)?

The doll was programmed with a set of limitations, variables and statements.  The doll was created so that it would have a limited lifecycle similar to a human being, except the doll’s lifecycle would end at the end of the exhibition. Throughout the exhibition as time passed and as the doll recorded increasing amounts of memory data, the program started fragmenting and losing the data so that the doll’s responses became less fluent and sometimes just noises and tones.  This fragmenting and data loss explored the way in which human beings lose data and memory, as they get older.

What did I learn from Alex Davies’ interactive installations?

By studying these four interactive installations: Dislocation, Pugilist449, Drift and Reflux I gained a better understanding and awareness of what sorts of technologies and possibilities are available with regards to audio, visual and the process of interacting with it.  I particularly liked the Drift installation, whilst it was a site-specific installation, each time the participant interacted with the telescope, they had the freedom to choose whether they wanted to view a scene in real time or warp time and space together.  I found the Pugilist449 installation to be a lit bit ‘childish’ amongst all the other complex interactive installations Alex had produced, because the installation did not require much interaction other than sight and hearing.  Although the installation was relatively simple, I think this installation would need a specific audience.

I learnt that many of the programs were created using Max/MSP, which has also shown me what is possible for my teams’ own major project.

References

[1 + Alex Davies Profile Image] (1996). "Asialink." Retrieved 02,04, 2008, from http://www2.asialink.unimelb.edu.au/arts/residencies/VApastresThailand.html

[2 + Installation Images] Davies, A. (15/04/07). Retrieved 10,04, 2008, from http://schizophonia.com/installation/index.htm

Critical literature review - haptic interfaces (late submission)

Apologies for the lateness, I too find it difficult to juggle both employment and uni work effectively (but im working on it).

Haptics is the study of the touching behaviour in humans, as well as how this can be incorporated with computers and technology. “Haptic interfaces can be defined as the communication with a computer through a tactile method involving a device that senses body movement, such as a data glove [1]”. “Haptic technology refers to the technology which interfaces the user via the sense of touch by applying forces, vibrations and also other motions” [2]. The many uses for haptic interfaces include educational purposes, not only to teach people the many concepts associated with haptics, but to also give them virtual environments to help with the learning process.

An example of a multi-fingered haptic interface robot (image source: http://www.jamd.com/search?assettype=g&assetid=53040767&text=haptic+interface)

Haptic technologies work by providing sensory feedback through the use of specialised hardware devices, such as data gloves and joysticks. When used correctly this hardware senses the body’s movements, enabling the user to “not only feed information to the computer but to receive information from the computer in the form of a felt sensation on some part of the body” [3].

A 3d haptic mouse (image source: http://www.3pointd.com/20070109/novint-3d-haptic-ships-at-higher-price/)

Advantages

Haptic interfaces open a new avenue of communication between humans and computers. Previously, computers could only communicate to humans through two of the five senses – those of sound and sight, but now through the use of haptic interfaces this now includes touch (see video for some examples). This allows users to actively engage in learning a vast range of skills and ideas, literally putting the control of learning in their own hands.

Disadvantages

There are a few disadvantages associated with haptic interfaces, which include extremely complex applications which often require highly specialised hardware and loads of processing power, not to mention the high cost that may be associated with obtaining the necessary equipment.

[1]Webopedia Computer Dictionary, “Haptic Interface”,

http://www.webopedia.com/TERM/H/haptic_interface.html

[2]Wikipedia Online Encyclopaedia, “Haptic Interface”, http://en.wikipedia.org/wiki/Haptic_interface

[3]Webopedia Computer Dictionary, “Haptic”, http://www.webopedia.com/TERM/H/haptic.html

Friday, April 4

Critical Literature Review - Robotic Interaction and Cyborgism

Robotic interaction and cyborgism relates to extending the capabilities of humans, through artificially enhancing and the use of artificial limbs. Robotic interaction involves the use of humanoids, intelligent vacuums and other such devices, which are capable of being configured. They are therefore mechanical extensions which can possibly be interactive. Cyborgism is the extension rather than improvement of a human’s physical and mental impulses. This includes but is not limited to external limbs and walking robotic exoskeletons. This is usually accomplished by embedding responsive sensors into the body. As there is anatomical modification, ethical have been raised from the introduction of robotic interaction, and especially with cyborgism.

Robotic interaction has many sub categories. Robotic interaction can either be configurable to complete a set task, or work neutrally imitating the notions of the human brain.

Human enhancement refers to attempts to overcome limitations of the human body via an artificial means. Technological means are used to select or alter human characteristics, whether or not they exceed the existing range of human capabilities. This is particularly useful for giving people with disabilities a quality of life up to the standard of able bodied persons. An example of this would be attaching a prosthetic limb to a person who had their original limbs amputated, and then connecting it to respond to their brain’s impulses.

http://tardis-db.co.uk/2008/02/data-streaming-mind-controlled-human.html

Humanoids refer to any structure which resembles the structure of the human body. Humanoid robots are therefore robots with an overall appearance of a human. Humanoid robots are autonomous in that they have the ability to adapt to their environment and interact in its environment and have the ability to solve simple problems.

Because of these abilities, humanoid robots are usually more technologically advanced than other kinds or robots. They possess qualities which allow them to sense, actuate, plan and control; rather than having a human input directions controlling its actions.

Humanoid robots were initially created to imitate actions that humans would undertake on a regular basis. These days they are also used for research purposes, as a study of human body behaviour and structure. Global commercialism and marketing has also resulted in humanoid robots being sold as toys to children, and interact as ‘pets’. In countries such as Japan, humanoid robots are even used as a means of entertainment where people pay to watch robots interact in a controlled environment.

Intelligent vacuums such as iRobot’s Roomba have made life much more convenient as they do the job that humans generally do. They interact with their environment a large contact sensor on the front of the unit, and an infrared sensor at the front centre of it. Some models have capabilities such as dirt detection and a designated home base.

http://irobot.com/sp.cfm?pageid=122

IRobot have been a major force in intelligent robots, creating other robots which have capabilities such as:

Scooba – Floor washing

Dirt Dog – Shop Sweeping

Verro – Pool Cleaning

Looj – Gutter Cleaning

ConnectR – Virtual Visiting

Cyborgism is the extension of physical or mental impulses of human. This can be achieved in the form of human limbs, walking robots and electronic exoskeletons. Human characteristics are generally included by embedding sensors into the body. Because such features are trying to simulate humans and their actions, they simply extend human capabilities rather than improve them.

The term transhumanism has been used to refer to artificial human enhancements. It is an international, intellectual and cultural movement supporting new sciences and technologies which have the ability to enhance human physical and mental capabilities.

Prosthetic limbs have helped to enhance the quality of life of people who have had to have limbs such as arms or legs removed. Not only can they allow everyday tasks that people engage in, but it can also help in the rehabilitation process.

Electronic exoskeletons are mechanical structures which offer assistance in making movements, especially during the rehabilitation process of major accidents or surgery. For example, an exoskeleton of a leg can assist in leg movements in the forward direction and keeping lateral balance. They also help to support the physical load of the person they are supporting.

Radio-frequency identification (RFID tags) are used as a method of automatic identification, where data is stored and remotely retrieved via transponders. They can be embedded into persons or other beings with the purpose of identification and authentication, improving efficiency within a system. They are also used in supermarket chains and other stores as a means of security on stock, as they are contained within a small circuit connected to other receivers which are permanently in the store. However, people have begun to create ways to overpass this type of security, such as “RFID Blockr” where a faraday; cage which stops magnetic fields; protects RFID tags.

http://www.rfidblockr.com/

Although cyborgism devices are capable of extending human capabilities, they don’t necessarily improve them. This is because a computer cannot think for itself unless information is inputted into the device and it has been trained to adapt to a situation or an environment. Therefore, it is only capable of extending the abilities of what it already knows, without the ability to exceed these traits. For example, a hearing aid can assist a person’s ability to hear sounds, therefore extending their capabilities in that field. However, it does not improve their actual hearing ability beyond that of a regular person’s hearing ability.

Cyborgism and its reliance on fringe science has proven to become controversial as many ethical issues have been raised. People have argued that devices which support cyborgism are ‘playing god’ as they are not allowing the human body to exist the way that they were created to. Religious and conservative groups have disputed the ethical value of artificially enhancing the ability of the human body. It can also be argued that manipulating the capabilities of the human body will end up manipulating the future, therefore changing the way the world functions. Despite this, this field of artificial intelligence continues to grow at a rapid rate as there is clearly a high demand for it in many areas including medical and entertainment purposes.

Assignment Task 1ii - Critical Literature Review: Haptic Interfaces

Hahah took me a while to remember my login, I kept on typing my Hotmail account!

Assignment Task 1ii - Critical Literature Review: Haptic Interfaces

Introduction

Haptics changes the way we interact with computers. Consider haptic interfaces as the connection between “the human hand and manual sensing and manipulation” [i] with machine. Haptics are used often in sync with robotics to create information obtained by actively touching and exploration of an object – known as feedback. This feedback can come from interactions between humans and machines whether it be in real or virtual space. The keyboard, mouse and trackball are much known tools. However they are known to be “passive”
[ii] as the users control everything. On the other hand, the “active” such as data gloves allow not only the user to control, but to feel objects.

Advantages

I believe that haptics will become everyday entities as it will engage humanity and improves our (virtual) lives. Haptic applications are in various streams such as in medicine, there are tasks which manipulate robotics for performing surgery, simulations for training medical staff, and help the disabled. A device can measure how an instrument moves, and when information is given to the computer, it can cause a slight push to simulate resistance.
Video games and simulators allow users to manoeuvre solid substances, liquid forms and different tools for a new type of entertainment. A sense often ignored in games, surprisingly it can improve a user’s senses and allow them to be more emerged into the virtual game dissolving the line of virtual and reality. Haptic features in games can involve joysticks vibrating as vehicles travel on rough terrain and signaling the speed by the frequency of vibrations. Humans and a controller would share the same control interface like a steering wheel, and by having authority over this, people can be more free to negotiate a task more freely.
[iii]
For designers, manipulating objects in environments would be simple if we can incorporating haptic systems into CAD. Selecting, moving, and resizing can be a time consuming act, however with a haptic technology, the accuracy and speed of the ability to take and replace the object would be more efficient. You can execute tasks in 3D, rather than on normal 2D levels. Visual art displays bring about a whole new level of art with haptic features such as touching virtual musical instruments to play sound using exoskeletal gloves.

Technology

Much money has been put into the development of the industry. A recent example is a Maglev Haptic control unit at Carnegie Mellon University which uses electromagnetics (alone costing tens of thousands of dollars). The two yellow bowls contain a joystick which the user interacts with shapes that are displayed on a screen. You can move objects, get reaction from the collision, and artificially feel the weights from the magnets. “Hollis said that it's the fastest-moving haptic technology, and can be made to vibrate so fast it produces music.”
[iv]
The mainstream consumer product that is commercially available of haptic interfaces is the wireless Wii remote. The best thing about the Wii is that you can use this anywhere within five meters of the sensor bar, allowing space for the user, and the capability to be everywhere else. The haptic ability allows you to easily navigate through the menus simply by making a ‘physical’ bump when you scroll over a button. In golf, you can feel the impact when you hit the ball; in boxing, when you get hit, the vibrations that shake your hands make you even try at it; and it even senses a tilt when you swing in bowling. So it translates our hand and body movements “into 3D spatial co-ordinates, and force/direction of acceleration data, which it then transmits to the Wii's processors for input data.”[v]



This short film shows how maybe people in the future would be too reliant on technology. This particular piece shows the extent of how haptic technology can evolve into real life.

Future

We are fed by our lust in creating things that were deemed impossible a decade ago. Where would the future of haptics interfaces lead us? Even now, I search for haptics, and there are articles which connect it to ‘virtual sex’ – and claiming it would be better in years ahead with the extra sensory responses placed in the application.
Would all our newly graduated doctors have only practiced on precision haptic applications achieving an accurate sense of touch? There is a $1.3 million Da Vinci system already from Intuitive Surgical which allows robotic arms to be controlled by doctors allowing them to operate.
And in gaming, would devices turn into a full body simulation where how you move in reality determines your movements in the game?

Summary

Pointing tools and GUI input devices require agility and motor capabilities to use. Haptic devices provide us with tactile clues such as simple bumps, giving us more sense in the task at hand. Thus, the human interaction is enhanced through the feedback giving us performance achieved quickly and easier. It is then possible to have haptic experiences that may not be available in the real world for it would definitely change computing.


[i] Kay M. Stanney, 2002, Handbook of Virtual Environments: Design, Implementation, and Applications, Lawrence Erlbaum Associates, Mahwah, NJ, pp. 20
[ii] Biggs, J and Srinivasan, M, Laboratory for Human and Machine Haptics (The Touch Lab), Massachusetts Institute of Technology, Cambridge, MA, pp. 93
[iii] Griffiths, P and Gillespie, B, (2005) Sharing Control between Humans and Automation Using Haptic Interface: Primary and Secondary Task Performance Benefits, “Human Factors” Vol 47, iii, pp. 574
[iv] Beschizza, R, 2008, Article: Hands-On With Maglev Haptic Control Technology, Wired (Accessed via Firefox, 2nd April 2008) (Online) http://blog.wired.com/gadgets/2008/03/hands-on-with-m.html
[v] The Wii: Truly a Wanda for Everyone? (Accessed via Firefox, 2nd April 2008) (Online) http://www.virtualworldlets.net/Resources/Hosted/Resource.php?Name=WiiWandaForAll

Critical Literature Review: Human–Robot Interaction & Cyborgism.

The term ‘robot’ in the average person’s mind conjures up images of the Hollywood style movie depictions of shiny metal (gold in the case of Star Wars’ C3P0) humanoids with uncanny human-like characteristics and behaviours. When in reality the term ‘robot’ is used to describe any mechanical, artificial agent, which, by its appearance or movements, conveys a sense that it has intent or agency of its own [1]. Now this leaves scope for quite a variety of things that could be classed as ‘a robot’ and this ambiguity is a problem when it comes to actually deciding what is and what is not a robot. In general terms robots are said to have most but not necessarily all of the following properties [2]:

• Is not 'natural' i.e. artificially created
• Can sense its environment, and manipulate or interact with things in it
• Has some degree of intelligence or ability to make choices based on the environment, often using automatic control or a pre-programmed sequence
• Is programmable
• Moves with one or more axes of rotation or translation
• Makes dexterous coordinated movements
• Appears to have intent or agency

Major research in human – robot interaction (HRI) has been to create robots with a more natural intelligence and behaviour; to enable them to act and respond in a more life-like, human manner. The prime example of this can be seen in the development of humanoid robots such as Honda’s ASIMO [ASIMO Image]. According to Honda - ASIMO is a 120 cm tall people-friendly robot that is capable of operating freely in the human living space [3]. Honda has gone into some depth explaining the reason behind the physical design of ASIMO - revolving around the robots height. ASIMO’s size “allows the robot to operate light switches and door knobs, and work at tables and work benches. Its eyes are located at the level of an adult's eyes when the adult is sitting in a chair. A height of 120cm makes it easy to communicate with.”

ASIMO’s two eyes not only serve a purpose for the robot such as facial recognition, environment recognition etc but also serve a purpose in the human side of things. Humanoids are anthropomorphic robots. Duffy states that the ‘exterior design of the robot has therefore aimed to define its functionality and facilitate its assertion of its identity as distinct from that of a mere object’ [4]. When we talk to a human we talk to their face or more specifically we look at their eyes. What it feels like is for a genuine interaction between a robot and ourselves we feel the need to make them anthropomorphic – to give them a face or even a personality. This can be seen in other examples including SONY’s AIBO robotic dog toy and PINO the humanoid. As highlighted in the PINO research, “the aesthetic element [plays] a pivotal role in establishing harmonious co-existence between the consumer and the product” [5].

As the advancement in robot AI becomes more and more lifelike you have to wonder whether ethics has a role in robotics? If you give a robot a personality/identity should you think of them differently from other technical machines? Should we give robots the ability to ‘feel’? Roboethics is study surrounding this topic and has highlighted whether the early-envisaged ‘Three laws of Robotics’ by Isaac Asimov still have meaning today?

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Discussions within roboethics have also asked what role will these intelligent autonomous humanoids play in our society? Duffy provides some examples such as a museum guide, a corporate building butler, or a post office clerk but more importantly states that the fundamental issue is to ascertain what the robot, as a machine, is inherently good at [6]. They’re really just more elaborate forms of what autonomous machines perform today – jobs that require high level of accuracy (the museum guide robot could have access to the internet to bring up endless information and facts); jobs that a menial and boring; jobs that require speeds not possible by humans.

While scientists are striving ahead with the advancement in humanoids others are turning the science on themselves. Kevin Warrick, a British scientist and professor of Cybernetics has successfully conducted experiments that by definition have made him a Cyborg. A Cyborg is a cybernetic organism (i.e., an organism that is a self-regulating integration of artificial and natural systems).[7] For an in depth reading I suggest viewing his personal website at www.kevinwarrick.com but his pioneering experiment involved the neuro-surgical implantation of a device into the median nerves of his left arm in order to link his nervous system directly to a computer in order to assess the latest technology for use with the disabled.

While Warrick is conducting experiments on a scientific basis, cyber artist Stelarc is exploring cyborgism in a more artistic sense. He has produced many cyber influenced art pieces including Fractal Flesh in which Stelarc plugged himself into muscle-stimulation circuitry controlled by a Mac. The Mac, in turn, was connected, via the Internet, to Paris's Centre Pompidou, Helsinki's Media Lab, and Amsterdam's Doors of Perception conference. By pressing a color-coded 3-D rendering of a human body on a touch screen, participants at the three sites jolted the artist's (literally) wired body into action [8].

Depending on where you stand in term of robotics depends on how you feel about cybernetics. Those who feel that robotics implemented into the body used to better understand the human body will have different opinions from those who feel that cyborgs are people who want to improve the human body as a tool.

Where do you stand on robotics/cyborgs? Should robotics be inspired by humans in order to realise sophisticated machines, or use machines to better understand humans?




[1] [2] Robots Wikipedia : http://en.wikipedia.org/wiki/Robot
[3] Honda ASIMO: http://world.honda.com/ASIMO/technology/concept.html
[4] [6] Anthropomorphism and Robotics , Brian R. Duffy; Available From: http://www.prism.ucd.ie/publications/pub2002/AISB02-Duffy.pdf
[5] PINO http://www.symbio.jst.go.jp/~yamasaki/
[7] Kevin Warrick Homepage www.kevinwarrick.com
[8] FRACTAL FLESH: STELARC'S AESTHETIC OF PROSTHETICS http://www.stelarc.va.com.au/fractal/index.html

Thursday, April 3

Task 2 - Critical Literature Review: Tele-immersive Network Interaction


Tele-immersion could be the future of the world; it has the ability to revolutionize the way we communicate for work, education and day-to-day life. Tele-immersion is an innovative technology that allows people to communicate as though face-to-face when they may be geographically situated far from one another.

How does it work?
Tele-immersion requires several cameras, arranged in a semi-circular shape around the room, allowing a variety of camera angles to capture each individual movement of the participant. A kind of “living statue” is created from the camera scans and this is projected onto a large screen for the other participants in the tele-immersion network to see (Frauenfelder, 2001). To create a stereoscopic view and thus a totally believable scene, participants must wear polarized glasses so that the retina of each eye is able to see a slightly different image. Several other pieces of equipment are also required such as microphones, speakers and head mount displays.

Benefits of tele-immersion
The benefits of tele-immersion will not be fully recognised until its use is put into practice in a variety of fields such as education, business and general communication. However, we can envisage the kind of effect it will have on so many people just as video-conferencing has already proved to be a great success.

Video-conferencing is a tool much like tele-immersion used for collaboration between people who may not be situated within the same area. It uses real-time transmission of audio and video but it does not provide the same experience as tele-immersion as it does not give the effect of being in the same room as the other participants as only one camera angle is usually used thus not creating a three-dimensional effect. Nevertheless, video-conferencing has recently taken off and is now commonly being used for business, as well as in other fields. The first major benefit of video-conferencing and thus tele-immersion is that it considerably more economical for a group of people to communicate over the internet than for a number of people to have to travel to a particular place to conduct a meeting. Although there may be initial costs involved with tele-immersion such as computers, cameras, speakers etc. the costs involved with actual face-to face meetings are substantial (e.g. travel costs, accommodation) and also endless assuming that the meetings will take place on more than one occasion. Secondly, tele-immersion has major benefits as it alleviates participants from travelling to a meeting point to communicate the same message. This is of particular significance for people with disabilities or who are less mobile for the reason that with tele-immersion they can collaborate with others from the comfort of their own home. Similarly, the convenience that tele-immersion provides, allowing people to communicate from such different locations, encourages further collaboration because of its simplicity and accessibility.

So why isn’t everyone using it now?
While the benefits of using tele-immersion can be seen there are significant pitfalls that need to be overcome before it will become more prevalent. Firstly, tele-immersion requires a huge bandwidth because it generates such large amounts of data per second. While the large amounts of data is also quite taxing on computer processing power, the principal concern is that the data must be transmitted quickly to allow uninterrupted real time collaboration between the tele-immersion participants. Secondly, the costs involved with tele-immersion are quite high during initial set-up. Although the ongoing costs are minimal, at the current price of the equipment required it is unrealistic for individuals or groups that are not part of large companies to be expected to purchase it.

Uses for tele-immersion
As the use of tele-immersion becomes more widespread education will be one of the areas that will benefit significantly. Tele-immersion networks will be able to be used for schools where the students are not all located in the same area. For instance, students of the School of the Air would be able to keep in touch with fellow schoolmates face-to-face and with their teacher encouraging greater collaboration and thus further education. Tele-immersion could also be used to keep in touch with exchange schools overseas – becoming the modern day version of a pen pal which would promote further study of other cultures and languages.

Some businesses no doubt are already using tele-immersion to conduct business meetings with international offices however as tele-immersion advances are made and its use becomes more dominant a greater global marketplace would be created. Businesses around the world would become more accessible thus not restricting clients to companies within a certain range and this therefore would increase competition consequently benefiting the customer.

Everyday communication with family and friends will be completely revolutionised, as tele-immersion will further encourage social networking. After the initial costs, keeping in touch via tele-immersion will become as simple as sending an email today.


As bandwidths increase and computers become more powerful tele-immersion will become more prevalent in our society. Until then further research is required into such technologies to make networking more simple, attractive and less confronting to the everyday person.


Ditlea, S., 2001, Tele-immersion: Tomorrow’s Teleconferencing. (Accessed via Safari on 26th March 2008). (Online). http://www.cs.unc.edu/Research/stc/inthenews/pdf/CGW_2001_jan.pdf

Frauenfelder, M., 2001, The Year iin ideas: A to Z; Tele-immersion. (Accessed via Safari on 26th March 2008). (Online). http://query.nytimes.com/gst/fullpage.html?res=980DE4DD143CF93AA35751C1A9679C8B63

University Of Pennsylvania, (2002, November 20). Tele-Immersion System Is First "Network Computer," With Input, Processing And Output In Different Locations. ScienceDaily. (Access via Safari 26th March 2008). (Online). http://www.sciencedaily.com­/releases/2002/11/021120072242.htm

Image:
Fuchs, H., & Towles, H., 2005, Group Tele-Immersion. (Access via Safari 28th March 2008). (Online). http://www.cs.unc.edu/Research/ootf/Projects/groupti.html

Critical Literature Review: Interaction Design

Note: Apologies for the brevity of this. I fail at work/uni balance :(

Interaction Design

The field of Interaction Design (IxD) is defined by Wikipedia as:
The discipline of defining the behaviour of products and systems that a user can interact with. The practice typically centres around complex technology systems such as software, mobile devices, and other electronic devices. However, it can also apply to other types of products and services, and even organizations themselves. Interaction design defines the behaviour (the "interaction") of an artefact or system in response to its users. (Wikipedia, 2008a)
Interaction Design is an inherently multidisciplinary field having ties to Human Centred Computing Interaction (HCC), industrial design and graphic design. By using human factors – made up of the fields of usability and psychology – interaction designers can follow an informed, iterative process to improve the user experience and usability of interactions through research, observation, and qualitative and quantitative analysis of various metrics.

An Interaction Process

An Interaction Designer will begin this process by researching users to gain an understanding of their goals and needs. This is achieved through the use of questionnaires, interviews and observation.

Some questions which an interaction designer tries to answer include (adapted from Wikipedia, 2008b):

• Who are the users of the product?
• What are the users’ tasks and goals?
• What are the users’ experience levels with the product, and other products like it?
• What functions do the users need from the product?
• What information might the users need, and in what form do they need it?
• How do users think the product should work?

The results from these studies can then be compiled into qualitative analyses where themes and issues can be identified through comparisons of users; and also into quantitative analyses where these issues can be ranked and values assigned.

The second stage is the creation of an interaction which the interaction designer expects users to follow. This process is guided by the design research, and a collaboration of many different design, science and engineering fields – Human-Computer Interaction, User Experience Design and Psychology. This stage creates the structure of the interaction, the steps involved and the elements needed to create the interaction.

Finally, these elements are combined to create a prototype interface for the interaction. These may be low-fidelity prototypes such as wireframes, sketches and role-plays. This enables potential-users to test fully featured versions of the interaction, without being fixated on the final design. The first stage is then repeated on these designs to provide data on its effectiveness.
During the low-fidelity prototyping stage of this process, interaction designers can make informed changes quickly and easily. These changes are then retested to check for improvement in their effectiveness.

When satisfied with these results, the interaction designer can continue on to the creation of high-fidelity prototypes, which can appear to be working versions of the interaction. These can help test the product in a more complex way, providing more detailed information on the effectiveness of a design solution while also allowing iterative changes and testing to be made without massive expense.

Finally, the product is implemented as a final solution -although the task of testing and minor changes may continue later into the product-development cycle. These final-stage changes tend to be limited to any remaining issues remaining in the product due to the expense and difficulty of reimplementation.

Summary

Interaction Design has become a major part of modern technology. Interaction designers are employed to develop the interfaces on modern mobile phones, operating systems and applications. Interaction designers also progress the field by researching new ways of interacting through haptic interfaces such as Microsoft Surface, gestural machine-vision based interfaces and speech recognition.

Interaction design is also involved in taking the field of computing into new territory by creating physical interfaces to interactions – for example by manipulating physical objects.

References
Interaction Design, Wikipedia (2008a). http://en.wikipedia.org/wiki/Interaction_design
Usability, Wikipedia (2008b). http://en.wikipedia.org/wiki/Usability
Knemeyer, Dirk and Svoboda, E (2005). User Experience - UX. Interaction-Design.org http://www.interaction-design.org/encyclopedia/user_experience_or_ux.html
Apple Human Interface Guidelines, Apple Inc (2008) http://developer.apple.com/documentation/UserExperience/Conceptual/OSXHIGuidelines/XHIGIntro/chapter_1_section_1.html
User Centered Design, SAP AG (2006) http://www.sapdesignguild.org/resources/ucd_process.asp

Assignment 2 Critical Essay Robotics Interaction and Cyborgism

Robotic interaction + cyborgism

Cyborgism, Cyborgology and cyborgs are cybernetic organism which is an organism that is combining artificial and natural systems or it can be defined as “the melding of the organic and the mechanic, or the engineering of a union between separate organic systems” quoted by Gray, Mentor, and Figueroa-Sarriera. Robotic interaction is the study of a robotic interacting with a human or another robot. These robots will be able to have facial expressions, speech recognition, gestures and even a personality. [1]

Cyborgism is a human to have robotic features. Many movies such as The Terminator series, Bicentennial Man, AI and Star Wars have given the idea that cyborgism is a robot which has the brain of a human. They are able to think and have emotions like humans. However our technology has not advanced to that level. Cyborgism today can be recognised as the technology a quadriplegic patient would use or when an athlete has had to have a limb replaced, the artificial leg or arm is a cybrog. [2]

The relationship between cyborgism and robotic interaction is that the research to create robots which reflect and resemble humans in a physical and mental way is becoming more demanding. However, humans are very unique species; they have two distinct features; unique physical nature and emotions / feelings. To implement this into a robot is very difficult because to simply model the movement of a human is difficult, let alone to model the emotional features of humans. Another factor to consider is that when a robot is interacting with another human or object, it will keep a safe distance between the object and itself. This applies to humans as well. Two humans interacting together, they have a certain distance between them which makes them feel more comfortable. Robotic interaction is more than just navigating a human and interacting with them. The robot needs to understand and recognise human social behaviour and act in a way which will not cause conflict. Research in robotic-human interaction is developing in understanding how to make a human feel more comfortable with the presence of a robot. With this research, scientists are trying to develop robots which will be helpful in the workplace and at home. [3]

An example of how a robot can be used in at home is in the movie “The Bicentennial Man”, a robot was designed to do housework. It acted as a nanny or carer. During the movie, the robot started to become interested in how humans interacted together, hence his ambition to “turn” into human began. He first got human skin made for him and then his organs where modelled for him, in the end he became mortal and was the longest living human in history. Now, for that to be possible in the real world is still a question. But it does raise the issue that if these robots were to really become available to humans and they wanted to become a human and they did, are they really considered human?

Firstly, robotic-human interaction needs to be more than simple speech recognition. When a human is communicating with another human, they rely heavily on facial expression and body language. Non verbal gestures and eye contact play an important role in visual feedback. It will regulate the flow of conversation and provide a good interaction between the two people. Robots will need to be able to recognise and produce non-verbal communication cues to be an effective collaborative partner.

Common interactive robots which have been designed are robots which are placed in museums to guide people through and to answer any questions from the viewer. Sage, who was created by Nourbakhsh et al in 1999 was placed the Dinosaur Hall at the Carnegie Museum of Natural History. Visitors at that museum would interact with Sage through basic speech recognition and a LCD screen. His speeches include humour (which is basic communication tool which humans use to interactive with other humans) to interact and engage with the visitors. Sage also had facial expressions which would express his mood when confronted with different type of questions. Sage shows not only how speech affects communication, but also how the form of speech and non-verbal communication influences how well communication takes place.[3]

Another type of interactive robot which is going into development are robots which assist with the elderly and disabled. For example, reminding and helping the elderly to take their medicine, helping them get things off shelves or assist them with the refrigerator, getting in and out of bed and inform other people that there is an emergency. The reason of designing robots which take care of the elderly is that many aged people enjoy independency and to be put into a home can be intimidating and expensive. Having a robot which assists with all the day to day needs helps the family financially, and keeps the elderly happy. [4]

With the intense research to create robots to have a human resemblance in both physical and mental aspects, the problem is what will humans do when the robots know how we think and start to become smarter than us? Do we rely too much on technology that technology one day may take in control? The idea to have a robot which humans can interactive is a good idea. But we should always look at the situation in the long terms. Robotics and cyborgism should be only researched to benefit humans which cannot be benefited in any other way, for example, the loss of a limb or a hearing aid. Before we get carried away with all this research and development, there should be a time where we stop and think what it is we really need. There is a difference between making something which is something we need and something we want. The increase of robotic use in our homes and workplace means a decrease of employment for humans. There is a consequence to a rising unemployment rate which would instead affect humans.

References

[1] [2]“Cyborgology 101” 2006
http://www.sjsu.edu/faculty/butryn/whatisa.htm#Cyborgs%20on%20the%20Web

[3] Janaka B, & Chandrajith M, & Watanabe K “Adaptive Personal Space for Humanizing Mobile Robots” http://s.i-techonline.com/Book/Human-Robot-Interaction/ISBN978-3-902613-13-4-hri01.pdf

[4] Songmin,J &Kunikatsu, T “Development of Service Robot System With Multiple Human User Interface”
http://s.i-techonline.com/Book/Human-Robot-Interaction/ISBN978-3-902613-13-4-hri07.pdf

Critical Literature Review: Haptic Interfaces for the Blind

Haptics is the study of how to combine the human sense of touch with a computer-generated world. So why develop haptic interfaces? Ivan Sutherland, a founding father of virtual reality suggested, “human kinaesthetic sense is as yet another independent channel to the brain, a channel whose information is assimilated quite subconsciously” [1]. This inevitably led to the development of haptic interfaces.

The anatomy and physiology is very important in designing haptic interfaces for a human. The hands are most commonly used in haptic interfaces- humans constantly use their hands to explore environments that are poor in visibility, particularly for those visually impaired. The fingers are one of the most sensitive parts of the surface of the skin, with an amazing 135 sensors per square centimetre at the fingertip. While humans are very good at identifying 3D objects with their hands, we are not as capable at identifying 2D objects. However ways of exploring such spaces are through raised surfaces on a plane, where we are able to gather information by first identifying an edge and then following a contour, illustrating the importance of touch.

Why develop multi-finger tactual displays?

The motivation of design was to explore the sense of touch as an alternative communication channel. This will act as a sensory substitution for individuals who have hearing or visual disabilities. It also serves as a new haptic interface used as an exploration to human-computer interactions through a tactual channel

The potential to receive information tactually is been well demonstrated by natural methods of tactual speech. One particularly noteworthy is the ‘Tadoma’ method used by individuals who are both deaf and blind. The method of speech reading is based on vibrotactile reception of the articulatory movements and actions that occur during the creation of speech [2]. The hand of the individual is placed over the face and neck of the talker, so that the thumb is positioned lightly on the lips and the fingers fan out over the cheek and neck.

This simple method produces remarkable abilities to experienced Tadoma users. Individuals could understand every day speech, through observing a variety of actions associated with speech production at very high levels, allowing rich two-way conversations between both familiar and novel talkers.

What makes Tadoma successful is that it is received by hand. A talking face is eternally rich, concurrently displaying various stimulating qualities that engage both the kinaesthetic and cutaneous sensory systems. Trying to recreate this in tactual devices has been mostly unsuccessful, with one major problem being the nature of the output display. These displays are commonly composed of multiple stimulators that send a moderately homogenous cutaneous stimulation, with few distinguishing perceptual qualities [3].

Several studies conducted by Reed and Delhorne [4], show that- long term users of artificial tactile aids demonstrated limited improvements in their speech reading abilities, while normal subjects with imitated deafness and blindness can achieve a certain level of skill in discriminating and identifying speech through the Tadoma method after a limited amount of training.

The studies found that long-term users of artificial aids had little gain, as they were observed for the reception of words in isolated sentences, only averaged around 25%. The benefits of current wearable tactual devices to speech reading appear to be limited, even after subjects had worn these aids for several years.

Studies with inexperienced users of the Tadoma method suggest that some ability to receive speech through the tactual sense alone can be obtained through limited amounts of training. In an analytical study of the Tadoma method conducted by Reed et al, reported that with training of 100 hours each, two inexperienced Tadoma users performed in a way comparable with that of an experienced Tadoma user on consonant and vowel identification tasks. They also demonstrated the ability to receive sentences constructed from a limited vocabulary [4].

The difference between Tadoma users and users of artificial aids may reflect differences in the amount of information available in a talking face, as compared with that available from current artificial tactual aids.

The New Age: FEELit Mouse

Recognition of the need for richer tactual displays is now prevalent. Rarely new technologies have emerged that have the potential to greatly enhance the computing experience, particularly those visually impaired. The FEELit Mouse, designed by Immersion Cooperation is one such technology. Whilst most blind computer users have a screen reader combined with synthetic speech and or Braille display, giving them access to text on the screen, but not to the graphics. But with the FEELit Mouse, it grants the remarkable ability to enable computers to convey realistic sensations to users, representing an essential milestone in the evolution of human-computer interaction. It is a sophisticated ‘information technology’, allowing users to ‘touch’ their software, taking advantage of the same tactile sense and physical instinct that guide and inform us in real world interactions [5].

Traditionally, a mouse is confirmed only through direct visual feedback. As a result, cursor-targeting tasks demand significant visual attention, not only slowing execution, but is unusable for those visually disabled. The FEELit Mouse can manipulate the interface, feeling each encounter between the cursor and the items visually displayed on the screen e.g. the edge of a window feels like a groove carved into the desktop and when the cursor slides into the groove you feel a forceful physical engagement. Tasks such as resizing the window will be effortless.

The FEELit Mouse has Important Network Implications making it also a valuable Internet tool, allowing users to feel elements drawn within like HTML web pages. It enhances targeting, manipulation and exploration. One additional benefit of the FEELit Mouse that is exceptional is allowing multiple users to interact with each other through feel. Imagine exploring a shared environment, and being able to interact with other users through direct physically contact. Being able to feel the actions of others greatly enhances communication and understanding, extending the idea of computer access for blind persons, from being inaccessible to bringing it to a whole new level.

[1] Smith, C (2004). “Human Factors in Haptic Interfaces”, http://www.acm.org/crossroads/xrds3-3/haptic.html
[2] Reed, C M. “The implications of the Tadoma method of speechreading for spoken language processing”, http://www.asel.udel.edu/icslp/cdrom/vol3/1002/a1002.pdf
[3] Tan H Z and Rabinowitz W M (1996). “A Multi-finger Tactual Display”, http://www.ecn.purdue.edu/HIRL/projects/papers/tactuator/C19_Tan_ASME1996.pdf
[4] Tan H Z et al. (1999). “Information transmission with a multifinger tactual display”, http://www.ecn.purdue.edu/HIRL/projects/papers/tactuator/J08_Tan_PP1999.pdf
[5] (2005). “FEELit Mouse”, http://www.cwhonors.org/search/his_4a_detail.asp?id=592

Critical Literature Review - Tele-immersive Network Interactions

Tele-immersive network interaction refers to the collaboration of people, who are geographically distant, through the use of multiple cameras at each location, whose inputs are reconstructed into a 3D virtual reality. Participants from all locations are able to see themselves, as well as the others they are working with, actually moving and interacting with each other in real time in a shared virtual environment.

Participants are generally represented as realistic computer generated depictions of themselves. The environment they share can be anything from a realistic everyday room, such as an office, to a large endless plane. Tele-immersive interactions generally have real world objects and/or settings captured in real time rendered in the virtual environment. What makes tele-immersion different from the usual virtual reality is that “the environment transmits gestures as well as audio and video, so users have a greater sense of presence in the shared space than they would with other collaborative media” [1].

Tele-immersion creates a dramatic new medium for groups of people who are physically separated from each other to work together and share experiences collectively in an immersive 3D virtual environment, just as though they were all within the same physical room. The ultimate goal of tele-immersion is not to reproduce a real face-to-face meeting in every detail, but “to provide the "next generation" interface for collaborators, world-wide, to work together in a virtual environment that is seamlessly enhanced by computation and large databases” [2]. The potential inherent in tele-immersion will revolutionise the way we communicate and collaborate with one another.

Tele-immersive networks would bring many advantages in fields which require group collaborations and/or interactions. An already popular form of communication in the business world is videoconferencing, using internet telephone technology and video cameras to bring together people from widely spread destinations on video monitors. However, the ability for people to interact in these situations is restricted to a certain degree. Tele-immersion takes the concept of videoconferencing a step further. The interactive nature of tele-immersive networking would allow business men to physically interact with each other through their virtual representations, creating a stronger sense of a conference or meeting atmosphere. This technology would allow for them to meet more frequently as they would not have to go out of their way to travel to certain locations, there wouldn’t be extra expenses due to travel costs or to hire out a room if it were to be a big conference meeting. Videoconferencing would then be made a lot easier and efficient if done so using tele-immersion.

Also, as tele-immersion technology advances, the pace of surgical innovations has increased dramatically. The need for safer means of training surgeons in various forms of surgical practices has shown another advantage from the further research which has gone into tele-immersive network interactions. The conventional video recorded lessons, used for surgical training, have also long been available to help surgeons learn new procedures. However, these videos are generally seen as only slightly effective due to a number of reasons: the fixed point of view that is integral to the narration, lack of depth perception and interactivity, and the little bits of missing information which the camera does not always pick up but could end up being vital [3]. Overall, the experience of watching a video is just not as effective a learning method as being there and seeing the procedure done in person. “Tele-immersion would allow trainees to freely and naturally walk around a life-sized, high-fidelity, 3D graphical reconstruction of the original time-varying events, pausing or stepping forward and backward in time to satisfy curiosity or allay confusion” [4], thus making it a much more effective and safer means of training for surgical procedures.

Furthermore, the time zones and distance of researchers have caused a limit to the productivity and the efficiency of collaborative research projects. Using tele-immersive networking technology would prove to be more productive because it would eliminate the barriers of space and time so researchers will be able to interact with each other, synchronously or asynchronously, without the interruptions and expenses of travel or time-dependent studies. Researchers from one location would be able to work on and manipulate the project and then leave the environment, so that researchers in another location could enter the environment at their appropriate times and see and interact with the changes that have been made. “In tele-immersion, participants are not talking about a storm; they are standing inside it. They are not looking at a scale model of a car design; they are standing inside the engine block. They can then change parameters on a supercomputing simulation of the storm or engine block and collectively study the impact of those changes. With these enhanced capabilities, collaborators may choose to work virtually even if more traditional face-to-face meetings are possible” [5].

Tele-immersive network interaction would prove to be a major advantage in areas which require collaboration and interactions from participants who are geographically distant from one another because having physical body representations can be very helpful in aiding conversation and understanding in the virtual space as you can see where your collaborators are, and what they are looking at or pointing at. Tracking the user's head and hand position and orientation allows articulated avatars to transmit a decent amount of body language, and are very useful in task oriented situations [6].
However, in order for tele-immersion to be at its most efficient, more research needs to go into making it more commercial. At the moment, current bandwidth limits, camera quality and processing speeds of everyday computers are not strong enough to support the use of tele-immersion in everyday communicating activities. Nevertheless, when the technology is readily available and cheap enough to be made commercial, the way we communicate would be revolutionised, allowing newer and easier ways of communication to occur.

[1] [5] Leigh, J et. al. (1999). “Visualization in Teleimmersive Envinronments”, Scientific Visualization, http://www.evl.uic.edu/aej/papers/computer99.pdf
[2] [6] Imai, T. et. al. (2000). “Overcoming Time-Zone Differences and Time Management Problem with Tele-Immersion”, http://www.startap.net/startap/images/PDF/timezone.pdf
[3] [4] Brown University Computer Graphics Group - Applications, http://graphics.cs.brown.edu/research/telei/teleImmersionApplications.html