in IEEE Symposium on Visual Language (VL'99) Conference Proceedings, (Tokyo 1999), pp. 90-95.
reprint only with permission by the authors.
 

We are artists working on the creation of interactive computer systems that use the audiences’ participation as essential input for the creation of image structures within the systems [1]. In 1998 and 1999 we created two interactive web sites for the Internet called VERBARIUM and LIFE SPACIES. These two web sites allow users on the Internet to write text messages that are instantly translated into visual three-dimensional forms. Our custom designed text-to-form editor takes the letters, syntax and sequencing of a text message as genetic code and updates the design functions for the creation of three-dimensional forms according to the text message parameters. As the text messages of the different users on the Internet are usually unique and diverse, unique and personal three-dimensional visual forms and shapes can be created by the users.

1. Introduction

"colorless green ideas sleep furiously"

(Noam Chomsky)

According to Noam Chomsky, human language acquisition is based on a universal grammar that is genetically embedded within the human mind of all normal children, allowing them to learn their native languages naturally and seemingly effortlessly. [2]

It was also Chomsky who coined the above phrase of

"colorless green ideas sleep furiously," an expression that might not make much logical sense to a more scientifically oriented person, but does have quite a lot of meaning for a more visual or art minded person. Though this sentence, as Chomsky has shown, is grammatically correct, its meaning cannot be grasped through logic alone. When we hear this sentence for the first time we see pictures or forms or shapes appearing in our minds. These forms are vague, yet they are defined to a certain degree and can certainly create visual sensations and emotions. Inspired by Chomsky’s sentence and based on the idea of translating words or sentences into visual

forms, we created two interactive systems for the Internet, called VERBARIUM and LIFE SPACIES.
 
 

These systems each consist of an Internet web page, that allows users to write text messages into a text window.

Through our specifically designed text-to-form editor we are able to translate these text messages into three-dimensional visual forms and shapes. The following chapters will describe the text-to-form editor for each of the works in more detail.

2. LIFE SPACIES - from text to form on the Internet

Fig. 1 "Life Spacies" web site

The "Life Spacies" web page (Fig. 1) allows people throughout the world to interact with the system. By simply typing and sending an email message to the "Life Spacies" web site (http://www.ntticc.or.jp/~lifespacies), one can create an artificial three-dimensional creature. As soon as the message is sent it creates a creature that starts to live in the interaction environment at the ICC Museum in Tokyo [3].

Fig. 2 "Life Spacies" Interaction Setup at the ICC Tokyo

The interaction setup consists of two independent interaction sites (Fig. 2) that are linked together via a data line, allowing visitors at remote locations to be displayed and interact in the same virtual three-dimensional space.

On-site visitors can directly interact with the creatures through touching and catching them. Once a creature is caught by the visitor, it will clone itself. However, if two remotely located people are in the same virtual space, they can each catch a creature with their hands, which causes these two creatures to mate and to create an offspring by genetically exchanging the parents’ codes.

2.1. "Life Spacies’s" text-to-form editor

To create the creatures, we developed a special text-to-form editor that enables us to translate the written text of the text messages into the genetic code of a creature. The text-to-form editor is based on the idea of linking the characters and syntax of a text to specific parameters in the creature’s design.

Fig. 3 Creature with two bodies and one pair of limbs

The default form of a creature is a body composed of a sphere consisting with 100 vertices, that is, 10 rings with 10 vertices each. All vertices can be modified in x, y and z axes to stretch the sphere and create new body forms.

Several bodies can be attached to each other provided that their attachment point is located on the x-axis. If the attachment point is not on the x axis, a limb is created instead of a body; this limb is copied and the copy is attached at a position symmetric to the original position. Figure 3 show a creature with two spheres as bodies and one pair of limbs.

According to the sequencing of the characters in the text, the parameters of x, y and z for each of the 100 vertices can be stretched and scaled, the color values and texture values for each body and limb can be modified, the number of bodies and limbs can be changed and new locations for attachment points of bodies and limbs can be created. Since each of the vertex parameters is changeable and all of the bodies and limbs can be changed as well, about 50 different functions for the creature’s design parameters are available. The design functions are subsumed in a design function table (Fig. 4).

function1 stretch default body/limbs in x

function2 stretch default body/limbs in y

function3 stretch default body/limbs in z

function4 set the next stretch function to global

function5 set the next stretch function to a vertex point

function6 set the next stretch function to a ring

function7 create a new location for an attachment point

function8 copy a new location for an attachment point

function9 compose a new texture for body/limbs

function10 copy texture of body/limbs

function11 change parameters of RED in body/limbs texture

function12 change parameters of GREEN in body/limbs texture

function13 change parameters of BLUE in body/limbs texture

function14 change patterns of body/limbs texture

function15 exchange positions of bodies/limbs

function16 copy body/limbs

function17 create a new body/limbs

function18 add or replace some of the above functions

function19 randomize the next parameters

function20 copy parts of the previous operation

function21 modify life span (default is 24 hours)

function22 add the new parameter to previous parameter

function23 ignore the current parameter

function24 ignore the next parameter

function25 replace the previous parameter by new parameter

.............

function50

Fig. 4 "Life Spacies" design function table

Next, in translating the characters of the text message into

these design function values, we first assign an ASCII value to each character. This is done according to the standard ASCII table shown in Figure 5.

33 ! 34 " 35 # 36 $ 37 % 38 & 39 '

40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 /

48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7

56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ?64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G

72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O

80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W

88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _

96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g

104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o

112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w

120 x 121 y 122 z 123 { 124 | 125 } 126 ~

Fig. 5 ASCII table

Each character refers to an integer. We can now proceed by assigning this value to a random seed function rseed. In our text example from Figure 4, T of This has the ASCII value 84, hence the assigned random seed function for T becomes rseed(84). This random seed function now defines an infinite sequence of linearly distributed random numbers with a floating point precision of 4 bytes (float values are between 0.0 and 1.0). These random numbers for the first character of the word This will become the actual values for the modification parameters in the design function table. Note that the random number we use is a so-called "pseudo random," generated by an algorithm with 48-bit precision, meaning that if the same rseed is called once more, the same sequence of linearly distributed random numbers will be called. Which of the design functions in the design function table are actually updated is determined by the following characters of the text, i.e., his; we then assign their ASCII values (104 for h, 105 for i, 115 for s ...), which again provide us with random seed functions rseed(104), rseed(105), rseed(115). These random seed functions are then used to update and modify the corresponding design functions in the design function look-up table, between design function1 and function50. For example, by multiplying the first random number of rseed(104) by 10, we get the integer that assigns the amount of functions that will be updated. Which of the 50 functions are precisely updated is decided by the following random numbers of rseed(104) (as there are 50 different functions available, the following floats are multiplied by 50 to create integers). Figure 5 shows in detail how the entire assignment of random numbers to design functions operates. As mentioned above, the actual float values for the update parameters come from the random seed function of the first character of the word, rseed(84). An example of the entire procedure is given in Figure 6.

Example word: This

T => rseed(84) => {0.36784, 0.553688, 0.100701,...}

(actual values for the update parameters)

h => rseed(104) => {0.52244, 0.67612, 0.90101,...}

# 0.52244 * 10 => get integer 5 => 5 different

functions are called within design function table

# 0.67612 * 50 => get integer 33 => function 33

within design function table will be updated by value 0.36784 from 1. value of rseed(84)

# 0.90101 * 50 => get integer 45 => function 45

within design function table will be updated by value 0.553688 from 2. value rseed(84)

........ until 5. value

Fig. 6 Example of assignment between random functions and design functions

As explained earlier, the basic "module" is a sphere with white default color and no texture. When messages are sent, the incoming text modifies and "sculpts" this default module by changing its form, size, color, texture, number of bodies/limbs, copying parts and so forth. Depending on the complexity of the text, the body and limbs of the creature become increasingly shaped, modulated and varied. As there is usually great variation among the texts sent by different people, the creatures themselves also vary greatly in appearance, thus providing a personal creature for each author. Figure 7 shows an example of a short and simple email message sent to the "Life Spacies" web site.

Date: Sun, 01 Nov 1998 13:14:32 +0900

From: Christa Sommerer <christa@mic.atr.co.jp>

To: life@lc.ntticc.or.jp

CC: christa@mic.atr.co.jp

Subject: test creature1

This is a test creature.

Fig. 7 Example of email message to "Life Spacies"

2.2. Picture of the "Life Spacies" Creature

Fig. 8 Creature created by email in Fig. 7

As soon as this message is sent to the server in Tokyo, the creature starts to live in its virtual environment and the author of the text receives a picture of his or her creature in return. Figure 8 shows an image of the creature created by the text message of Fig. 7. Because the text message was rather short, the corresponding creature consists just of one body and one pair of limbs, similar to the default case but with long limbs and a heart-shaped body.

2.3. Variations in the Creature’s Design

By sending more complex messages with more characters, words and varied syntax, it becomes possible to create more elaborate creatures with more bodies, limbs and variation in body form, texture, size and color. Figure 10 shows an example and Fig. 9 is the corresponding text message.

Date: Sun, 01 Nov 1998 13:20:32 +0900

From: Christa Sommerer <christa@mic.atr.co.jp>

To: life@lc.ntticc.or.jp

CC: christa@mic.atr.co.jp

Subject: example #4

this is not a sentence, it is a creature, it is now in Tokyo, where it lives.

it is a creature, this is not a sentence, where it lives, it is now in Tokyo.

it is now in Tokyo, this is not a sentence, it is a creature, where it lives.

where it lives, it is a creature, it is now in Tokyo, this is not a

sentence.

Fig. 9 Complex email message

Fig. 10 Creature created by complex email message

3. VERBARIUM - a verbal herbarium

In 1999 we developed a second work for the Internet using the idea of the text-to-form editor. This system is called VERBARIUM and was created for the Cartier Foundation in Paris. The web site can be found at:

http://www.fondation.cartier.fr/verbarium.html.

This time on-line users can create forms and shapes directly in real-time by writing text messages with the web site’s interactive text editor. The site was created by using JAVA applet programming.

Fig. 11 VERBARIUM web page - example

Each of the incoming text messages functions as a genetic code to create a visual three-dimensional form. Depending on the composition of the text, forms can either be simple or complex, abstract or organic. The form can be viewed by the user instantly at the left-side window of the web page ( Fig. 11). Additionally, all incoming text messages become part of a collective image that is displayed on the right side of the web page (Fig. 11). This collective image functions as a "virtual and verbal herbarium" (hence the name VERBARIUM) of forms composed and created through the different text messages (i.e., verbs). The text messages can be written in many languages: the instruction editor displays instructions in about 20 different languages.

3.1. VERBARIUM’s Text-to-Form Editor

Similar to the "Life Spacies" text-to-form editor, we link the characters and syntax of a text message to specific parameters in the forms’ design. This time the default form is a ring with 10 vertices. This ring can be extruded in x, y and z axes, and during the extrusion process the rings’ vertices can be modified in x, y and z axes as well. Through addition and constant modification of the ring parameters the whole structure starts to grow, branch and develop. Different possible manipulations, such as scaling, translating, stretching, rotating and branching of the ring and segment parameters creates diverse and constantly growing structures, as for example those shown in Figure 12.

Fig.12 Example of VERBARIUMS’s growing structure

Figure 12a shows the basic ring with 8 vertices and Figure 12b the extruded ring that forms a segment. Figures 12c and 12d show branching possibilities, with branching taking place on the same place (=internodium) (12c) or on different internodiums (12d). There can be several branches attached to one internodium. Figure 12e shows an example of segment rotation, Figure 12h the combination of rotation and branching. Figure 12f and 12g are different examples of scaling. In total there are about 50 different design functions, they are subsumed in the design function look up table (Fig. 13). These functions are responsible for "sculpting" the default ring through modifications of its vertex parameters.

function1 translate ring for certain amount (a) in x

function2 translate ring for certain amount (a) in y

function3 translate ring for certain amount (a) in z

function4 rotate ring for certain amount (b) in x

function5 rotate ring for certain amount (b) in y

function6 rotate ring for certain amount (b) in z

function7 scale ring for certain amount (c) in x

function8 scale ring for certain amount (c) in y

function9 scale ring for certain amount (c) in z

function10 copy whole segment(s)

function11 compose a new texture for segment(s)

function12 copy texture of segment(s)

function13 change parameters of RED in segment(s)texture

function14 change parameters of GREEN insegment(s)texture

function15 change parameters of BLUE in segment(s)texture

function16 change patterns of segment(s)texture

function17 exchange positions of segments

function18 add segment vertices

function19 divide segment in x to create branch

function20 divide segment in y to create branch

function21 divide segment in z to create branch

function22 create new internodium(s) for branch(es)

function23 add or replace some of the above functions

function24 randomize the next parameters

function25 copy parts of the previous operation

function26 add the new parameter to previous parameter

function27 ignore the current parameter

function28 ignore the next parameter

function29 replace the previous parameter by new parameter

.............

function50

Fig. 13 VERBARIUM’s design function table

The translation of the text characters into design function values is done by assigning ASCII values for each text character according to the standard ASCII table shown in Figure 5. The assignment of random seed functions to the actual updated values for the design functions in the design function table is done in the same way as explained in Chapter 2.a and in the example of Fig. 6.

As a result, each individual text message will create a very specific three-dimensional structure that can at times look like an organic tree or at times more like an abstract form. Figure 14 shows another example of forms created by a text message, this time the text was written in French.

Fig. 14 VERBARIUM web page - example
 
 
 
 

3.2. VERBARIUM’s virtual diary: retrieving texts from images

On-line users not only help to create images through text messages and thus develop the complexity of the virtual VERBARIUM, they also have the option of clicking any part within the collective image to retrieve messages sent earlier by other users. The web page thus functions as a kind of virtual diary or virtual herbarium where personal text messages are stored and transformed into images. In their totality they build a common image that can become increasingly complex as more users send messages. As a result, the system will develop toward more and more complex structures that represent the amount of interaction created through the on-line users participation.

4. Conclusions

We have created two interactive web sites that enable Internet on-line users to create visual forms from written language. By using our text-to-form editors, we can treat text messages as genetic code and translate the text’s parameters, such as letters, syntax and sequencing of text, into three-dimensional structures.

While various artists [5, 6, 7, 8] have been using the Internet to create web-based interactive art works, "Life Spacies" and "VERBARIUM" are the first systems to translate written text into visual three-dimensional forms. Inspired by the idea of creating visual language, we translated written language into visual forms to create

artistic and audience participatory systems for the Internet.

5. Future

Although our systems were not primarily aimed toward the field of Visual Language Studies, their relevance to this field of research is in the novel idea of translating written text into visual forms. Traditional visual language theory has for almost two decades been concerned with specif