Wednesday, November 14, 2007

Major Studio Interface : final project proposal --- soundFishing



My final project for the Major Studio Interface will be a portable tool to capture sounds from the environment based on certain logics; I like to call it soundFishing.


What brought this idea to life is both the combination of my curiosity and the environment where I’m currently living in; I’m deeply fascinated about the everyday sounds that surround us and the fact that usually we don’t care about them strikes me. We are constantly immersed in them and this condition denies us to perceive both their power and value.
I want to build a tool which helps to re-discover the value of these perceptions, an object able to analyze the sonic environment around us, filter it and collect precious data for us, out of our absolute control.

Domain of research

There are various research path connected to the soundFishing interface design synthesized by these key ideas:

Sounds as an intimate diary: the audio captured from the environment will be collected building up a sequence of the events happened during the user's everyday life flow.
This collection of events can then be easily related to the concept of a diary, a sonic diary.

Non conscious action: A fundamental difference lays between a traditional textual diary and this diary of sounds; usually the first is filled up consciously by the user who has the power to explicitly decide what and when write something into it. The sonic diary produced by the soundFishing interface instead will be composed unconsciously by the user who has just the power to set the basic logical rules that will control the capturing of the events to be recorded, but can not decide explicitly what and when to record. In my vision this loss of control on the tool can re-establish a sense of surprise and curiosity towards the everyday perception of hearing sounds.

Generative sampling - Automation: the soundFishing tool will be intimately procedural and algorithmic as the user will define a set of rules that will control the recording process, leaving then the interface working on its own without direct control.
The interface will act as an autonomous audio filtering agent who continuously listens to the environment for events to happen in order to start the sound capture.

This process can be linked to Lev Manovich concept of Automation:

"The numerical coding of media (principle 1) and the modular structure of a media object (principle 2) allow for the automation of many operations involved in media creation, manipulation and access. Thus human intentionality can, in part, be removed from the creative process."

" 'High Level' automation of media creation, which requires a computer to understand, to a certain degree, the meanings embedded in the objects being generated, that is, their semantics."

"The Internet, which can be thought of as one huge distributed media database, also crystallized the basic condition of the new information society: overabundance of information of all kinds. One response was an idea of software 'agents' designed to automate searching for relevant information. Some agents act as filters that deliver small amounts of information..."

instead of the Internet the soundFishing explores the sonic layer of our reality.

Multiplicity :
Generative processes leads to Multiplicity... I explain it better, mind the differences between these two statements:

"I want to record the sound of the police car siren that is passing now in the street."


"I want to record all the loud sounds I'm going to encounter today."

a huge difference exist between them as the first sentence leads to a simple but rather obvious result, on the contrary the second statement opens a door to many possible results giving the user a glimpse of the almost infinitely wide spectrum of possibilities that exists in our everyday life experience depicting a space of potential in the Mitchell Whitelaw perspective:

"Multiplicity here is a way to get a perceptual grasp on something quite abstract - that space of possibility. We get a visual "feel" for that space, but also a sense of its vastness, a sense of what lies beyond the visualisation."

"Multiplicity refers to the specific space of potential in any single system, by actualising a subset of points within it;"

Expanded Cinema - Sound sharing & mixing:
The audio captured by soundFishing can be valuable to the individual based on certain personal parameters, but what about its value related to other people? Why some people find sounds coming from the life of another person interesting or useful?

One input can come simply from curiosity, another because these sounds can be used to produced something else, a musical piece for example, or a video game sound effect or anything related to audio remixing and production.

In my vision the soundFishing output can be redirected into two different directions. The first can be a social network community as Youtube, but instead of video audio samples taken from the life of every user can be shared and mashed. Another direction can be one in the spirit of Expanded Cinema:

"Youngblood defines the technosphere as a symbiosis between man and machine. The computer liberates man from specialisation and amplifies intelligence (pp180-182). He draws comparisons between computer processing and human neural processing (pp183-184). Logic and intelligence is the brain's software. He predicts that computer software will become more important than hardware and that in the future super-computers will design ever more advanced computers (pp185-188). His vision of the future is the Aesthetic Machine: "Aesthetic application of technology is the only means of achieving new consciousness to match our environment" (p189). Creativity will be shared between man and machine."


"fm transposes non-metaphoric systems and grammar theory (of computer languages, abstraction and data containers) to the realm of expanded cinema. the base proposal concerns the development of a scripting language, data structures, and suitable filesystem for the automated production and grammatical expression of endless cinema."

By this perspective the soundFishing interface becomes an extension of the human ear and memory, allowing a more powerful perception of the sonic environment and a more durable memorization of sound in the form of digital samples.
These samples can then feed another generative system which can assemble them algorithmically to produce new and unexpected sonic experiences.

Design Issues

"The Data Avalanche

Another important consequence of having a computer with us all the time is the ease with
which information can be gathered. Data can be ‘beamed’ to us from nearby electronic devices or from the PDAs of other people we encounter, where and when we encounter them. And since the PDA is itself active, it can be commanded to pick up information automatically, as data become available, without the need to issue explicit instructions each time. The range of data we might profitably gather automatically is enormous: receipts from sales transactions, telephone numbers dialled, even overheads from presentations we attend. These are all things we might ask our PDA to capture, just in case the information might be useful someday. But there is an obvious snag.

Information overload is already a familiar problem we all have to contend with. When
these extra data start flowing in, the task of organising them will become even more daunting. It is unlikely we will suddenly find the time or develop the inclination to index all of it. In any case, we often don’t know what information we have been given. For example, we often receive information that some person, or some system, thinks will be useful to us — we then find we can’t file it away because we have not read it, but nevertheless we are reluctant to throw it away, just in case it might be needed.

Much of this information snowstorm will be difficult for the computer to index automati-
cally on our behalf based simply on its content. For example, sketches, photographs and video
are notoriously difficult for computers to analyse. It looks like intimate computers, when used as automatic data gatherers, could make our lives much harder. If this is the prospect, then we still won’t be able to make good use of the information we encounter — and just won’t bother to collect it.

Context as a Retrieval Key

Fortunately, the very features of PDAs that contribute to information overload can also
come to the rescue. The user’s context can itself provide a valuable key for indexing information automatically. A detail from a past event in which the user was involved might be difficult to recall, the name of a document, for example. But the context of the event can be easier to remember. For example, we may be able to recall: the place where the document was received, the people present when it was handed over, or the task being carried out at the time."

Precedents, Resources

Brad Rhodes - 'Remembrance Agent'

Microsoft Research Cambridge - Sensecam

Mitchell Whitelaw - 'More is More: Multiplicity and Generative Art'

Lev Manovich - 'The Language of New Media'

Gene Youngblood - 'Expaned Cinema'

M. Lamming and M. Flynn, 1994 - 'Forget-me-not': Intimate Computing in Support of Human Memory. In Proceedings of FRIEND21, '94 International Symposium on Next Generation Human Interface, Meguro Gajoen, Japan.


No comments: