Audio Animation - kodasblog.com

Audio Animation

by Koda

Preface: I developed the theory of Audio Animation in the mid-1970s and spent ten years of my life trying to get it financed before I gave up and shelved the project. Articles about it appeared in MIX, June 1984, Soundscapes in 2000, and it was mentioned in the Journal of the Audio Engineering Society. I wrote this short story to make explaining the technology more entertaining for lay persons.

*****

Gus was born blind, dumb and grumpy. He has spent all day, every day since 1974, sitting on a curb by the street in the hot sun, in the blowing rain and freezing snow, feeling completely unappreciated. All he does is listen and think. He has spent so much time in that one spot listening to every nuance in the sounds around him that he could tell you the number and height of the trees across the street. He can recognize different people not only by their voices but from the sound of their individual footsteps. Gus knows how many cars are parked nearby and if anyone were to tell him the make and model he could tell you that information as the vehicles come and go.

But almost no one ever talks to him. In fact, people often end up acting like he isn’t there because he doesn’t respond to their comments, or worse, they refer to him as an object, saying things like if “that” wasn’t there they could park where they wanted. Things like that have a lot to do with why Gus is so grumpy.

Saying he is dumb means that he can’t talk, not that he is stupid. Far from it. Gus isn’t distracted by needing a job to survive or the ups and downs of personal relationships. He has no ambition to consume his mind, and only one hope, that the day will come when someone will be able to get him to open up so everything that has been pent up inside him can spew out in a never ending gush of release.

But Gus can’t talk, or write, or communicate in any way at all. All he can do is listen and think, so he thinks a lot about the sounds he hears.

He asked himself how it is that he can tell where a sound comes from, and it took a really long time before he figured it out. First he realized that a sound coming from his right reached his right ear before it reach his left ear. It was louder, and it sounded brighter, having more high frequency content because the sound wasn’t blocked by his head. And because he was such a critical thinker he also realized the sound waves must be slightly out of phase due to the time difference in the arrival of the sound wave at each ear. In the same way, sounds coming from his left took longer to reach his right ear, and all the other details were simply reversed.

But what about sounds coming from behind him? That question puzzled Gus for a long time, but one day he realized that the shape of his ears reduced some frequencies more than others, and those changes in frequency content also told his brain when sounds were coming from in front of him, above or below.

And there were other things involved, like the way sounds reflected off surfaces creating really fast but faint echoes or reverberation, which helped him know if the sound was bouncing off a car or a tree, and those reflections helped him to zero in on the source of the sound.

One day Gus was sitting in his spot immersed in the center of his spherical universe made of sound, listening to birds above and behind him, to cars speeding past from side to side and water moving below him in the gutter — when a eureka moment occurred. He realized that if he were to put microphones inside his ears and make a recording, which was then played back through headphones, all the dimensional sound information would be reproduced and any listener would believe they were sitting exactly where he was. If someone walked past on the sidewalk behind him during the recording, the listener would also hear someone walk behind them.

Yeah, Gus thought he had invented something pretty cool, but it only made him even more grumpy because he had no way to tell anyone how clever he was.

Then one sunny afternoon Koda walked up to Gus and talked to him like he actually existed.

“I know you can’t respond,” said Koda, “but that thing you think you invented has been done before. It is called holophonics, and recordings have been made where people wearing headphones can hear the sounds of someone getting their hair cut and it feels just like they are getting their own hair cut. People have actually put microphones in the heads of dead people to record a symphony so listeners wearing headphones feel like they are in the middle of the orchestra. Holophonics is pretty cool, but Audio Animation goes way beyond that”

Gus was angry to begin with and he wasn’t happy to learn that someone else had beaten him to his big idea, but then again he would never be able to do anything with it so he was somewhat pleased just to know it could actually be done. But this guy just told him something even more amazing could be done with sound and Gus didn’t believe it, ‘cause no one knew sound better than he did.

“Imagine you have those microphones in your ears and I snap my fingers right here,” said Koda. “Then I snap my fingers over here, and over here.”

Koda walked around Gus repeatedly snapping his fingers all around Gus’s head.

“Imagine that rather than snapping my fingers I used a really small source of sound, like the tip of a spark plug, and when electricity jumped the tiny gap it would make a loud but very small, broad spectrum ‘snap’ so all the locational information could be recorded by the microphones in your ears. Then I move the spark plug just a little bit to the left and make another recording, then I move it again and again, constantly making recordings as the spark plug is placed in every possible position in every direction around you.”

Gus didn’t move even a tiny bit but continued to listen to Koda’s explanation.

“Just as an example,” he went on, “let’s say we made digital recordings of each ‘snap’ and used a computer to play back selected sounds one after another very quickly.”

Koda stood in front of Gus and moved his arm to trace a circle in the air while repeatedly snapping his fingers, and Gus could hear a circle being drawn in the air in front of him.

“A computer would play back much smaller points of sound and they would come one after another much, much faster,” said Koda, “ so the circle you are hearing would be much more clearly defined. And see,” said Koda as he moved all around Gus, “I can make a circle behind you, above your head, and down here by the ground. If I could snap my fingers fast enough I could make one circle after another come at you and pass behind you so you would feel like you were moving through a tunnel, a curving tunnel that twists and turns as you pass through it. The tunnel could stop and change directions so you would travel through it backwards, and you could ‘see’ which way the tunnel was going to turn without having to turn your head to look behind you.”

“I’m blind,” thought Gus, “so I am not going to be turning my head anyway.”

“You could find yourself in the center of a rotating cube ten feet across,” said Koda enthusiastically, “and the cube could shrink till it was only one-inch across rotating in the center of your head. Of course, all those sound samples would need to be analyzed to find out the exact variables that tell us how to position a sound where we want it, but we could use that information to process musical sounds so each instrument in an orchestra could become a different geometric shape spinning and dancing in choreographed movement in every direction around us at once! 3-D sound shapes like spaceships could happen all around people in movie theaters as the spaceships zoom by shooting laser cannons at each other.

“Peaow! Peaow!” said Koda, dancing all around Gus simulating spaceships shooting at each other.

“This stuff could be broadcast on radio and television,” said Koda, “and just think about how cool animated, 3-D sound shapes would be in video games!”

Gus just continued to sit as still as stone, his attention distracted by the sound of an approaching firetruck.

“Yeah, well, I guess you aren’t that impressed,” said Koda,” but since you can’t see you might be interested to know that Audio Animation can be used as a mobility aid for the blind. A video camera and microchip inside a pair of eyeglasses could convert black and white images into 2-dimensional sound shapes so blind people can recognize everything around them, even read street signs and house numbers. Add depth detection technology and the world around a blind person could be displayed as 3-D sound shapes. Gus, you could see clouds for the first time in your life!”

Gus remained motionless, but Koda thought he could see a bit of moisture collecting near Gus’s nose.

Inside Gus was about to explode with joy. From the sound of the approaching firetruck siren Gus knew the truck was slowing down, and to him, the siren sounded like an orchestra playing the kind of emotionally charged, beautiful music that happens as a movie reaches it’s dramatic climax. The siren because so loud Gus couldn’t hear himself think, and when the truck stopped directly in front of him with the siren still blaring Gus knew it was the end of people letting their dogs pee on him. A fireman rushed toward Gus, clamped a huge pipe wrench on his nose and tore it right off. Then a hose was attached where his nose used to be and the square nob on the top of Gus’s head was turned. At that moment Gus experienced true spiritual orgasm as his one and only wish finally came true and everything inside him went gushing out.

END

*****

Links to some of my other work

Books and E-Books on Amazon

Rational Spirituality by Koda

Koda’s Psychic Party G ames

Music on Spotify

Psychic Trance Fur (spacey/psychedelic)

The Early Years by Koda (acoustic guitar ballads and instrumentals)

(the music is also available on Apple Music)

Other Links

YouTube: Cosmic Koda