Ideas
It's ART! Or, a medium with the potential for art.
I think it would be cool to have a way to print sounds onto paper, and then read
them back again and hear them. You could do all sorts of neat things, like a
exhibition of visual art, where each picture contains an additional level of
artistry via a sound. Printed media like magazines, instead of having to include
a CD or other physical audio medium like flexidisk, could simply embed the sound
into a printed page.
640 * 480 = 300000 * 3 = 900000 (1 MB = 1000K)
Telephone 8K/sec
CD 44K/sec
If low-quality audio is 10K/sec, then an image gives 90 seconds. But, this is an
idealized case not achievable in reality. Things like color balance, light
balance etc. introduce many significant bits of error into each R, G, and B.
Also, a fully literal representation would probably be uninteresting visually;
reducing the potential for aesthetic impact.
To convert from RGB to HSB may be a good idea, since it could smooth out the
perceived colors, making brightness shifts into a single noise value, instead of
all three RGB shifting. It could also help the human eye to see relationships
between the sound and image, since HSB is more like the human visual conceptual
model.
Printers operate on a CMYK color model, dealing with reflected rather than
emitted light. This means there are colors possible with RGB that are not
possible in CMYK, and these colors will not print accurately. We should avoid
these colors, which limits the amount of information the medium can carry
through the whole process. (To do: What colors are they? What is the RGB -> CMYK
algorithm?)
A raw audio wave is single-valued, an amplitude wave. One very simple approach
would be to use the raw HSB values, one of them or a combination, directly as
amplitude. This algorithm (basically a codec) seems like a good candidate to
implement first, to test the software framework.
There are really an infinite number of codecs possible. A modular software
architecture would allow any programmer to write their own and plug it in.
People could attempt to create codecs which give more aesthetically pretty
results, while maintaining the fidelity of the audio encoding.
Ideas
These ideas first arose during conversation with sound artist Maryann Amacher in San Francisco in May 2001.