Print and play sound on paper
Is it possible to encode sound in the form of an image that is read by the camera and played in real time?
A QR code or some of its counterparts immediately come to mind. In this code, you can write a link to mp3, or even a very small audio file. But in the first case, an output to the network is required, and in the second, a higher code density and increased requirements for image quality. In addition, neither method provides instant playback.
What if the audio is not encoded in digital, but in analog format? That is, to allow the possibility of large losses of information while preserving the basic “outlines” of the sound message. For example, such losses will occur in poor light, poor camera, small picture, crumpled or torn paper with a code. Let with scary artifacts, but the picture should sound.
')
For several years I have been studying the Soviet photoelectronic synthesizer ANS , created by engineer Evgeny Murzin from 1938 to 1958. The synthesizer is based on the earlier technology of the so-called “drawn sound”, which can be read in more detail, for example, in this article or on the Generation Z site. The trick of ANS is that music is created by drawing a spectrogram and then played by a kind of modern scanner - a vertical scanner shines through the score, projecting light onto special discs (a kind of optical Fourier transducer) that are sent to the photocells. Such a format would be ideal for recording sound in graphic form. A damaged or reduced spectrogram may lose some harmonics, but will save the big picture for later playback. And by the way, I created a software simulator of the ANS synthesizer - Virtual ANS , which can not only play such musical scores, but also create it by converting the audio file into a picture of the spectrum. Exactly what is needed!
It remains to answer the last question - how to mark the borders of a picture, so that the scanner program plays exactly what is needed without capturing extra information. It was decided to use only horizontal markers above and below. Refusal of markers on the sides of the code, firstly, simplifies the decoding process, and secondly, allows you to play the spectrogram without special expenses immediately after the detection of horizontal markers. At the same time, the playback process can be controlled manually: the listener himself sets the speed and direction by freely scanning the image. Manual mode is fascinating, but the uneven movement and shaking hands imprint on the sound. Therefore, an automatic mode should be added to the program when the current frame from the camera freezes and is scanned uniformly without user intervention.
The project will give the name PhonoPaper. The code specification is as follows:
A free scanner program was written in approximately 2 months in Pixilang language using the Virtual ANS engine (for playing a sonogram in real time). Currently it is available for Android and iOS platforms. Support for other systems is in the plans.
I foresee the question of practical application. Therefore, here is a small list with examples of using PhonoPaper codes:
Let's see how it works live:
Of the features I want to note the ability to change the shape and color of such codes with little or no harm to the audio information stored in them. I will give a couple of examples:
A QR code or some of its counterparts immediately come to mind. In this code, you can write a link to mp3, or even a very small audio file. But in the first case, an output to the network is required, and in the second, a higher code density and increased requirements for image quality. In addition, neither method provides instant playback.
What if the audio is not encoded in digital, but in analog format? That is, to allow the possibility of large losses of information while preserving the basic “outlines” of the sound message. For example, such losses will occur in poor light, poor camera, small picture, crumpled or torn paper with a code. Let with scary artifacts, but the picture should sound.
')
For several years I have been studying the Soviet photoelectronic synthesizer ANS , created by engineer Evgeny Murzin from 1938 to 1958. The synthesizer is based on the earlier technology of the so-called “drawn sound”, which can be read in more detail, for example, in this article or on the Generation Z site. The trick of ANS is that music is created by drawing a spectrogram and then played by a kind of modern scanner - a vertical scanner shines through the score, projecting light onto special discs (a kind of optical Fourier transducer) that are sent to the photocells. Such a format would be ideal for recording sound in graphic form. A damaged or reduced spectrogram may lose some harmonics, but will save the big picture for later playback. And by the way, I created a software simulator of the ANS synthesizer - Virtual ANS , which can not only play such musical scores, but also create it by converting the audio file into a picture of the spectrum. Exactly what is needed!
It remains to answer the last question - how to mark the borders of a picture, so that the scanner program plays exactly what is needed without capturing extra information. It was decided to use only horizontal markers above and below. Refusal of markers on the sides of the code, firstly, simplifies the decoding process, and secondly, allows you to play the spectrogram without special expenses immediately after the detection of horizontal markers. At the same time, the playback process can be controlled manually: the listener himself sets the speed and direction by freely scanning the image. Manual mode is fascinating, but the uneven movement and shaking hands imprint on the sound. Therefore, an automatic mode should be added to the program when the current frame from the camera freezes and is scanned uniformly without user intervention.
The project will give the name PhonoPaper. The code specification is as follows:
A free scanner program was written in approximately 2 months in Pixilang language using the Virtual ANS engine (for playing a sonogram in real time). Currently it is available for Android and iOS platforms. Support for other systems is in the plans.
I foresee the question of practical application. Therefore, here is a small list with examples of using PhonoPaper codes:
- voice prompts or thematic pieces of music on business cards, advertising posters, products;
- paper postcards with a printed audio message;
- audio examples in textbooks or children's books;
- audio tags on plates, discs, cassettes;
- sound signature on the back of a paper photo;
- art experiments;
- Learn the basics of building spectral music and sound through the manual drawing of PhonoPaper codes.
Let's see how it works live:
Of the features I want to note the ability to change the shape and color of such codes with little or no harm to the audio information stored in them. I will give a couple of examples:
Source: https://habr.com/ru/post/220061/
All Articles