Acoustics Example — IoTPY: Python + Streams

Acoustics and signal processing Example

This section describes different examples using streams of acoustic data. The first example takes a sound as input and creates an output which adds reverberation to the input sound. This example is an illustration of the use of IoTPy; it is not a discussion of acoustics. The model of acoustics and echoes that we use here is simplistic. These examples were developed by S. Deepak Narayanan and Atishay Jain, IIT (Indian Institute of Technology) Gandhinagar, with Julian Bunn and Mani Chandy at Caltech.

echoes and Reverberation of sound

The diagram shows the agents (boxes) and streams (arrows) of a simple reverberation generator which assumes that sound is generated at the center of a cubic room and a microphone at the center of the room records what is heard. When a sound wave hits the walls of the room it is reflected. Let T be the time (in number of sample points) for sound to traverse from one wall to the opposite wall. An impulse generated at time 0 will cause an echo at time T, 2T, 3T,… as each echo causes a further echo.

The reflected wave is an attenuation of the incoming wave. Let the attenuation factor from all the walls jointly be A. Then the microphone will pick up the original impulse of intensity 1 at time 0 followed by an impulse of magnitude A**n (i.e. A raised to the n-th power) at time n*T for n = 1, 2, 3, … For example, if the attenuation factor is 0.5 and the delay is 4 sample points then the microphone will record [1, 0, 0, 0, 0.5, 0, 0, 0, 0.25, 0, 0, 0, 0.125, ….].

A sound wave may be scattered when it is reflected from a surface, and one gross approximation to scattering is to use a vector of attenuations and represent each echo by a dot product of the incoming wave and the attenuation vector.

Heard sound is the sum of the original sound and the echo. The echo is an attenuated delay of the heard sound.

The diagram shows a process that creates the heard sound which includes reverberation added to the original sound. The original sound stream is the only input stream to the process and the heard sound stream is the only output stream from the process. The heard sound stream may be processed further in other processors or it may cause an actuator (e.g. speaker) to generate the sound or it may be stored in a file. The original sound stream may be generated by an electronic instrument or microphone or may be generated from a stored sound (e.g. .wav) file.

def make_echo(spoken, delay, attenuation):
    echo = Stream(name='echo', initial_value=[0]*delay)
    heard = spoken + echo
    r_mul(heard, echo, attenuation)
    return heard

If the sound was generated in a rectangular room, rather than a cube, then we would have multiple echoes from different walls, and we would represent that by multiple echo agents. Consider a room which was much longer than it was broad. The echo from the far-away walls would re-echo at a time period determined by the length of the long-side of the room while the echo from the near walls would re-echo at a period determined by its breadth. The heard sound at the center of the room would consists of the sum of the original sound and both echoes. The diagram is shown below without showing delays explicitly.

In our code, the sound stream is generated from a file; parameters such as the rate of data generation can be set so as to simulate a microphone or other source. The source generator executes within its own thread. In this example the heard sound is stored in a file.

The computational function of the process is shown in the code below. The code for the source thread which generates the original sound stream is shown later.

def compute_func(in_streams, out_streams):
        out_streams[0] = make_echo(in_streams[0], delay, attenuation)
        stream_to_file(in_stream=out_streams[0], filename='heard.txt')
        stream_to_file(in_stream=in_streams[0], filename='spoken.txt')

The response to an impulse in the center of a cube is shown in the diagram below where the top plot shows the original sound — the impulse — the middle plot shows the echo and the the lowest plot shows the heard sound. (The echoes in the response may look like they are getting longer in the plot, but that’s an artifact of the plot; in fact each echo is of the same duration.)

The Source Thread

If the source data is in a list then we use source_list_to_stream to create a thread that copies data from the source into the stream.

    def generate_original_sound(original_sound):
        return source_list_to_stream(
            in_list=original_sound_list,
            out_stream=original_sound,
            time_interval=0)

The time interval is the rate at which data from the list is copied to the stream; you can simulate different types of source devices, such as microphones, by setting the time interval appropriately.

Processes

In this example we run the application as a single process. Since this process does not communicate with remote processes we make this process a shared_memory_process rather than a distributed_process.

    # Create processes
    proc = shared_memory_process(
        compute_func=compute_func,
        in_stream_names=['original_sound'],
        out_stream_names=['heard_sound'],
        connect_sources=[('original_sound', generate_original_sound)],
        connect_actuators=[],
        name='proc')

This process has a single input stream named ‘original_sound’ and a single output stream named ‘heard_sound’ and its source thread generate_original_sound puts data into the input stream called ‘original_sound’. The process in this example has no actuators and so connect_actuators is the empty list. (Note: You don’t have to include the connect_actuators = [] line because that’s the default.)

Creating and Running the Multiprocess Application

Later, if you wished, you could process the heard sound in another processor; however, for now let’s restrict the application to a single process. Since the application has only one process, this process has no connections to other processes. We now create a multiprocess application consisting of a single process, proc, and no connections.

    mp = Multiprocess(
        processes=[proc],
        connections=[])
    mp.run()

More Functions in Signal Processing Examples

Generate Waves: Generates different types of waves such as square waves and sine waves.