record and analyse real-time audio

For a project im working on i need to record audiodata from a microphone and analyse it simultaneously. I found some threads on mathworks but yet couldnt figure out how to do it.

I startet very basic with this:

fs = 8000                           %or whatever suits best
recorder = audiorecorder(fs,24,1);  %setting up the recorder

recordblocking(recorder,2);         %record 2 secs
data = getaudiodata(recorder);      %extract the recorded 2 secs
pitch(data,fs);                     %plot

This works fine for just 2 secs, but i cant find a way to do this with a 15 minute long recording, extracting every "new" 2 seconds...

From

https://de.mathworks.com/matlabcentral/answers/157897-how-to-record-and-analyze-audio-in-real-time

i tried to obtain the function/timer combination but that didnt work.

I´d be glad to get some help,

thanks in advance!

[Jonas]

please have a look into the audiodevicereader object. that object can read data much faster and buffer wise (e.g. 1024 samples). look into the documentation of it and come back if you need further assistance

if you have problems with lag, look into your drivers and in doubt, have a look into the Project Asio4all.

Answer 1

[jibrahim]

Hi Oskar,

When working with real-time audio, you should consider audioDeviceWriter, audioDeviceReader and audioPlayerRecorder.

Here is a possible pattern:

fs = 8000; % Sample rate

% Create microphone object
adr = audioDeviceReader(SampleRate=fs);

% Create a buffer to save microphone samples
buff = dsp.AsyncBuffer(Capacity = fs*2.5); 

numIterations = floor(15*fs/adr.SamplesPerFrame);
for index=1:numIterations % run as long as you want
    % Read from the microphone
    frame = adr();
    % Save data to buffer
    write(buff,frame);
    % If you have enough data, do you analysis
    if buff.NumUnreadSamples >= 2*fs
        data = read(buff);
        p = pitch(data,fs);
    end
end

[Oskar Kilgus]

Hi @jibrahim, thanks for the suggestion!

When i try to run your code it seems like matlab is doing all the recording and calculations first, then plot the graph. Am i doing something wrong?

[jibrahim]

If you are calling pitch with no output arguments, then you probably need a call to drawnow to make sure the graphics callbacks are executed in the loop:

fs = 8000; % Sample rate
% Create microphone object
adr = audioDeviceReader(SampleRate=fs);
% Create a buffer to save microphone samples
buff = dsp.AsyncBuffer(Capacity = fs*2.5); 
numIterations = floor(15*fs/adr.SamplesPerFrame);
for index=1:numIterations % run as long as you want
    % Read from the microphone
    frame = adr();
    % Save data to buffer
    write(buff,frame);
    % If you have enough data, do you analysis
    if buff.NumUnreadSamples >= 2*fs
        data = read(buff);
        pitch(data,fs);
        drawnow limitrate
    end
end

Note that calling pitch like this in a loop is not efficient, as the function creates a new figure every time it executes, which might lead to dropping some microphone samples. You might be better off using a time scope to visualize the pitch:

fs = 8000; % Sample rate
% Create microphone object
adr = audioDeviceReader(SampleRate=fs);
% Create a buffer to save microphone samples
buff = dsp.AsyncBuffer(Capacity = fs*2.5);
numIterations = floor(15*fs/adr.SamplesPerFrame);

% 100 Hz corresponds to 1/0.01 s, the hop size in pitch
ts = timescope(SampleRate=100,YLimits=[0 500]);

totalDrops = 0;
for index=1:numIterations % run as long as you want
    % Read from the microphone
    [frame,drops] = adr();
    totalDrops = totalDrops + drops;
    % Save data to buffer
    write(buff,frame);
    % If you have enough data, do you analysis
    if buff.NumUnreadSamples >= 2*fs
        data = read(buff);
        p = pitch(data,fs);
        ts(p)
    end
end
totalDrops

[Oskar Kilgus]

That seems to take me a lot further on my journey, thank you very much!!

[Oskar Kilgus]

Hey @jibrahim, as i´m working on and with the code i wondered why there is a constant "offset" of 0.02 seconds in the timescope. According to:

https://de.mathworks.com/help/dsp/ug/configure-time-scope.html

the timescope displays the time on the bottom right corner. When i run it, i "lose" 0.02 secs on every tick.

Is that caused by any properties/settings etc. or does it simply not count in full seconds and is just confusing?

Changing the SamplingRate in timescope properties simply changed the scaling of the x-axis...

Thanks in advance!

[jibrahim]

Hi Oskar,

I think this is due to how the pitch function works.

When you feed pitch a long audio signal, it computes pitch values on overlapped windows of audio. For example, if thw window length is 416 samples and the overlap length is 336 samples (defaults for fs=8000), the function generates an estimate for the first 416 samples of audio, then shifts the window by 416-336 = 80 samples, and computes an estimate for the window corresponding to samples 497:912, then shifts the window by 80 more samples, etc.

The first pitch estimate is based on 416 "new" samples, wheras all subsequent pitch estimates in the same function call correspond to 80 "new" samples, all the other samples coming from the previous window. So, the "update rate" of pitch computation is basically 80 samples (or 100 Hz), but not quite, because of the very first call. timescope thinks it is 100 Hz, but it is a little off on the first call, which explains this weirdness.

pitch was designed for "batch" scenarios primarly, not real-time scenarios.

You can either ignore this misleading time read-out, or you can alternatively use an ArrayPlot scope instead. It might not be as convenient as timescope, but it allows you to control the x-labels. For example:

fs = 8000; % Sample rate
% Create microphone object
adr = audioDeviceReader(SampleRate=fs);
% Create a buffer to save microphone samples
buff = dsp.AsyncBuffer(Capacity = fs*2.5);
numIterations = floor(15*fs/adr.SamplesPerFrame);

scope = dsp.ArrayPlot(XDataMode="Custom",PlotType="Line",...
                      YLimits=[0 500],XLabel="time (s)");
totalDrops = 0;
currentTime = 0;
for index=1:numIterations % run as long as you want
    % Read from the microphone
    [frame,drops] = adr();
    totalDrops = totalDrops + drops;
    % Save data to buffer
    write(buff,frame);
    % If you have enough data, do you analysis
    if buff.NumUnreadSamples >= 2*fs
        data = read(buff);
        % loc represent the location associated with each pitch decision.
        [p, loc] = pitch(data,fs);
        scope(p)
        % Set the x ticks
        scope.CustomXData = currentTime + loc/fs;
        currentTime = currentTime + loc(end)/fs;
    end
end
totalDrops