F0 detection

Loading...

En provenance du cours de Universitat Pompeu Fabra of Barcelona

Audio Signal Processing for Music Applications

207 notes

Universitat Pompeu Fabra of Barcelona

Audio Signal Processing for Music Applications

207 notes

In this course you will learn about audio signal processing methodologies that are specific for music and of use in real applications. We focus on the spectral processing techniques of relevance for the description and transformation of sounds, developing the basic theoretical and practical knowledge with which to analyze, synthesize, transform and describe audio signals in the context of music applications. The course is based on open software and content. The demonstrations and programming exercises are done using Python under Ubuntu, and the references and materials for the course come from open online repositories. We are also distributing with open licenses the software and materials developed for the course.

À partir de la leçon

Harmonic model

Harmonic model equation; sinusoids-partials-harmonics; polyphonic-monophonic signals; harmonic detection; f0-detection in time and frequency domains. Demonstrations of pitch detection algorithm, of the harmonic model interface of the sms-tools package and of its use in the analysis and synthesis of sounds. Implementation of the detection of the fundamental frequency in the frequency domain using the TWM algorithm in Python and presentation of the harmonicModel functions from the sms-tools package, explaining how to use them.

F0 detection16:53

harmonicModel14:26

Rencontrer les enseignants

Xavier Serra
Full Professor
Dept. of Information and Communication Technologies, UPF
Prof Julius O Smith, III
Professor of Music and (by courtesy) Electrical Engineering
CCRMA

Welcome to the course on audio signal processing for musical applications.

This week, we are talking about the harmonic model.

And in this programming lecture, I want to talk about the implementation of it.

In particular, the first part of it.

The part that requires detecting the fundamental frequency so

that we can then identify the harmonics of a given sound.

So we will be talking about one particular algorithm, the two way mismatch algorithm,

that's an algorithm that we presented in the theory lecture and

it's a frequency the main algorithm that basically tries to identify

harmonic series, possible harmonic series that match the peak of spectrum.

So in this plot we see the measured peaks that we have identified.

And then we keep trying different predicted

fundamental frequencies and the harmonics of it.

And we measure the error,

we measure the distance between these two lists of values.

And we did that by measuring two errors, the predicted two measures.

So the difference between, the distance between the predicted and

the measured values.

And also we have another measure which is the measure to predict that error.

But let's go directly to the actual code.

Okay, in the sms tools package, in the util functions file,

there is the code for a way we match algorithm.

The core of it is a function called two way mismatch In fact,

there is a C version and a Python version.

Now we will go through the Python version.

When we run it, we normally use the C version because it's more efficient.

So this algorithm, what it does,

is it receives the peaks, the frequencies, and magnitudes of the peaks.

It receives a list of candidates,

of frequencies of candidates of fundamental frequency.

And it basically identifies which is the candidates that has the smallest error.

So it does that by measuring the two errors, the error predicted to measure

just this part and then measure to predict it, which is this part.

Within it it keeps identifying

all the distances between all the balance of the harmonic series and the peaks.

And it has different ways of comparing those.

We're not going to go into the tail of that, but of course,

feel free to go into it.

And then finally, it just creates an error array,

which is the list of errors of all the candidates, okay?

So we have, in the array, we have all the errors for every single candidate.

And then what we do is we choose the minimum of those errors, and

the fundamental frequency is going to be that candidate that has the minimum error.

And then this function is wrapped by another one

that is the one responsible for generating the candidates and calling the function.

So this Fz, F02wn, receives

again the peaks of the spectrum, and then it receives the control parameters,

so like the maximum error allowed, this is the error that will be allowed for

the fundamental frequency to be accepted as such, and then the range

of the fundamental frequencies are from minimum and maximum at 0.

And then there is one value which is kind of a memory, a tracking value,

that is basically the fundamental frequency of the previous frame.

And this will allow us to refine the fundamental frequency

by restricting that the fundamental frequency should be as smooth as possible.

But the algorithm here is very simple.

In fact, it just takes the list of peaks that are within

the minimum and maximum value of the frequencies.

And the rest, it just makes a few more comparisons about that.

There's a lot of room for

improvement in this algorithm in the sense of generating more candidates so

that we do a more exhaustive trial of different frequencies,

but for efficiency reasons we made this simple implementation so

that allows us to compute this quite efficiently.

Okay, so I wrote a little script that basically does

an analysis of a single spectrum and then it computes the errors of all candidates.

Okay, so in here, I have this little script that from a sound,

the subtle sound, it just computes one DFT.

And so here is, it computes one DFT of that particular sound.

It finds the peaks.

It finds the, interpolates the peaks and

then it generates possible candidates of the fundamental frequency

in a similar way than what we just saw but even more simply

in the sense that we are taking the candidates as all the peaks that lie

within the ranges that we specified.

And then it called the two way mismatch algorithm but

I modify the functions what I have it here in a way that,

instead of returning just one value.

The fundamental frequency that is the minimum error, it returns all the errors.

So it returns the array of all the errors for all the candidates,

so that we can look at them and see how they behave.

Okay, and then it prints, here it just prints the spectrum and the pixel.

You can understand a little bit what's going on.

Okay, so let's run this.

So let's run test.

Okay, and this is the spectrum.

The magnitude spectrum and the peaks that we found.

So let's maybe zoom in a little bit.

So then we can see a little bit what is going on, okay.

So these are clearly the harmonics of the sound.

It has also found peaks like before the fundamental frequency and

one after the fundamental frequency.

But that's so that's what basically in

terms of the identified just the harmonics.

So now let's plot or let's print some of the intermediate values of all these.

So clearly the first thing is the candidates.

So if we print the f0c.

This is the candidate and is going to be a pics that lie with in

the frequency of range we specified which was between 50 and 2000 hertz.

So the candidates are the first five pics and

if we print their frequencies so by doing ipfreq.

And f0 candidates, those are the frequencies

that lie within the frequencies range, and that we're going to test in the algorithm.

So we're going to test 166 hertz, 440, 637, etc, etc.

So now let's print the errors that it returns.

So F0Errors, which is

the output of this algorithm will have the errors for every one of these values.

So for 166, we have an error 4.8, 440 has minor 0.13,

so clearly this is the smallest of all errors and

this is indeed the fundamental frequency.

The candidate that is the best one for, as a fundamental frequency.

So these error values are really misleading because they are not

bounded within a particular range.

It can even be negative, like in this case.

But clearly the larger the error, the less

chance is that it's going to be from the mental frequency.

So we will be focusing on the lowest errors that we have.

Okay, so this works quite well.

Now we can go into another

file that basically does this for the whole sound.

So we will be iterating for the whole sound.

We just doing the exact same thing.

We're taking the sotto sound and we are trying a different window.

We keep doing it and see if we can get a different type of result.

We take the FFT, we find the minimum and maximum and

we call a function F0 detection.

Which in fact is on the harmonic model file.

In the harmonic model file there is this function

called F0 detection that does all what we talked about.

Basically accepting gifs from the input sound.

The sampling rate window FFT size and the values given by the user,

it iterates over the whole sound and it calls the DFT, the peak detection,

peak interpolation, the two-way mismatch algorithm and

then it decides which one repeat the base, the fundamental frequency.

That is if you sort of constraints to make sure that the fundamental

is stable in time, it related with this track that we talk about.

So but basically it returns just the fundamental frequency that's

considered to be the best, okay.

So let's now look at test1 and let's run it.

So let's run test1, okay and

now in fact we can just show the f0.

Okay so this is the values that it has returned,

while the hop size that is specified was quite large, 1,000 so

it is not that many samples, so that's easier to look at.

And okay, clearly there is not a perfect

fundamental frequency identify.

It kind of varies.

It goes from 439.9 to 440 something.

So in fact, if we plot this array,

we will see the variation that we will have here.

Okay, well let's maybe get rid of this and

let's get rid of these and now lets plot it again.

Okay, now we can zoom in to the very top, okay.

So clearly it moves around 440 and

these variations are caused by clearly by the errors

of the peak detection algorithm and the interpolation so

that we are not really exactly at 440 but,

of course, the error is very small.

It's less than 1 Hz error.

So, this is 440, and 440.5 and a little bit below.

So this is clearly a quite small deviation

from the nominal value that is 440.

If we change these values, we might get better results.

For example, instead of having the window being 1,001,

let's make it twice as much.

And a 50 size, let's make it twice as much.

Okay, and, no, times 2, okay.

And now we will, so let's see that before we were 439.95, 440 something,

let's see if it does any different,

any different by looking now at these values, okay?

So this is what we got now and it's a little bit better.

So we can see the difference between these two values.

Now the error is smaller than before.

In fact, if we plot now this have zero.

Well there is the exponential to the minus,

to the minus 2, so clearly this is a very small,

a smaller error range than what we had before,

so the lowest now is 439.9 and

the highest value is 440.042.

So that means that as the window gets larger and

the FFT gets larger and the increases we will have better values.

Okay, now let's look at the real sound and let's finish

by running it on this over sound and basically do exactly the same thing.

So I can just run this test2, okay,

this will compute a fundamental frequency of this other sound.

And now if I plot this at 0,

Okay and now we will have to zoom into the meaningful range and

well there is definitely also a variation, but

here there is both the variation that may be caused by errors.

And the variation that is clearly natural to the plane of the over sound.

So for example, this sound is clearly higher then 440,

so the over sound was played a little bit higher in

frequency than 440, so around 442.

And there is kind of a periodic oscillation that make sense

to be present in the sound and of course it might be sound of

this oscillation and maybe caused by some error but

this is a very interesting way to try to understand what is going on.

And both in terms of algorithm and in terms of the sound in terms of this other

sound and natural oscillations that may be caused by either the acoustics or

the performer that is playing this note.

Okay, and that's all I wanted to say.

So basically, we have talked about the implementation

of the two-way mismatch algorithm and I think that has given us

a view on the issues of how to detect it from the mental frequency.

Of course, we have used Python and a number of these packages and

the implementations that we have in the SNS tools package.

So that's all.

So this was the first programming class on this harmonic model week and

then on the next lecture we will then add the whole model.

And include these fundamental frequencies into a harmonic analysis,

and we'll be able to do both analysis and synthesis of sounds.

So thank you very much and I'll see you next lecture.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2019 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play