Audio Processing is likely one of the most essential software domains of digital sign processing (DSP) and machine studying. Modeling acoustic environments is a necessary step in growing digital audio processing methods corresponding to: speech recognition, speech enhancement, acoustic echo cancellation, and so on.
Acoustic environments are crammed with background noise that may have a number of sources. For instance, when sitting in a espresso store, strolling down the road, or driving your automobile, you hear sounds that may be thought of as interference or background noise. Such interferences don’t essentially observe the identical statistical mannequin, and therefore, a combination of fashions could be helpful in modeling them.
These statistical fashions will also be helpful in classifying acoustic environments into totally different classes, e.g., a quiet auditorium (class 1), or a barely noisier room with closed home windows (class 2), and a 3rd possibility with home windows open (class 3). In every case, the extent of background noise could be modeled utilizing a combination of noise sources, every taking place with a unique likelihood and with a unique acoustic degree.
One other software of such fashions is within the simulation of acoustic noise in numerous environments primarily based on which DSP and machine studying options could be designed to resolve particular acoustic issues in sensible audio methods corresponding to interference cancellation, echo cancellation, speech recognition, speech enhancement, and so on.
A easy statistical mannequin that may be helpful in such situations is the Gaussian Combination Mannequin (GMM) wherein every of the totally different noise sources is assumed to observe a selected Gaussian distribution with a sure variance. All of the distributions could be assumed to have zero imply whereas nonetheless being sufficiently correct for this software, as additionally proven on this article.
Every of the GMM distributions has its personal likelihood of contributing to the background noise. For instance, there could possibly be a constant background noise that happens more often than not, whereas different sources could be intermittent, such because the noise coming via home windows, and so on. All this must be thought of in our statistical mannequin.
An instance of simulated GMM knowledge over time (normalized to the sampling time) is proven within the determine under wherein there are two Gaussian noise sources, each of zero imply however with two totally different variances. On this instance, the decrease variance sign happens extra usually with 90% likelihood therefore the intermittent spikes within the generated knowledge representing the sign with greater variance.
In different situations and relying on the appliance, it could possibly be the opposite approach round wherein the excessive variance noise sign happens extra usually (as shall be proven in a later instance on this article). Python code used to generate and analyze GMM knowledge can even be proven later on this article.
Turning to a extra formal modelling language, let’s assume that the background noise sign that’s collected (utilizing a high-quality microphone for instance) is modeled as realizations of unbiased and identically distributed (iid) random variables that observe a GMM as proven under.
The modeling drawback thus boils all the way down to estimating the mannequin parameters (i.e., p1, σ²1, and σ²2) utilizing the noticed knowledge (iid). On this article, we shall be utilizing the methodology of moments (MoM) estimator for such objective.
To simplify issues additional, we are able to assume that the noise variances (σ²1 and σ²2) are recognized and that solely the blending parameter (p1) is to be estimated. The MoM estimator can be utilized to estimate a couple of parameter (i.e., p1, σ²1, and σ²2) as proven in Chapter 9 of the e book: “Statistical Sign Processing: Estimation Concept”, by Steven Kay. Nonetheless, on this instance, we are going to assume that solely p1 is unknown and to be estimated.
Since each gaussians within the GMM are zero imply, we are going to begin with the second second and attempt to get hold of the unknown parameter p1 as a perform of the second second as follows.
Observe that one other easy methodology to acquire the moments of a random variable (e.g., second second or greater) is through the use of the second producing perform (MGF). A great textbook in likelihood idea that covers such matters, and extra is: “Introduction to Chance for Information Science”, by Stanley H. Chan.
Earlier than continuing any additional, we want to quantify this estimator by way of the basic properties of estimators corresponding to bias, variance, consistency, and so on. We are going to confirm this later numerically with a Python instance.
Beginning with the estimator bias, we are able to present that the above estimator of p1 is certainly unbiased as follows.
We will then proceed to derive the variance of our estimator as follows.
It’s also clear from the above evaluation that the estimator is constant since it’s unbiased and likewise its variance decreases when the pattern measurement (N) will increase. We can even use the above formulation of the p1 estimator variance in our Python numerical instance (proven intimately later on this article) when evaluating idea with sensible numerical outcomes.
Now let’s introduce some Python code and do some enjoyable stuff!
First, we generate our knowledge that follows a GMM with zero means and normal deviations equal to 2 and 10, respectively, as proven within the code under. On this instance, the blending parameter p1 = 0.2, and the pattern measurement of the information equals 1000.
# Import the Python libraries that we'll want on this GMM instance
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
# GMM knowledge era
mu = 0 # each gaussians in GMM are zero imply
sigma_1 = 2 # std dev of the primary gaussian
sigma_2 = 10 # std dev of the second gaussian
norm_params = np.array([[mu, sigma_1],
[mu, sigma_2]])
sample_size = 1000
p1 = 0.2 # likelihood that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the part
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, substitute=True,
p=mixing_prob)
# GMM_data is the GMM pattern knowledge
GMM_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx),
dtype=np.float64)Then we plot the histogram of the generated knowledge versus the likelihood density perform as proven under. The determine exhibits the contribution of each Gaussian densities within the total GMM, with every density scaled by its corresponding issue.
The Python code used to generate the above determine is proven under.
x1 = np.linspace(GMM_data.min(), GMM_data.max(), sample_size)
y1 = np.zeros_like(x1)
# GMM likelihood distribution
for (l, s), w in zip(norm_params, mixing_prob):
y1 += stats.norm.pdf(x1, loc=l, scale=s) * w
# Plot the GMM likelihood distribution versus the information histogram
fig1, ax = plt.subplots()
ax.hist(GMM_data, bins=50, density=True, label="GMM knowledge histogram",
shade = GRAY9)
ax.plot(x1, p1*stats.norm(loc=mu, scale=sigma_1).pdf(x1),
label="p1 × first PDF",shade = GREEN1,linewidth=3.0)
ax.plot(x1, (1-p1)*stats.norm(loc=mu, scale=sigma_2).pdf(x1),
label="(1-p1) × second PDF",shade = ORANGE1,linewidth=3.0)
ax.plot(x1, y1, label="GMM distribution (PDF)",shade = BLUE2,linewidth=3.0)
ax.set_title("Information histogram vs. true distribution", fontsize=14, loc="left")
ax.set_xlabel('Information worth')
ax.set_ylabel('Chance')
ax.legend()
ax.grid()After that, we compute the estimate of the blending parameter p1 that we derived earlier utilizing MoM and which is proven right here once more under for reference.
The Python code used to compute the above equation utilizing our GMM pattern knowledge is proven under.
# Estimate the blending parameter p1 from the pattern knowledge utilizing MoM estimator
p1_hat = (sum(pow(x,2) for x in GMM_data) / len(GMM_data) - pow(sigma_2,2))
/(pow(sigma_1,2) - pow(sigma_2,2))With a view to correctly assess this estimator, we use Monte Carlo simulation by producing a number of realizations of the GMM knowledge and estimate p1 for every realization as proven within the Python code under.
# Monte Carlo simulation of the MoM estimator
num_monte_carlo_iterations = 500
p1_est = np.zeros((num_monte_carlo_iterations,1))
sample_size = 1000
p1 = 0.2 # likelihood that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the part
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, substitute=True,
p=mixing_prob)
for iteration in vary(num_monte_carlo_iterations):
sample_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx))
p1_est[iteration] = (sum(pow(x,2) for x in sample_data)/len(sample_data)
- pow(sigma_2,2))/(pow(sigma_1,2) - pow(sigma_2,2))Then, we examine for the bias and variance of our estimator and evaluate to the theoretical outcomes that we derived earlier as proven under.
p1_est_mean = np.imply(p1_est)
p1_est_var = np.sum((p1_est-p1_est_mean)**2)/num_monte_carlo_iterations
p1_theoritical_var_num = 3*p1*pow(sigma_1,4) + 3*(1-p1)*pow(sigma_2,4)
- pow(p1*pow(sigma_1,2) + (1-p1)*pow(sigma_2,2),2)
p1_theoritical_var_den = sample_size*pow(sigma_1**2-sigma_2**2,2)
p1_theoritical_var = p1_theoritical_var_num/p1_theoritical_var_den
print('Pattern variance of MoM estimator of p1 = %.6f' % p1_est_var)
print('Theoretical variance of MoM estimator of p1 = %.6f' % p1_theoritical_var)
print('Imply of MoM estimator of p1 = %.6f' % p1_est_mean)
# Under are the outcomes of the above code
Pattern variance of MoM estimator of p1 = 0.001876
Theoretical variance of MoM estimator of p1 = 0.001897
Imply of MoM estimator of p1 = 0.205141We will observe from the above outcomes that the imply of the p1 estimate equals 0.2051 which may be very near the true parameter p1 = 0.2. This imply will get even nearer to the true parameter because the pattern measurement will increase. Thus, we’ve numerically proven that the estimator is unbiased as confirmed by the theoretical outcomes completed earlier.
Furthermore, the pattern variance of the p1 estimator (0.001876) is nearly equivalent to the theoretical variance (0.001897) which is gorgeous.
It’s at all times a contented second when idea matches observe!
All photographs on this article, except in any other case famous, are by the creator.