Maximum Likelihood (ML) Estimation

Maximum Likelihood (ML) Estimation#

Bayesian estimation relies on the posterior probability distribution $p (α | \vec{y})$ , through Bayes’ theorem, it requires the knowledge of both:

the prior distribution $p (α)$ , and
the likelihood function $p (\vec{y} | α)$ .

In contrast, ML estimation just requires the knowledge of

the likelihood function $p (\vec{y} | α)$
without the need for any prior distribution or cost function.

Formally, the ML estimation focuses solely on maximizing the likelihood function $p (\vec{y} | α)$ with respect to the parameter $α$ .

Mathematically, the ML estimator is given by:

{\hat{α}}_{ML} (\vec{y}) = \arg max_{\hat{α} (\vec{y})} p (\vec{y} | α)

Determine ML Estimate#

Suppose we have $m$ independent observations $y_{1}, y_{2}, \dots, y_{m}$ of a random variable $y$ , each dependent on an unknown parameter $α$ . Each observation has a probability density function (pdf) $p (y_{i} | α)$ .

Since the observations are independent, the joint pdf of all observations (i.e., the likelihood function) is the product of the individual pdfs:

p (\vec{y} | α) = \prod_{i = 1}^{m} p (y_{i} | α)

The maximum-likelihood estimate ${\hat{α}}_{ML}$ of the parameter $α$ is determined by finding the maximum of this likelihood function based on the observed data $\vec{y}$ .

Derivation Directly From the Likelihood Function#

The likelihood function, i.e., the joint probability of observing the data $\vec{y}$ given the parameter $α$ , is given by

L (α) = p (\vec{y} | α) = \prod_{i = 1}^{m} p (y_{i} | α)

Compute the derivative of the likelihood function with respect to $α$ , and then set the derivative equal to zero:

\frac{\partial}{\partial α} \ln L (α) = 0

Solve the resulting equation for $α$ to find the ML estimate ${\hat{α}}_{ML}$ .

Derivation From the Log-Likelihood Function#

Since the logarithm is a monotonic function, we may have a simpler derivationi if using the log-likelihood function

Specifically, take the natural logarithm of the likelihood function to obtain the log-likelihood function

\ln L (α) = \ln p (\vec{y} | α) = \sum_{i = 1}^{m} \ln p (y_{i} | α)

Note that, the logarithm turns the product into a sum, which is easier to differentiate.

Compute the derivative of the log-likelihood with respect to $α$ , and set the derivative equal to zero:

\frac{\partial}{\partial α} \ln L (α) = \sum_{i = 1}^{m} \frac{\partial}{\partial α} \ln p (y_{i} | α) = 0

Solve the resulting equation for $α$ to find the maximum likelihood estimate ${\hat{α}}_{ML}$ .

Note that this method only works if the (log)-likelihood function is differentiable with respect to $α$ .

Example with Normally Distributed Observations#

Assume each observation $y_{i}$ is normally distributed with unknown mean $α$ and known variance $σ^{2}$ :

y_{i} \sim N (α, σ^{2})

Each individual PDF is given by

p (y_{i} | α) = \frac{1}{\sqrt{2 π σ^{2}}} \exp (- \frac{(y_{i} - α)^{2}}{2 σ^{2}})

Method 1: Compute directly from the likelihood function

Thus, the likelihood function is obtained as

L (α) = \prod_{i = 1}^{m} \frac{1}{\sqrt{2 π σ^{2}}} \exp (- \frac{(y_{i} - α)^{2}}{2 σ^{2}})

Notice that each term in the product is an exponential function. Thus, the entire product can be rewritten as:

L (α) = {(\frac{1}{\sqrt{2 π σ^{2}}})}^{m} \exp (- \sum_{i = 1}^{m} \frac{(y_{i} - α)^{2}}{2 σ^{2}})

Differentiate $L (α)$ with respect to $α$ :

\frac{d L (α)}{d α} = {(\frac{1}{\sqrt{2 π σ^{2}}})}^{m} \exp (- \sum_{i = 1}^{m} \frac{(y_{i} - α)^{2}}{2 σ^{2}}) \cdot (\frac{d}{d α} [- \sum_{i = 1}^{m} \frac{(y_{i} - α)^{2}}{2 σ^{2}}])

Simplifying the derivative inside:

\frac{d}{d α} [- \sum_{i = 1}^{m} \frac{(y_{i} - α)^{2}}{2 σ^{2}}] = \sum_{i = 1}^{m} \frac{(y_{i} - α)}{σ^{2}}

Therefore:

\frac{d L (α)}{d α} = L (α) \cdot (\frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - α))

To find the maximum likelihood estimate (MLE), set $\frac{d L (α)}{d α} = 0$ , where $\hat{α}$ is a root:

L (\hat{α}) \cdot (\frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - \hat{α})) = 0

Since $L (\hat{α})$ is always positive, we can ignore it, leading to:

\sum_{i = 1}^{m} (y_{i} - \hat{α}) = 0 \Rightarrow \hat{α} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}

Method 2. Compute using the log-likelihood function

The log-likelihood function is

\ln L (α) = \sum_{i = 1}^{m} [- \frac{1}{2} \ln (2 π σ^{2}) - \frac{(y_{i} - α)^{2}}{2 σ^{2}}]

Simplify the sum, we have

\ln L (α) = - \frac{m}{2} \ln (2 π σ^{2}) - \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} (y_{i} - α)^{2}

Next, we differentiate the log-likelihood function to find $α$ that returns its maximum value

Specificaly, we compute the derivative with respect to $α$ :

\frac{\partial}{\partial α} \ln L (α) = - \frac{1}{2 σ^{2}} \cdot 2 \sum_{i = 1}^{m} (y_{i} - α) (- 1)

Simplify:

\frac{\partial}{\partial α} \ln L (α) = \frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - α)

Set the derivative equal to zero, assuming $\hat{α}$ is the root:

\frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - \hat{α}) = 0

Multiply both sides by $σ^{2}$ :

\sum_{i = 1}^{m} (y_{i} - \hat{α}) = 0

Simplify the sum:

\sum_{i = 1}^{m} y_{i} - m \hat{α} = 0

Solve for $\hat{α}$ :

\hat{α} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}

Using either methods, the maximum likelihood estimate of the unknown mean $α$ is the sample mean:

{\hat{α}}_{ML} = \bar{y} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}

Discussion: The Use of Logarithms

The natural logarithm is a monotonic function, meaning it preserves the order of the likelihood values.
Maximizing the log-likelihood is equivalent to maximizing the likelihood itself.
Using the log-likelihood simplifies calculations, especially when dealing with products of probabilities, as it converts them into sums.

For example, in general case, when we attempt to differentiate a product of multiple functions with respect to $α$ , we must apply the product rule from calculus.

Specifically, for a product of two functions, $f (α)$ and $g (α)$ , the derivative is:

$\frac{d}{d α} [f (α) \cdot g (α)] = f^{'} (α) \cdot g (α) + f (α) \cdot g^{'} (α)$

Extending this to $m$ terms becomes increasingly complex, as the product rule must be applied iteratively.

The number of terms in the derivative grows exponentially with $m$ , making the calculation cumbersome and error-prone for large $m$ .

Discussion: Applicability

This method is applicable when the derivatives of the likelihood or log-likelihood functions exist.
In cases where the likelihood function is complex or does not have a closed-form solution, numerical optimization techniques may be employed to find the maximum likelihood estimate.

The ML estimation provides a practical and efficient method for parameter estimation

based solely on observed data
without requiring prior distributions or cost functions.

Bayes vs. ML Estimation#

Bayesian estimation can yield more accurate estimates when the prior distribution and cost functions are correctly specified.
- However, it may lead to incorrect inferences if these are inaccurately modeled.
On the other hand, ML estimation is often favored for its simplicity and because it depends solely on the observed data, making it easier to compute.
- However, it is sometimes criticized for ignoring prior information about the parameter (in case if such prior information is already available).

MAP vs. ML Estimation#

To intuitively understand the difference between ML estimation and MAP estimation, let’s start with Bayes’ theorem, which relates the posterior probability distribution $p (α | \vec{y})$ to the likelihood $p (\vec{y} | α)$ and the prior $p (α)$ :

p (α | \vec{y}) = \frac{p (\vec{y} | α) p (α)}{p (\vec{y})}

recall that:

$p (α | \vec{y})$ is the posterior probability of the parameter $α$ given the observed data $\vec{y}$ .
$p (\vec{y} | α)$ is the likelihood of observing the data $\vec{y}$ given the parameter $α$ .
$p (α)$ is the prior probability of the parameter $α$ .
$p (\vec{y})$ is the marginal likelihood or evidence, calculated as $p (\vec{y}) = \int p (\vec{y} | α) p (α) d α$ .

MAP Estimation#

In MAP estimation, we aim to find the value of $α$ that maximizes the posterior distribution $p (α | \vec{y})$ .

This involves solving:

{\hat{α}}_{MAP} = \arg max_{\hat{α} (\vec{y})} p (α | \vec{y})

To find this maximum, we can take the derivative of the logarithm of the posterior distribution with respect to $α$ and set it to zero:

\frac{\partial}{\partial α} \ln p (α | \vec{y}) = 0

Using Bayes’ theorem, the logarithm of the posterior becomes:

\ln p (α | \vec{y}) = \ln p (\vec{y} | α) + \ln p (α) - \ln p (\vec{y})

Taking the derivative with respect to $α$ :

\frac{\partial}{\partial α} \ln p (α | \vec{y}) = \frac{\partial}{\partial α} \ln p (\vec{y} | α) + \frac{\partial}{\partial α} \ln p (α) - \frac{\partial}{\partial α} \ln p (\vec{y})

We can observe that the term $\ln p (\vec{y})$ is a constant with respect to $α$ because it does not depend on $α$ .

Therefore, its derivative is zero:

\frac{\partial}{\partial α} \ln p (\vec{y}) = 0

Simplifying, we get:

\frac{\partial}{\partial α} \ln p (α | \vec{y}) = \frac{\partial}{\partial α} \ln p (\vec{y} | α) + \frac{\partial}{\partial α} \ln p (α)

ML Estimation as a Special Case of MAP Estimation#

In ML estimation, we seek the value of $α$ that maximizes the likelihood function $p (\vec{y} | α)$ , disregarding the prior $p (α)$ :

{\hat{α}}_{ML} = \arg max_{\hat{α} (\vec{y})} p (\vec{y} | α)

This involves solving:

\frac{\partial}{\partial α} \ln p (\vec{y} | α) = 0

Connection Between MAP and ML#

If the prior distribution $p (α)$ is broad and relatively flat, i.e., wide dispersion, (e.g., a uniform distribution over a large interval), then $\ln p (α)$ is approximately constant with respect to $α$ .

Consequently, its derivative is nearly zero:

\frac{\partial}{\partial α} \ln p (α) \approx 0

Therefore, under a non-informative prior, the MAP estimation equation simplifies to:

\frac{\partial}{\partial α} \ln p (α | \vec{y}) \approx \frac{\partial}{\partial α} \ln p (\vec{y} | α)

This shows that the MAP estimate ${\hat{α}}_{MAP}$ approaches the ML estimate ${\hat{α}}_{ML}$ when the prior is non-informative.

We can say that:

ML estimation relies solely on the likelihood function $p (\vec{y} | α)$ and does not require prior knowledge about $α$ .
MAP estimation incorporates prior information through $p (α)$ , potentially improving the estimate if the prior is accurate.
When Prior is Non-Informative: The influence of the prior diminishes, and ML and MAP estimates converge.

Therefore:

Advantage of ML Estimation:
- ML estimation is valuable when no prior information is available
- or when we prefer an estimate based purely on observed data.
Limitation of ML Estimation: Without incorporating prior knowledge (when it is available), the ML estimate may be less accurate than the MAP estimate if relevant prior information exists.

Properties of ML Estimation#

ML estimations have several important properties under regularity conditions:

Regularity Conditions#

For the following properties to hold, certain regularity conditions must be satisfied:

The likelihood function $p (\vec{y} | α)$ must be sufficiently smooth (differentiable) with respect to $α$ .
The parameter space should be open, and the true parameter $α$ should be an interior point.
The Fisher information $F (α)$ must exist and be finite.
The model should not have singularities or discontinuities in the parameter space.

Efficiency#

Recall that an estimator is efficient if it achieves the lowest possible variance, i.e., minimum-variance, among all unbiased estimators for a parameter $α$ .
Cramér-Rao Lower Bound (CRLB): The variance of any unbiased estimator is bounded below by the CRLB:

$Var (\hat{α}) \geq \frac{1}{F (α)}$

where $F (α)$ is the Fisher information defined as:

$F (α) = - E [\frac{\partial^{2}}{\partial α^{2}} \ln p (\vec{y} | α)]$
If an efficient estimator exists, it can be the ML estimation.

Invariance under Transformations#

Recall that an estimator is invariant under a transformation $g (α)$ if the estimator of $g (α)$ is obtained by applying $g$ to the estimator of $α$

Parameter $α$
transformation $g (α)$
estimate of the parameter ${\hat{α}}_{ML}$
estimate of the transformation ${\hat{g}}_{ML}$

We have that, if $g (α)$ is an invertible function, then

ML estimate is invariant ⟺ {\hat{g}}_{ML} = g ({\hat{α}}_{ML})

Proof

Given ${\hat{α}}_{ML}$ is the ML estimation of $α$ , and $g (α)$ is an invertible function.

We want to prove that, the ML estimation of $g (α)$ is:

{\hat{g}}_{ML} = {\hat{θ}}_{ML} = g ({\hat{α}}_{ML})

Let $θ = g (α)$ .

The likelihood function for $θ$ is:

p (\vec{y} | θ) = p (\vec{y} | α)

where $α = g^{- 1} (θ)$ .

Maximizing $p (\vec{y} | θ)$ with respect to $θ$ is equivalent to maximizing $p (\vec{y} | α)$ with respect to $α$ .

Therefore, ${\hat{θ}}_{ML} = g ({\hat{α}}_{ML})$ .

Application. This property allows us to directly obtain the ML estimation of any invertible transformation of $α$ by applying the transformation to ${\hat{α}}_{ML}$ .

Generalized Likelihood Ratio Test (GLRT): The invariance property of ML estimations facilitates hypothesis testing using the GLRT, where unknown parameters are replaced by their ML estimates.

Asymptotic Properties#

Under regularity conditions, ML estimations exhibit desirable asymptotic behavior as the sample size $m$ increases, i.e., $m \to \infty$ :

Asymptotical Efficience#

ML estimations are asymptotically efficient, meaning they achieve the CRLB as the sample size $m$ approaches infinity.

Consistency#

Recall that an estimator $\hat{α}$ is consistent if it converges in probability to the true parameter value $α$ as $m \to \infty$ :

$\hat{α} \overset{p}{\to} α$
ML estimations are consistent estimators under certain conditions, meaning they become increasingly accurate as more data are collected.

Asymptotic Normality#

We define that an estimator is asymptotically normal if the distribution of $\sqrt{m} (\hat{α} - α)$ converges to a normal distribution as $m \to \infty$ :

$\sqrt{m} (\hat{α} - α) \overset{d}{\to} N (0, \frac{1}{F (α)})$
ML estimation Asymptotic Normality: The distribution of the ML estimation approaches a normal distribution centered at $α$ with variance $1 / (m F (α))$ .

Best Asymptotically Normal (BAN)#

We define that an estimator is BAN if it is asymptotically normal with the smallest possible variance among all consistent estimators.
ML estimations achieve the lowest asymptotic variance permitted by the CRLB, making them BAN estimators.

Function of Sufficient Statistics#

Recall that a statistic is sufficient for a parameter $α$ if it captures all the information about $α$ present in the data $\vec{y}$ .
ML estimations are functions of sufficient statistics.
This means that the ML estimation can be calculated from these statistics without any loss of information.

Example: Maximum Likelihood Estimation of the Mean in a Gaussian Distribution#

This example, based on [B2. Ex. 10.14], repeats the above example in greater detail (and different notation).

It also continues from a previous example, C3.7, to compare with Bayes and MAP estimators.

Let’s consider the problem of estimating the true mean $μ$ of a Gaussian (normal) distribution with a known variance $σ^{2}$ , based on $m$ i.i.d. observations $y_{1}, y_{2}, \dots, y_{m}$ .

Each observation $y_{i}$ follows the probability density function (pdf):

p (y_{i} | μ) = f_{y_{i}} (y_{i}; μ, σ) = \frac{1}{\sqrt{2 π σ^{2}}} \exp (- \frac{(y_{i} - μ)^{2}}{2 σ^{2}}), for i = 1, 2, \dots, m

Formulate the Likelihood Function#

Since the observations are independent, the joint pdf (likelihood function) of all observations given $μ$ is the product of the individual pdfs:

p (\vec{y} | μ) = \prod_{i = 1}^{m} p (y_{i} | μ) = {(\frac{1}{\sqrt{2 π σ^{2}}})}^{m} \exp (- \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} (y_{i} - μ)^{2})

Simplify the expression:

p (\vec{y} | μ) = \frac{1}{(2 π σ^{2})^{m / 2}} \exp (- \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} (y_{i} - μ)^{2})

Compute the Log-Likelihood Function#

Taking the natural logarithm of the likelihood function to simplify differentiation:

\ln p (\vec{y} | μ) = \ln (\frac{1}{(2 π σ^{2})^{m / 2}}) - \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} (y_{i} - μ)^{2}

Simplify further:

\ln p (\vec{y} | μ) = - \frac{m}{2} \ln (2 π σ^{2}) - \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} (y_{i} - μ)^{2}

Differentiate the Log-Likelihood with Respect to $μ$ #

Compute the derivative:

\frac{\partial}{\partial μ} \ln p (\vec{y} | μ) = - \frac{1}{2 σ^{2}} \cdot \frac{\partial}{\partial μ} (\sum_{i = 1}^{m} (y_{i} - μ)^{2})

Calculate the derivative inside the summation:

\frac{\partial}{\partial μ} (y_{i} - μ)^{2} = - 2 (y_{i} - μ)

Therefore, the derivative becomes:

\frac{\partial}{\partial μ} \ln p (\vec{y} | μ) = - \frac{1}{2 σ^{2}} \cdot (- 2 \sum_{i = 1}^{m} (y_{i} - μ)) = \frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - μ)

Set the Derivative to Zero to Find the Maximum#

Set the derivative equal to zero to find the maximum likelihood estimate ${\hat{μ}}_{ML}$ .

Since a realization ${\hat{μ}}_{ML}$ is a root of this differential equation, we can rewrite it as:

\frac{\partial}{\partial μ} \ln p (\vec{y} | μ) = \frac{1}{σ^{2}} \sum_{i = 1}^{m} (y_{i} - \hat{μ}) = 0

Multiply both sides by $σ^{2}$ :

\sum_{i = 1}^{m} (y_{i} - \hat{μ}) = 0

Simplify the summation:

\sum_{i = 1}^{m} y_{i} - m \hat{μ} = 0

Solve for $\hat{μ}$ :

m \hat{μ} = \sum_{i = 1}^{m} y_{i} \Rightarrow \hat{μ} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}

The maximum likelihood estimate of $μ$ is the sample mean ${\hat{μ}}_{ML}$ :

{\hat{μ}}_{ML} = \bar{y} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}

Examine the Properties of the Sample Mean as an Estimator#

The sample mean ${\hat{μ}}_{ML}$ possesses several desirable properties (as discussed above for a general ML estimator):

Unbiasedness#

E [{\hat{μ}}_{ML}] = μ

The expected value of the sample mean equals the true mean, indicating that it is an unbiased estimator.

Consistency#

{\hat{μ}}_{ML} \overset{p}{\to} μ as m \to \infty

The estimator converges in probability to the true mean as the sample size increases.

Efficiency#

The sample mean achieves the minimum possible variance among all unbiased estimators of $μ$ .
It attains the CRLB

$Var ({\hat{μ}}_{ML}) = \frac{σ^{2}}{m}$

Best Asymptotically Normal (BAN)#

As $m \to \infty$ , the distribution of ${\hat{μ}}_{ML}$ approaches a normal distribution:

$\sqrt{m} ({\hat{μ}}_{ML} - μ) \overset{d}{\to} N (0, σ^{2})$
It is the best estimator in terms of asymptotic normality.

Sufficiency#

The sample mean is a sufficient statistic for $μ$ .

Indeed, we have the factorization as

p (\vec{y} | μ) = \underset{h (\vec{y})}{\underset{⏟}{{(\frac{1}{\sqrt{2 π σ^{2}}})}^{m} \exp (- \frac{1}{2 σ^{2}} \sum_{i = 1}^{m} y_{i}^{2})}} \cdot \underset{g (\bar{y}, μ)}{\underset{⏟}{\exp (\frac{m μ \bar{y} - \frac{m μ^{2}}{2}}{2 σ^{2}})}}

$h (\vec{y})$ : This part of the factorization depends only on the data $\vec{y}$ and not on the parameter $μ$ .
$g (\bar{y}, μ)$ : This part depends on the data only through the sample mean $\bar{y}$ and the parameter $μ$ .

Thus, according to the Fisher Factorization Theorem, since the joint pdf can be expressed as a product of a function that depends on the data only through $\bar{y}$ and another function that does not depend on $μ$ , the sample mean $\bar{y}$ is a sufficient statistic for $μ$ .

Furthermore, the sufficiency of $\bar{y}$ simplifies analysis and computation because it reduces the data $y$ to a single summary statistic $\bar{y}$ without losing information about $μ$ .

Generalization. ML Estimator as Undesirable Estimator#

In this scenario, the maximum likelihood estimator (ML estimation) coincides with the sample mean, which has all the properties of an excellent estimator.

The favorable properties observed here do not always hold for ML estimations in other contexts.

In some situations, the ML estimation may be biased.

Sample Variance#

Estimating the variance $σ^{2}$ when $μ$ is unknown.

The ML estimation of $σ^{2}$ is:

${\hat{σ}}_{ML}^{2} = \frac{1}{m} \sum_{i = 1}^{m} (y_{i} - {\hat{μ}}_{ML})^{2}$
This estimator is biased because:

$E [{\hat{σ}}_{ML}^{2}] = (\frac{m - 1}{m}) σ^{2} < σ^{2}$

Sample Mean Squared#

Estimating $μ^{2}$ using $({\hat{μ}}_{ML})^{2}$ results in a biased estimator.
The bias arises because:

$E [({\hat{μ}}_{ML})^{2}] = μ^{2} + \frac{σ^{2}}{m}$

which is greater than $μ^{2}$ .

While the ML estimation of the mean in a Gaussian distribution with known variance is an ideal estimator, this is not universally the case for all parameters or distributions.

In some cases, ML estimations can be biased or lack efficiency.

Therefore, it’s important to analyze the properties of the ML estimation in each specific context to understand its suitability and potential limitations.

Maximum Likelihood (ML) Estimation

Contents

Maximum Likelihood (ML) Estimation#

Determine ML Estimate#

Derivation Directly From the Likelihood Function#

Derivation From the Log-Likelihood Function#

Example with Normally Distributed Observations#

Bayes vs. ML Estimation#

MAP vs. ML Estimation#

MAP Estimation#

ML Estimation as a Special Case of MAP Estimation#

Connection Between MAP and ML#

Properties of ML Estimation#

Regularity Conditions#

Efficiency#

Invariance under Transformations#

Asymptotic Properties#

Asymptotical Efficience#

Consistency#

Asymptotic Normality#

Best Asymptotically Normal (BAN)#

Function of Sufficient Statistics#

Example: Maximum Likelihood Estimation of the Mean in a Gaussian Distribution#

Formulate the Likelihood Function#

Compute the Log-Likelihood Function#

Differentiate the Log-Likelihood with Respect to μ#

Set the Derivative to Zero to Find the Maximum#

Examine the Properties of the Sample Mean as an Estimator#

Unbiasedness#

Consistency#

Efficiency#

Best Asymptotically Normal (BAN)#

Sufficiency#

Generalization. ML Estimator as Undesirable Estimator#

Sample Variance#

Sample Mean Squared#

Differentiate the Log-Likelihood with Respect to $μ$ #