What is the log-likelihood function?

Jul 20, 2016

It is a term used to denote applying the maximum likelihood approach along with a log transformation on the equation to simplify the equation.

Explanation:

For example suppose i am given a data set $X \in {R}^{n}$ which is basically a bunch of data points and I wanted to determine what the distribution mean is. I would then consider which is the most likely value based on what I know. If I assume the data comes from the normal distribution $N \left(\mu , {\sigma}^{2}\right)$ with $\mu$ as the mean and ${\sigma}^{2}$ as the variance then we have $f \left(X | \mu , {\sigma}^{2}\right) = {\prod}_{i}^{n} \frac{1}{\sqrt{2 \pi {\sigma}^{2}}} {e}^{- \frac{1}{2 {\sigma}^{2}} {\left({x}_{i} - \mu\right)}^{2}}$.

If $\mu$ is not known then I would try to estimate it by way of maximum likelihood or using the equation I would state

$l \left(\mu | X , {\sigma}^{2}\right) = {\prod}_{i}^{n} \frac{1}{\sqrt{2 \pi {\sigma}^{2}}} {e}^{- \frac{1}{2 {\sigma}^{2}} {\left({x}_{i} - \mu\right)}^{2}}$

Here the equation is the same but the paramter of interest is $\mu$. To solve we take the derivative, set it equal to 0 and solve for $\mu$ so we have.

$\frac{\partial}{\partial \mu} {\prod}_{i}^{n} \frac{1}{\sqrt{2 \pi {\sigma}^{2}}} {e}^{- \frac{1}{2 {\sigma}^{2}} {\left({x}_{i} - \mu\right)}^{2}}$

However before doing so I see that I can apply the natural log before finding the derivative to solve for $x$ and simplify the equation thus ...

$\ln \left(l \left(\mu | X , {\sigma}^{2}\right)\right) = {\sum}_{i}^{n} \ln \left(\frac{1}{\sqrt{2 \pi {\sigma}^{2}}}\right) - \frac{1}{2 {\sigma}^{2}} {\left({x}_{i} - \mu\right)}^{2}$

$\frac{\partial}{\partial \mu} {\sum}_{i}^{n} \ln \left(\frac{1}{\sqrt{2 \pi {\sigma}^{2}}}\right) - \frac{1}{2 {\sigma}^{2}} {\left({x}_{i} - \mu\right)}^{2}$
$= \frac{1}{{\sigma}^{2}} {\sum}_{i}^{n} \left({x}_{i} - \mu\right) = 0$
$= \frac{1}{{\sigma}^{2}} {\sum}_{i}^{n} {x}_{i} = \frac{1}{{\sigma}^{2}} {\sum}_{i}^{n} \mu$
$= {\sum}_{i}^{n} {x}_{i} = n \cdot \mu$
$= \frac{1}{n} {\sum}_{i}^{n} {x}_{i} = \mu$

so an approximation of $\mu$ would be the average of the data or $\overline{x} = \frac{1}{n} {\sum}_{i}^{n} {x}_{i}$.

Using MLE we can also find out what the estimated standard deviation is.