Maximum Likelihood Estimator for Log Normal Distribution

Finding the UMVUE and CR Bound

Let $X_1, X_2, X_3, \dots, X_n$ be a random sample drawn from a Bernoulli distribution $B(1, \theta)$. We aim to find the lowest bound of an unbiased estimator of $\theta$ based on these observations, determine the uniformly minimum variance unbiased estimator (UMVUE) of $\theta$, and show that it attains the Cramér-Rao bound.

Step 1: Computing Fisher Information

The regularity conditions are easily satisfied, so the Fisher information is:

`I_X(\theta) = I_X(\theta) \cdot n`

`= n E \left[ \left( \frac{\partial}{\partial \theta} \log f(X_i, \theta) \right)^2 \right]`

Now, the probability mass function is:

`P(X = x) = \theta^x (1 - \theta)^{1 - x}`

Taking the logarithm:

`\log f(X, \theta) = x \log \theta + (1 - x) \log (1 - \theta)`

Computing the derivative:

`\frac{\partial}{\partial \theta} \log f(X, \theta) = \frac{x}{\theta} + \frac{1 - x}{1 - \theta}(-1)`

`= \frac{x - x \theta + \theta x}{\theta (1 - \theta)} = \frac{x - \theta}{\theta (1 - \theta)}`

Thus, Fisher information is:

`I_X(\theta) = n E \left( \frac{x - \theta}{\theta (1 - \theta)} \right)^2`

`= \frac{n}{\theta (1 - \theta)^2} \cdot \theta (1 - \theta)`

`= \frac{n}{\theta (1 - \theta)}`

Step 2: Finding the CRLB

The Cramér-Rao lower bound (CRLB) is given by:

`V(s(X)) \geq \frac{1}{I_X(\theta)} = \frac{\theta (1 - \theta)}{n}`

Step 3: Finding an Unbiased Estimator

Consider:

`\log f(X, \theta) = \left( \sum x_i \right) \log \theta + (n - \sum x_i) \log (1 - \theta)`

`\frac{\partial}{\partial \theta} \log f(X, \theta) = \frac{\sum x_i}{\theta} - \frac{(n - \sum x_i)}{(1 - \theta)}`

`= \frac{\left( \sum x_i / n \right) - \theta}{\left[ \theta (1 - \theta) \right] / n}`

Thus, the unbiased estimator is:

`s(X) = \frac{\sum X_i}{n}`

which is an unbiased estimator of $\theta$. Comparing with the Cramér-Rao lower bound, we see that $s(X)$ attains the bound, making it the UMVUE.