Simultaneous Parameter Estimation in AWGN

Most of the cases considered thus far have involved the estimation of a single parameter.

However, often it is important to perform simultaneous estimation of multiple parameters.

In this case, previous parameter estimation results can be extended and a generalization of the Cramér-Rao bound can be used for bounding the variances of the individual estimates.

In this section, we will assume that all estimated parameters are scalar unknowns rather than random variables.

Assume that \( \hat{\alpha}_i \), for \( i = 1, \dots, L \), are unbiased estimates of the \( L \) parameters \( \alpha_i \), \( i = 1, \dots, L \).

Then, the covariance of the individual unbiased estimates, \( \text{Cov}(\hat{\alpha}_i) \), has a Cramér-Rao bound given by

\[ \text{Cov}(\hat{\alpha}_i) \geq G_{ii}, \quad i = 1, \dots, L \]

where \( G_{ij} \) are elements of a covariance matrix obtained from the inverse of another matrix \( F \), i.e.,

\[ G = F^{-1} \]

and

\[\begin{split} \begin{align*} F &= F_{ij} \\ &= \mathbb{E} \left\{ \frac{\partial \ln p(\vec{y}|\vec{\alpha})}{\partial \alpha_i} \frac{\partial \ln p(\vec{y}|\vec{\alpha})}{\partial \alpha_j} \right\} \\ &= -\mathbb{E} \left\{ \frac{\partial^2 \ln p(\vec{y}|\vec{\alpha})}{\partial \alpha_i \partial \alpha_j} \right\}, \quad i, j = 1, \dots, L \end{align*} \end{split}\]

The matrix \( F \) is referred to as Fisher’s information matrix, and an unbiased estimate attains the lower bound if

\[ \frac{\partial \ln p(\vec{y}|\vec{\alpha})}{\partial \vec{\alpha}} = F(\vec{\alpha})(\hat{\vec{\alpha}} - \vec{\alpha}) \]

where taking a derivative with respect to a vector is defined in a number of textbooks.

Now, extending the result to a transformation of a \( q \)-dimensional function \( \vec{\Psi}(\vec{\alpha}) \) and an \( L \)-dimensional parameter vector \( \vec{\alpha} = [\alpha_1, \dots, \alpha_L]^T \), the Cramér-Rao lower bound can be represented and is given by

\[ \text{Cov}(\hat{\vec{\alpha}}) \geq \frac{\partial \vec{\Psi}(\vec{\alpha})}{\partial \vec{\alpha}} F^{-1}(\vec{\alpha}) \frac{\partial \vec{\Psi}(\vec{\alpha})^T}{\partial \vec{\alpha}} \]

where

\[\begin{split} \frac{\partial \vec{\Psi}(\vec{\alpha})}{\partial \vec{\alpha}} = \begin{bmatrix} \frac{\partial \Psi_1(\vec{\alpha})}{\partial \alpha_1} & \cdots & \frac{\partial \Psi_1(\vec{\alpha})}{\partial \alpha_L} \\ \vdots & \ddots & \vdots \\ \frac{\partial \Psi_q(\vec{\alpha})}{\partial \alpha_1} & \cdots & \frac{\partial \Psi_q(\vec{\alpha})}{\partial \alpha_L} \end{bmatrix} \end{split}\]

An additional expression for the Fisher matrix coefficients used to obtain the Cramér-Rao bound in the case of random parameters in Gaussian noise is provided as follows.

Assume that the Gaussian noise has a \( k \)-dimensional mean vector \( \vec{\mu}(\vec{\alpha}) \), a \( k \times k \) covariance matrix \( C(\vec{\alpha}) \), and an \( L \)-dimensional parameter vector \( \vec{\alpha} \).

Then the coefficients are given by

\[\begin{split} \begin{align*} F(\vec{\alpha})_{ij} &= \left[\frac{\partial \vec{\mu}(\vec{\alpha})}{\partial \alpha_i}\right]^T C^{-1}(\vec{\alpha}) \left[\frac{\partial \vec{\mu}(\vec{\alpha})}{\partial \alpha_j}\right] \\ &+ \frac{1}{2} \operatorname{tr} \left[ C^{-1}(\vec{\alpha}) \frac{\partial C(\vec{\alpha})}{\partial \alpha_i} C^{-1}(\vec{\alpha}) \frac{\partial C(\vec{\alpha})}{\partial \alpha_j} \right] \end{align*} \end{split}\]

where \( \operatorname{tr} \) is the trace of the matrix and

\[\begin{split} \frac{\partial \vec{\mu}(\vec{\alpha})}{\partial \alpha_i} = \begin{bmatrix} \left(\frac{\partial \vec{\mu}(\vec{\alpha})}{\partial \alpha_i}\right)_1 \\ \vdots \\ \left(\frac{\partial \vec{\mu}(\vec{\alpha})}{\partial \alpha_i}\right)_k \end{bmatrix} \end{split}\]

For a scalar parameter \( \alpha \), \(F(\vec{\alpha})_{ij}\) can be written as

\[\begin{split} \begin{align*} F(\alpha)_{ij} &= \left[\frac{\partial \vec{\mu}(\alpha)}{\partial \alpha}\right]^T C^{-1}(\alpha) \left[\frac{\partial \vec{\mu}(\alpha)}{\partial \alpha}\right] \\ &+ \frac{1}{2} \operatorname{tr} \left[ \left(C^{-1}(\alpha) \frac{\partial C(\alpha)}{\partial \alpha}\right)^2 \right] \end{align*} \end{split}\]

Example: Joint Estimation of Amplitude and Phase

This example [B2, Ex. 11.9] is an example of the application of the foregoing equations, consider the joint estimation of amplitude and phase.

The primary focus is on establishing the theoretical limits of estimating the amplitude \(a\) and phase \(\theta\) simultaneously in an AWGN environment.

Recall from the non-coherent amplitude estimation that the conditional pdf of the received signal, given the amplitude and phase, is given by

\[ p(\vec{y}|a, \theta) = \kappa_1 \exp\left(\frac{a}{\sigma^2} y_I \cos \theta + \frac{a}{\sigma^2} y_Q \sin \theta - \frac{a^2 k}{4\sigma^2}\right) \]

Now, with \( \mathcal{E} = k/2 \), we have

\[ p(\vec{y}|a, \theta) = \kappa_1 \exp\left(\frac{a}{\sigma^2} y_I \cos\theta + \frac{a}{\sigma^2} y_Q \sin\theta - \frac{a^2 \mathcal{E}}{2\sigma^2}\right) \]

The elements of the Fisher’s information matrix are then

\[ F_{11} = -\mathbb{E}\left\{\frac{\partial^2 \ln p(\vec{y}|a, \theta)}{\partial a^2}\right\} = \frac{\mathcal{E}}{\sigma^2} \]

\[ F_{12} = -\mathbb{E}\left\{\frac{\partial^2 \ln p(\vec{y}|a, \theta)}{\partial a \partial \theta}\right\} = F_{21} = 0 \]

and

\[\begin{split} \begin{align*} F_{22} &= -\mathbb{E}\left\{\frac{\partial^2 \ln p(\vec{y}|a, \theta)}{\partial \theta^2}\right\} \\ &= \frac{a \cos\theta}{\sigma^2} \mathbb{E}\{y_I\} + \frac{a \sin\theta}{\sigma^2} \mathbb{E}\{y_Q\} \\ &= \frac{a^2\mathcal{E}}{\sigma^2} \end{align*} \end{split}\]

Since \( \mathbb{E}\{y_I\} = a\mathcal{E}\cos\theta \) and \( \mathbb{E}\{y_Q\} = a\mathcal{E}\sin\theta \).

Therefore, Fisher’s information matrix and its inverse are given by

\[\begin{split} F = \begin{bmatrix} \mathcal{E}/\sigma^2 & 0 \\ 0 & a^2\mathcal{E}/\sigma^2 \end{bmatrix} \end{split}\]

and

\[\begin{split} G = F^{-1} = \begin{bmatrix} \frac{1}{\mathcal{E}/\sigma^2} & 0 \\ 0 & \frac{1}{a^2\mathcal{E}/\sigma^2} \end{bmatrix} \end{split}\]

Thus, \( G_{11} = \frac{1}{\mathcal{E}/\sigma^2} \) and \( G_{22} = \frac{1}{a^2\mathcal{E}/\sigma^2} \) represent the minimum variances for the amplitude and phase estimates, respectively.

Note that the minimum variances of the amplitude and phase estimates are computed in previous section, respectively.

For this example, there is no degradation in estimating performance.

From this resilience of the joint estimation of amplitude and phase, we can see that the variance of amplitude estimation when there is no knowledge of phase, i.e., the noncoherent case, is identical to that when there is perfect knowledge of phase, i.e., the coherent case.