Optimal Detector

MAP Detector

The goal of detection in the vector AWGN channel is to determine which signal \( \vec{s}_m \) was transmitted based on the received vector \( \vec{r} \). The Maximum A Posteriori (MAP) detector selects the message \( \hat{m} \) that maximizes the posterior probability of \( m \) given \( \vec{r} \). Using Bayes’ rule, the MAP decision rule is:

\[ \boxed{ \hat{m} = \arg \max_{1 \leq m \leq M} P_m p(\vec{r} | \vec{s}_m) } \]

where:

• \( P_m \) is the prior probability of message \( m \).

• \( p(\vec{r} | \vec{s}_m) \) is the likelihood of observing \( \vec{r} \) given that \( \vec{s}_m \) was sent.

Since the noise is additive white Gaussian noise (AWGN) with \( \vec{n} = \vec{r} - \vec{s}_m \), its probability density function (PDF) follows:

\[ p(\vec{r} | \vec{s}_m) = p_{\vec{n}}(\vec{r} - \vec{s}_m) = \left( \frac{1}{\sqrt{\pi N_0}} \right)^N \exp\left( -\frac{\|\vec{r} - \vec{s}_m\|^2}{N_0} \right) \]

Substituting this into the MAP rule:

\[ \boxed{ \hat{m} = \arg \max_{1 \leq m \leq M} P_m \left[ \left( \frac{1}{\sqrt{\pi N_0}} \right)^N \exp\left( -\frac{\|\vec{r} - \vec{s}_m\|^2}{N_0} \right) \right] } \]

MAP Decision Rule

Remove Constant Scaling Factor

Since the term \( \left( \frac{1}{\sqrt{\pi N_0}} \right)^N \) is independent of \( m \) and positive, it does not affect the \( \arg \max \) operation and can be dropped:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} P_m \exp\left( -\frac{\|\vec{r} - \vec{s}_m\|^2}{N_0} \right) \]

Apply Logarithm

Since \( \ln(\cdot) \) is a monotonically increasing function, taking the natural logarithm preserves the maximizing \( m \):

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \ln P_m - \frac{\|\vec{r} - \vec{s}_m\|^2}{N_0} \right] \]

Multiply by \( \frac{N_0}{2} \)

Since multiplying by a positive constant does not change the maximization:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \frac{N_0}{2} \ln P_m - \frac{1}{2} \|\vec{r} - \vec{s}_m\|^2 \right] \]

Expand the Squared Norm

Expanding \( \|\vec{r} - \vec{s}_m\|^2 \):

\[ \|\vec{r} - \vec{s}_m\|^2 = \|\vec{r}\|^2 + \|\vec{s}_m\|^2 - 2 \vec{r} \cdot \vec{s}_m \]

where \( \|\vec{s}_m\|^2 = \mathcal{E}_m \) is the energy of signal \( m \). Since \( \|\vec{r}\|^2 \) is independent of \( m \), it can be ignored in the maximization:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \frac{N_0}{2} \ln P_m - \frac{1}{2} \mathcal{E}_m + \vec{r} \cdot \vec{s}_m \right] \]

Define Bias Term

Define a bias term that incorporates prior probability and signal energy:

\[ \eta_m = \frac{N_0}{2} \ln P_m - \frac{1}{2} \mathcal{E}_m \]

The final MAP decision rule simplifies to:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} [ \eta_m + \vec{r} \cdot \vec{s}_m ] \]

Interpretation

• The term \( \vec{r} \cdot \vec{s}_m \) represents the correlation between the received vector and the candidate signal.

• The bias term \( \eta_m \) adjusts for prior probabilities and signal energy differences.

• The MAP detector selects the signal \( \vec{s}_m \) that maximizes this combined score.

Optimal MAP Decision Rule

The Maximum A Posteriori (MAP) decision rule derived earlier is:

\[ \boxed{ \hat{m} = \arg \max_{1 \leq m \leq M} [ \eta_m + \vec{r} \cdot \vec{s}_m ] } \]

where:

\[ \boxed{ \eta_m = \frac{N_0}{2} \ln P_m - \frac{1}{2} \mathcal{E}_m } \]

This rule is optimal in the sense that it minimizes the probability of detection error, balancing:

• Prior probabilities via \( P_m \), adjusting for how likely each message is before reception.

• Likelihood information, through the dot product \( \vec{r} \cdot \vec{s}_m \), which measures the alignment between the received signal \( \vec{r} \) and the candidate signal \( \vec{s}_m \).

• Energy differences across signals via \( \mathcal{E}_m \), ensuring fairness when signals have varying power levels.

Special Case: Equiprobable Signals

When the signals are equiprobable, i.e.,

\[ P_m = \frac{1}{M}, \quad \forall m \]

the MAP rule simplifies significantly.

Simplification of the Prior Term

Starting from the MAP rule:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \frac{N_0}{2} \ln \left( \frac{1}{M} \right) - \frac{1}{2} \|\vec{r} - \vec{s}_m\|^2 \right] \]

Since \( \ln \left( \frac{1}{M} \right) \) is a constant across all \( m \), it does not affect the maximization and can be ignored.

Thus, the decision rule reduces to:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ -\frac{1}{2} \|\vec{r} - \vec{s}_m\|^2 \right] \]

Converting to Minimum Distance Form

Since maximizing \( -\frac{1}{2} \|\vec{r} - \vec{s}_m\|^2 \) is equivalent to minimizing \( \|\vec{r} - \vec{s}_m\|^2 \), we obtain:

\[ \boxed{ \hat{m} = \arg \min_{1 \leq m \leq M} \|\vec{r} - \vec{s}_m\| } \]

This is the minimum-distance detector, also known as the nearest-neighbor detector.

Minimum-Distance Detector

Geometrically, we can see that

• The receiver compares \( \vec{r} \) to each \( \vec{s}_m \) in the \( N \)-dimensional space.

• It selects the signal closest to \( \vec{r} \) in terms of Euclidean distance.

• This is intuitive because when all messages are equally probable, the best estimate is the signal least perturbed by noise, i.e., the one closest to the received vector.

Relationship Between MAP and ML Detection

For equiprobable signals, the MAP detector reduces to the Maximum Likelihood (ML) detector, since the prior term is uniform and does not influence the decision:

\[ \hat{m}_{\text{MAP}} = \hat{m}_{\text{ML}} \]

Thus, in uniform priors, both MAP and ML detection follow the minimum-distance rule, further emphasizing its optimality under such conditions.

Special Case: Equiprobable and Equal-Energy Signals

If the signals are both equiprobable \( (P_m = \frac{1}{M}) \) and have equal energy \( (\mathcal{E}_m = \mathcal{E}) \) for all \( m \), the MAP decision rule simplifies further.

From the general MAP rule:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} [ \eta_m + \vec{r} \cdot \vec{s}_m ] \]

where:

\[ \eta_m = \frac{N_0}{2} \ln \left( \frac{1}{M} \right) - \frac{1}{2} \mathcal{E} \]

Since both \( \ln \left( \frac{1}{M} \right) \) and \( \mathcal{E} \) are constant across all \( m \), the term \( \eta_m \) does not affect the maximization and can be dropped. This simplifies the decision rule to:

\[ \boxed{ \hat{m} = \arg \max_{1 \leq m \leq M} \vec{r} \cdot \vec{s}_m } \]

Correlation Detector

• The receiver computes the dot product \( \vec{r} \cdot \vec{s}_m \) for each possible signal \( \vec{s}_m \).

• It chooses the signal that yields the largest value.

• Physically, this corresponds to projecting \( \vec{r} \) onto each \( \vec{s}_m \) and selecting the signal most aligned with the received vector.

This correlation detector is optimal and computationally efficient when all signals have equal energy, as it eliminates the need to adjust for energy differences.

Note: Dot Product as Correlation

The dot product (or inner product) between two vectors \( \vec{r} \) and \( \vec{s}_m \), denoted as \( \vec{r} \cdot \vec{s}_m \), is computed as:

\[ \vec{r} \cdot \vec{s}_m = \sum_{i=1}^{N} r_i s_{m,i} \]

where:

• \( r_i \) and \( s_{m,i} \) are the \( i \)-th components of vectors \( \vec{r} \) and \( \vec{s}_m \), respectively.

• \( N \) is the dimensionality of the vectors.

The dot product \( \vec{r} \cdot \vec{s}_m \) represents the correlation between the received signal \( \vec{r} \) and the candidate signal \( \vec{s}_m \). The dot product measures how much \( \vec{r} \) and \( \vec{s}_m \) are aligned.
In addition, the mathematical definition of the correlation coefficient is:

\[ \rho(\vec{r}, \vec{s}_m) = \frac{\vec{r} \cdot \vec{s}_m}{\|\vec{r}\| \|\vec{s}_m\|} \]

This is a normalized form of the dot product, ensuring values between \(-1\) and \(1\). The dot product itself captures raw correlation strength without normalization.

Thus, while the dot product is not the exact statistical correlation coefficient, it acts as a correlation measure in this MAP detection problem.

Minimum-Distance Detector vs. Correlation Detector

The vector AWGN channel model:

\[ \vec{r} = \vec{s}_m + \vec{n} \]

provides a robust framework for optimal detection. The MAP detector accounts for prior probabilities and signal energies, leading to two key simplifications:

1. Minimum-Distance Detector:

   • When signals are equiprobable (but not necessarily equal energy), the MAP detector reduces to minimum Euclidean distance detection:

     \[ \hat{m} = \arg \min_{1 \leq m \leq M} \|\vec{r} - \vec{s}_m\| \]

   • This is also the Maximum Likelihood (ML) detector under equal priors.

2. Correlation Detector:

   • When signals are equiprobable and have equal energy, the MAP detector reduces to a correlation detector:

     \[ \hat{m} = \arg \max_{1 \leq m \leq M} \vec{r} \cdot \vec{s}_m \]

   • This is a simpler and more efficient detection strategy that directly measures alignment between \( \vec{r} \) and each signal.

These simplifications reflect practical scenarios (e.g., digital communications with equal-probability signaling) and offer geometric insights into the detection process, bridging mathematical rigor with intuitive understanding.

Decision Regions in Vector AWGN Channels

In the vector AWGN channel, the optimal MAP detector assigns each possible received vector \( \vec{r} \) to one of the \( M \) possible transmitted signals \( \vec{s}_m \) based on the decision rule derived earlier.

The decision region \( D_m \) for a given signal \( \vec{s}_m \) is the set of all received vectors \( \vec{r} \) in the \( N \)-dimensional real space \( \mathbb{R}^N \) that the detector assigns to message \( m \). Mathematically, this region is defined as:

\[ D_m = \{ \vec{r} \in \mathbb{R}^N : \vec{r} \cdot \vec{s}_m + \eta_m > \vec{r} \cdot \vec{s}_{m'} + \eta_{m'}, \quad \forall \, m' \neq m, \, 1 \leq m' \leq M \} \]

where:

• \( \eta_m = \frac{N_0}{2} \ln P_m - \frac{1}{2} \mathcal{E}_m \) is the bias term, incorporating:

  • \( P_m \): The prior probability of message \( m \).

  • \( \mathcal{E}_m = \|\vec{s}_m\|^2 \): The signal energy.

• \( \vec{r} \cdot \vec{s}_m \) is the dot product between the received vector and the signal vector.

This definition means that \( \vec{r} \) belongs to \( D_m \) if the score \( \vec{r} \cdot \vec{s}_m + \eta_m \) for signal \( m \) exceeds the score for all other signals \( m' \neq m \).

Geometric Interpretation

Each decision region \( D_m \) is characterized by up to \( M - 1 \) decision boundaries, one for each competing signal \( \vec{s}_{m'} \). The boundaries between regions are determined by hyperplanes in \( \mathbb{R}^N \), separating the space into distinct areas corresponding to each transmitted signal.

• If signals are equiprobable and equal-energy \( (P_m = \frac{1}{M}, \mathcal{E}_m = \mathcal{E}) \), the decision boundaries are given by the perpendicular bisectors between the signal vectors. This forms a Voronoi diagram in \( N \)-dimensional space.

• When signals have different energies or priors, the boundaries shift, as some signals are favored due to higher prior probabilities or stronger energy levels.

• In low-dimensional cases (e.g., \( N = 2 \)), the decision regions can be visualized as polytopes (e.g., triangles, quadrilaterals). In higher dimensions, they become hyperregions.

Reducing Complexity in Practical Cases

In practice, not all \( M - 1 \) inequalities are always active or necessary. Depending on the geometry of the signal constellation \( \{\vec{s}_1, \vec{s}_2, \dots, \vec{s}_M\} \) and the values of \( \eta_m \), some inequalities may be redundant.

• If a signal \( \vec{s}_{m''} \) is far from \( \vec{s}_m \) compared to a closer signal \( \vec{s}_{m'} \), then the decision for \( \vec{s}_{m'} \) effectively dominates, making the condition involving \( \vec{s}_{m''} \) unnecessary.

• This means that the actual number of effective decision boundaries can be significantly lower than \( M - 1 \) in many cases, reducing computational complexity.

In summary:

• The MAP decision regions partition \( \mathbb{R}^N \) into areas where the received vector is most likely to belong to a particular transmitted signal.

• Decision boundaries are generally hyperplanes, but their placement depends on prior probabilities and signal energies.

• When signals are equiprobable and equal-energy, the detector reduces to a nearest-neighbor classifier in Euclidean space.

• In practical scenarios, not all boundaries are needed, as some conditions may be redundant due to the geometry of the signal set.

Decision Boundaries in Vector AWGN Channels

The boundary between two decision regions \( D_m \) and \( D_{m'} \) occurs where the decision rule assigns equal scores to both signals, i.e.,

\[ \vec{r} \cdot \vec{s}_m + \eta_m = \vec{r} \cdot \vec{s}_{m'} + \eta_{m'} \]

Deriving the Decision Boundary

Rearranging the equation:

\[ \vec{r} \cdot (\vec{s}_m - \vec{s}_{m'}) = \eta_{m'} - \eta_m \]

The decision region \( D_m \) is the set of all \( \vec{r} \) where this expression exceeds the right-hand side:

\[ \vec{r} \cdot (\vec{s}_m - \vec{s}_{m'}) > \eta_{m'} - \eta_m \]

This equation describes a hyperplane in the \( N \)-dimensional space.

Geometric Interpretation

• The vector \( \vec{s}_m - \vec{s}_{m'} \) is normal to the hyperplane, meaning it determines the direction of the boundary.

• The term \( \eta_{m'} - \eta_m \) determines the offset of the hyperplane from the origin, shifting it based on prior probabilities and signal energies.

A hyperplane is a flat, \( (N-1) \)-dimensional surface:

• In 2D, it is a line.

• In 3D, it is a plane.

• In general, it divides the \( N \)-dimensional space into two half-spaces.

Decision Regions as Convex Polytopes

The overall decision space is partitioned into \( M \) such regions, where each \( D_m \) is defined by multiple hyperplanes (one per competing signal \( \vec{s}_{m'} \)).

• If the signal constellation is well-structured (e.g., equal-energy, symmetric placement), each decision region is a convex polytope (a multi-dimensional generalization of a polygon).

• If priors or energy differences vary significantly, the decision regions may become irregular but remain convex in shape.

Metrics for Detection

To facilitate the analysis and implementation of detection rules, we introduce three useful metrics:

1️. Distance Metric
2️. Modified Distance Metric
3️. Correlation Metric

These metrics provide alternative ways to express the decision process, offering connections between the vector and waveform domains.

Distance Metric

\[ D(\vec{r}, \vec{s}_m) = \|\vec{r} - \vec{s}_m\|^2 = \int_{-\infty}^{\infty} (r(t) - s_m(t))^2 dt \]

Interpretation:

• This is the squared Euclidean distance between the received vector \( \vec{r} \) and the signal vector \( \vec{s}_m \).

• In the waveform domain, it corresponds to the energy of the difference signal \( r(t) - s_m(t) \), integrated over all time.

• It quantifies how far \( \vec{r} \) is from \( \vec{s}_m \), making it naturally suited for minimum-distance detection (as seen in the equiprobable case).

Modified Distance Metric

\[ D'(\vec{r}, \vec{s}_m) = -2 \vec{r} \cdot \vec{s}_m + \|\vec{s}_m\|^2 \]

Derivation:
Expanding the distance metric:

\[ \|\vec{r} - \vec{s}_m\|^2 = \|\vec{r}\|^2 - 2 \vec{r} \cdot \vec{s}_m + \|\vec{s}_m\|^2 \]

Since \( \|\vec{r}\|^2 \) is constant for a given \( \vec{r} \) and does not depend on \( m \), it can be omitted when comparing across \( m \), leaving:

\[ D'(\vec{r}, \vec{s}_m) = -2 \vec{r} \cdot \vec{s}_m + \|\vec{s}_m\|^2 \]

Interpretation:

• This metric is equivalent to the distance metric but shifted by a constant, focusing only on the terms that vary with \( m \).

• It provides an alternative way to express the decision process without needing \( \|\vec{r}\|^2 \) explicitly.

Correlation Metric

\[ C(\vec{r}, \vec{s}_m) = 2 \vec{r} \cdot \vec{s}_m - \|\vec{s}_m\|^2 \]

Expanding in the waveform domain:

\[ C(\vec{r}, \vec{s}_m) = 2 \int_{-\infty}^{\infty} r(t) s_m(t) dt - \int_{-\infty}^{\infty} (s_m(t))^2 dt \]

Interpretation:

• This is the negative of the modified distance metric (up to a factor of \( -1 \)), emphasizing correlation.

• The term \( \vec{r} \cdot \vec{s}_m = \int_{-\infty}^{\infty} r(t) s_m(t) dt \) represents the inner product (correlation) between the received signal and the transmitted signal.

• The term \( \|\vec{s}_m\|^2 = \mathcal{E}_m \) represents the signal energy.

• The factor of 2 is a convention that aligns this metric with the MAP rule’s structure, ensuring consistency in decision-making.

Note on Terminology
While called “metrics,” these quantities do not strictly satisfy the mathematical definition of a metric (which requires non-negativity, symmetry, and the triangle inequality).

For example:

• \( C(\vec{r}, \vec{s}_m) \) can be negative.

• \( D'(\vec{r}, \vec{s}_m) \) lacks the \( \|\vec{r}\|^2 \) term.

The term “metric” is used loosely here to indicate their role in measuring relationships between vectors rather than adhering strictly to metric space properties.

MAP Detection Rule with Metrics in Waveform Domain

Using the waveform equivalents, the inner product of the received and transmitted signals is given by

\[ \vec{r} \cdot \vec{s}_m = \int_{-\infty}^{\infty} r(t) s_m(t) dt \]

and the energy of the transmitted signal is

\[ \mathcal{E}_m = \|\vec{s}_m\|^2 = \int_{-\infty}^{\infty} (s_m(t))^2 dt. \]

The MAP detection rule

\[ \hat{m} = \arg \max_{1 \leq m \leq M} [ \eta_m + \vec{r} \cdot \vec{s}_m ] \]

with

\[ \eta_m = \frac{N_0}{2} \ln P_m - \frac{1}{2} \mathcal{E}_m \]

can be rewritten in the waveform domain as

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \frac{N_0}{2} \ln P_m + \int_{-\infty}^{\infty} r(t) s_m(t) dt - \frac{1}{2} \int_{-\infty}^{\infty} (s_m(t))^2 dt \right]. \]

This expression shows that the detector operates directly on the received signal \( r(t) \), correlating it with each \( s_m(t) \) while incorporating prior probabilities and signal energies.

Using the distance metric, the MAP rule can be expressed as

\[ \hat{m} = \arg \max_{1 \leq m \leq M} [ N_0 \ln P_m - D(\vec{r}, \vec{s}_m) ]. \]

Since

\[ D(\vec{r}, \vec{s}_m) = \|\vec{r} - \vec{s}_m\|^2, \]

this formulation aligns with the earlier derivation, where the detector selects the signal that minimizes the distance adjusted by priors.

Using the correlation metric, an equivalent form is

\[ \hat{m} = \arg \max_{1 \leq m \leq M} [ N_0 \ln P_m + C(\vec{r}, \vec{s}_m) ]. \]

Substituting

\[ C(\vec{r}, \vec{s}_m) = 2 \vec{r} \cdot \vec{s}_m - \|\vec{s}_m\|^2, \]

this matches the form

\[ \frac{N_0}{2} \ln P_m + \vec{r} \cdot \vec{s}_m - \frac{1}{2} \mathcal{E}_m \]

up to a scaling factor, confirming consistency. The correlation metric highlights the alignment between \( \vec{r} \) and \( \vec{s}_m \), making it particularly intuitive for implementation in practical detection algorithms.

Maximum Likelihood (ML) Detection

The Maximum Likelihood (ML) detector assumes no prior knowledge (or uniform priors), meaning that the term \( P_m \) is omitted from the MAP decision rule.

Starting from the waveform domain formulation of the MAP rule:

\[ \hat{m} = \arg \max_{1 \leq m \leq M} \left[ \int_{-\infty}^{\infty} r(t) s_m(t) dt - \frac{1}{2} \int_{-\infty}^{\infty} (s_m(t))^2 dt \right]. \]

Recognizing that this expression is exactly half of the correlation metric:

\[ \frac{1}{2} C(\vec{r}, \vec{s}_m), \]

the ML decision rule simplifies to:

\[ \boxed{ \hat{m} = \arg \max_{1 \leq m \leq M} C(\vec{r}, \vec{s}_m). } \]

Thus, the ML detector maximizes the correlation metric, which is equivalent to maximizing \( \vec{r} \cdot \vec{s}_m \) adjusted by the signal energy.

This result aligns with the equiprobable case in MAP detection, where the decision reduces to either a correlation-based rule or a minimum-distance rule, reinforcing the deep connection between these approaches in optimal signal detection.