$$\newcommand{\u}[1]{\boldsymbol{\mathsf{#1}}} \renewcommand{\b}[1]{\boldsymbol{#1}} \newcommand{\t}[1]{\textsf{#1}} \newcommand{\m}[1]{\mathbb{#1}} \def\RR{\bf R} \def\bold#1{\bf #1} \def\mbf#1{\mathbf #1} \def\uv#1{\hat{\usf {#1}}} \def\dl#1{\underline{\underline{#1}}} \newcommand{\usf}[1]{\boldsymbol{\mathsf{#1}}} \def\bs#1{\usf #1}$$

Linear Transformation

Definition A linear transformation from a Eucledian space $\mathbf{X}$ to a Eucledian space $\mathbf{Y}$ is a function

$\begin{align} \boldsymbol{A} : \mathbf{X} \rightarrow \mathbf{Y} \\ \boldsymbol{x} \mapsto \boldsymbol{y} = \boldsymbol{A}\boldsymbol{x} \end{align}$ such that $\begin{align} \boldsymbol{A}(\boldsymbol{x}_1 + \boldsymbol{x}_2) = \boldsymbol{A}\boldsymbol{x}_1 + \boldsymbol{A}\boldsymbol{x}_2 \end{align}$

Matrix form of Linear Representation

A matrix form for the linear transformation $\boldsymbol{A} : \mathbf{X} \rightarrow \mathbf{Y}$ is a matrix $A_{ij}$ that shows how basis elements $\boldsymbol{x}_j \in \mathbf{X}$ map to a linear combination of basis elements $\boldsymbol{y}_i \in \mathbf{Y}$ : $\begin{align} \boldsymbol{x}_j \rightarrow \boldsymbol{A}\boldsymbol{x}_j = \sum_i A_{ij}\boldsymbol{y}_i \end{align}$

Adjoint

Definition Given a linear transformation $\boldsymbol{A} : \mathbf{X} \rightarrow \mathbf{Y}$, then there exists a unique linear transformation (the adjoint) $\begin{align} \boldsymbol{A}^T : \mathbf{X} \rightarrow \mathbf{Y} \end{align}$ that preserves the inner product: $\begin{align} \langle \boldsymbol{y}, \boldsymbol{Ax} \rangle = \langle \boldsymbol{A}^T\boldsymbol{y}, \boldsymbol{x} \rangle \end{align}$ for all $\boldsymbol{x}$ and $\boldsymbol{y}$.

Note: Adjoint is independent of the choice of bases.

Matrix representation of adjoint

If the bases for $\mathbf{X}$ and $\mathbf{Y}$ are each orthonormal, then the matrix representation of the adjoint is the transpose of the matrix representation: $\begin{align} \boldsymbol{A}^T\boldsymbol{y}_i = \sum_j A_{ij}\boldsymbol{x}_j \end{align}$

To prove this relationship, we can verify the adjoint condition (6) for arbitary basis elements:

$$\begin{align} \langle \boldsymbol{A}^T\boldsymbol{y}_i,\boldsymbol{x}_j \rangle &= \langle \sum_k A_{ik}\boldsymbol{x}_k,\boldsymbol{x}_j \rangle \\ &= A_{ij} (\text{since the basis }\boldsymbol{x}_j \text{is orthonormal})\\ &= \langle \boldsymbol{y}_i,\sum_k A_{kj}\boldsymbol{y}_k \rangle (\text{since the basis }\boldsymbol{y}_i \text{is orthonormal}) \\ &= \langle \boldsymbol{y}_i,\boldsymbol{Ax}_j \rangle \end{align}$$

Note: If the bases are not orthogonal,then the transpose of the matrix representation is not the matrix representation of the adjoint.

Riesz Representation

Any linear function $\boldsymbol{X} \rightarrow \mathbb{R}$ can be expressed as $\boldsymbol{x} \mapsto \langle \boldsymbol{y},\boldsymbol{x} \rangle$ for a unique $\boldsymbol{y}$.

For some $\boldsymbol{y} \in \mathbf{X}$, the adjoint of the linear function

$\begin{align} \boldsymbol{y} : \mathbb{R} \mapsto \mathbf{X} \\ z \mapsto \boldsymbol{x} = z\boldsymbol{y} \end{align}$ is given by

$$\begin{align} \boldsymbol{y}^T : \mathbf{X} \rightarrow \mathbb{R}\\ \boldsymbol{x} \mapsto z = \langle \boldsymbol{y},\boldsymbol{x}\rangle \end{align}$$

Verification that the adjoint condition given in (1) holds:

$$\begin{align} \langle \boldsymbol{x},\boldsymbol{y}z\rangle &= \langle \boldsymbol{x},\boldsymbol{y}\rangle z \\ &= \langle \boldsymbol{y},\boldsymbol{x}\rangle z \\ &= \langle \langle \boldsymbol{y},\boldsymbol{x} \rangle,z\rangle \\ &= \langle \boldsymbol{y}^T\boldsymbol{x},z \rangle \end{align}$$

From above, we get

$$\begin{align} \boldsymbol{y}^T\boldsymbol{x} = \langle \boldsymbol{y},\boldsymbol{x} \rangle \end{align}$$

Matrix Form

Suppose $\begin{align} \boldsymbol{y} = \sum_i y_i\boldsymbol{x}_i \end{align}$

for a basis $\boldsymbol{x}_i$. Using the natural orthonormal basis 1 for $\mathbb{R}$, the matrix representation of the linear transformation $\boldsymbol{y}$ is : $\begin{align} 1 \rightarrow \sum_i y_{i}\boldsymbol{x}_i, \end{align}$ such that the vector with components $y_i$ define the matrix representation.

Note: For the adjoint $\boldsymbol{y}^T$, the matrix representation is not the transpose of the vector with components $y_i$, unless the basis $\boldsymbol{x}_i$ is orthonormal.

The matrix representation of the adjoint is:

$$\begin{align} \boldsymbol{x}_j \mapsto \langle \boldsymbol{y}, \boldsymbol{x}_j \rangle \boldsymbol{1} = \langle \sum_i y_i \boldsymbol{x}_i,\boldsymbol{x}_j \rangle 1 = \sum_i \langle \boldsymbol{x}_i,\boldsymbol{x}_j \rangle y_i 1 \end{align}$$

** For a non-orthonormal basis, the matrix representation of the adjoint is not $\boldsymbol{x}_j \mapsto y_j1$.