Biorthogonal Basis and Reproducing Kernels
This note initially explores the concept of a biorthogonal basis in a finite vector space. It subsequently applies a similar methodology to derive the reproducing kernel basis in function spaces, enabling the approximation of functions using their pointwise values and the associated dual basis.
Biorthogonal Basis in A Vector Space
Let $v_i$ denote basis vectors of a space, which are not necessarily orthogonal. There exists a biorthogonal basis ${w_i}$ such that
\[\langle v_i, w_j\rangle=\delta_{ij}\notag,\]where $\langle , \rangle$ denotes the dot product.
A vector $x$ can be projected onto the basis ${w_i}$ as follows:
\[x = \sum_i \alpha_i w_i, \label{eqn:projection}\]where
\[\alpha_i = \langle x, v_i\rangle. \label{eqn:dotproduct}\]Here, $\alpha_i$ are the coefficients obtained from the dot products of $x$ with the corresponding basis vectors ${v_i}$.
These Equations ($\ref{eqn:projection}$) and ($\ref{eqn:dotproduct}$) demonstrate that every vector $x$ can be expressed as a linear combination of the biorthogonal basis ${w_i}$, with the coefficients $\alpha_i$ determined by its dot products with the basis vectors ${v_i}$.
Calculation of Biorthogonal Basis
Consider matrix $V$ whose columns represent the basis vectors.
\[V= \left [ \begin{array}{c} \vert & & \vert & &\vert \\ v_1 & \cdots &v_i &\cdots & v_n \\ \vert & & \vert & &\vert \end{array} \right] \notag.\]Similarly, let $W$ be a matrix whose columns are the biorthogonal basis vectors.
By definition,
\[V^T W= W^T V = I, \notag\]where $I$ is the identify matrix. Thus, $W = \left( V^T\right)^{-1}$.
For example, in a two-dimensional space where $v_1=(1, 2)$, $v_2=(3,1)$, the corresponding biorthogonal basis vectors are $w_1 = (-\frac{1}{5}, \frac{3}{5})$ and $w_2=(\frac{2}{5}, -\frac{1}{5})$.
Figure 1 illustrates the basis vectors $v_1$ and $v_2$ along with their corresponding biorthogonal dual basis vectors $w_1$ and $w_2$.
This visualization clarifies the relationship between the original basis vectors and their biorthogonal counterparts.
Projection Onto Biorthogonal Dual Basis
Consider the projection of vector $x$ onto the biorthogonal dual baiss ${w_i}$:
\[x = W \beta \notag.\]Multiplying both sides by $V^T$ and using $V^T W =I$, we obtain
\[\beta = V^T x. \notag\]Thus, $\beta_i = \langle x, v_i\rangle$. as indicated by Equations ($\ref{eqn:projection}$) and ($\ref{eqn:dotproduct}$).
In the context of the two-dimensional example, suppose $x=(4,3)$ (see Figure 2). We verify that the projection to the biorthogonal basis:
Projection to the basis vectors:
\[\begin{align}\notag \alpha_1 &=\langle x, v_1\rangle =[1,2]^T\cdot [4,3]=10\\ \notag \alpha_2 &= \langle x, v_2\rangle =[3,1]^T\cdot [4,3]=15\notag \end{align}\]Constructing with the biorthogonal dual basis ${w_i}$:
\[\alpha_1 w_1 + \alpha_2 w_2 = 10 \left[-\frac{1}{5}, \frac{3}{5}\right] +15\left[\frac{2}{5}, -\frac{1}{5}\right] = \left[4, 3\right] = x \notag\]This confirms that the constructed vector matches the original vector $x$. This demonstrates the efficacy of projecting vectors onto their biorthogonal bases, ensuring consistency with the original vectors in the space.
Reproducing Kernel Bais in Function Space
The concept explored in the preceding sections extends naturally to function spaces. Let $\varphi_i(x)$ be the basis functions and $\psi_i(x)$ be their corresponding biorthogonal basis functions. Any function $f(x)$ can be represented as a linear combination of $\psi_i(x)$, with coefficients given by inner products of $f(x)$ and $\varphi_i(x)$:
\[f(x)= \sum_i \langle f(y), \varphi_i(y)\rangle \psi_i(x),\label{eqn:funcproj}\]where $\langle , \rangle$ denotes the generalized inner product, for example, $\langle f(y), \varphi_i(y)\rangle = \int_Y f(y) \varphi_i(y) d y$.
Specific basis functions $\varphi_i(x)$ can be chosen such that $\langle f(y), \varphi_i(y)\rangle = f(\lambda_i)$, where $\lambda_i$ are predefined constants. These basis functions $\varphi_i(x)$ are termed reproducing kernel bases, enabling any function $f(x)$ to be reconstructed using its values at $\lambda_i$ and the dual basis $\psi_i(x)$.
\[f(x) = \sum_i f(\lambda_i) \psi_i(x). \label{eqn:pointconstruct}\]Construction of Reproducing Kernel Basis
Let $\phi_k(x)$ denote orthonormal basis functions in the function space. The reproducing kernel basis functions. associated with $\lambda_i$ are constructed as follows:
\[\varphi_i(x) =\sum_k \phi_k(\lambda_i)\phi_k(x). \label{eqn:rk}\]Using the basis function of the Equation ($\ref{eqn:rk})$, we find:
\[\begin{align} \notag \langle f(y), \varphi_i(y)\rangle &= \langle{f(y), \sum_k \phi_k(\lambda_i)\phi_k(y)}\rangle \\ \notag & = \sum_k\phi_k(\lambda_i)\langle f(y), \phi_k(y)\rangle \\ \notag & = \sum_k \phi_k(\lambda_i) f_k \\ \notag & = f(\lambda_i), \end{align}\]which verifies Equation ($\ref{eqn:pointconstruct}$) using the reproducing kernel bases defined in Equation ($\ref{eqn:rk}$).
Dual Basis of the Reproducing kernels
With the reproducing kernels $\varphi_i(x)$ constructed by Equation ($\ref{eqn:rk}$), the dual basis $\psi_i(x)$ can be computed.
\[\langle \varphi_i, \psi_j\rangle = \delta_{ij} \label{eqn:rkdual}\]Express $\varphi_i$ and $\psi_i$ in terms of the orthonormal basis $\phi_i$ using matrices $A$ and $B$:
\[A_{ij}= \langle \varphi_i(x), \phi_j(x)\rangle,\ \ \ B_{ij}=\langle \psi_i(x) \phi_j(x)\rangle. \notag\]The condition in Equation ($\ref{eqn:rkdual}$) leads to
\[A^T B = B^T A = I. \notag\]Thus,
\[B = \left( A^T\right)^{-1}, \label{eqn:B}\]where the inverse is the pseudoinverse if $A$ is not square.
Therefore,
\[\psi_i(x) = \sum_k B_{ik} \phi_k(x). \label{eqn:psi}\]An Example
Consider approximating a function $f(x)$ on $x\in [-1,1]$ using reproducing kernels and their dual basis, we will construct them using orthonormal Legendre polynomials $\phi_i(x)$, normalized for $n=0, 1, 2, 3, 4$.
\[\begin{align}\notag \phi_1(x) &= P_0 (x) = \frac{1}{\sqrt{2}} \\ \notag \phi_2(x) & = P_1 (x) = \sqrt{\frac{3}{2}}x \\ \notag \phi_3(x) &= P_2(x) = \sqrt{\frac{5}{2}} (-1 + 3 x^2) \\ \notag \phi_4(x) &= P_3(x) = \frac{1}{2}\sqrt{\frac{7}{2}}(-3 x + 5 x^3) \\ \notag \phi_5(x) &= P_4(x) =\frac{3}{8\sqrt{2}}(3 - 30 x^2 + 35 x^4) \end{align}\]Choose $\lambda = [ -0.8, -0.4, 0, 0.4, 0.8]$. The reproducing kernel basis ${\varphi_i(x)}$ is constructed using Equation ($\ref{eqn:rk}$):
\[\varphi = A \left[ \begin{array}{c} \phi_1(x)\\ \vdots\\ \phi_3(x)\\ \vdots \\ \phi_5(x) \end{array} \right], \notag\]where
\[A = \left[ \begin{array}{c} \phi_1(\lambda_1), \cdots, \phi_3(\lambda_1), \cdots, \phi_5(\lambda_1) \\ \cdots \\ \phi_1(\lambda_3), \cdots, \phi_3(\lambda_3), \cdots, \phi_5(\lambda_3) \\ \cdots \\ \phi_1(\lambda_5), \cdots, \phi_3(\lambda_5), \cdots, \phi_5(\lambda_5) \end{array} \right].\]The resulting basis functions are plotted in Figure 3.
The dual basis ${\psi_i(x)}$ is computed using Equations ($\ref{eqn:B}$) and ($\ref{eqn:psi}$):
\[\psi = B \phi = \left(A^T \right)^{-1} \left[\begin{array}{c} \phi_1(x)\\ \vdots\\ \phi_3(x)\\ \vdots\\ \phi_5(x) \end{array} \right] \notag.\]Figure 4 displays the dual basis functions.
Any function $f(x)$ can thus be approximated by its value at $\lambda_i$ and the dual basis functions $\psi_i(x)$. For instance, $f(x) = \sin(3 x)$ can be approximated as:
\[\hat{f}(x) = \sum_i \sin(3 \lambda_i)\psi_i(x). \notag\]Figure 5 compares the approximation to the actual function.
Conclusion
In summary, we explored biorthogonal bases in finite vector spaces and function spaces. In finite spaces, we derived a biorthogonal basis $W$ from a given basis $V$, facilitating the projection of vectors onto $W$. Extending to function spaces, we introduced reproducing kernel bases $\varphi_i(x)$and their dual $\psi_i(x)$, allowing any function $f(x)$ to be represented using these bases and its values at selected points $\lambda_i$.
This exploration demonstrates the practical and theoretical importance of biorthogonal and reproducing kernel bases in functional analysis and numerical approximation.
References
Lessig, C (2014). A Tutorial on Reproducing Kernel Bases