Real Analytic Function in Hilbert space

$\quad$ We study representations of functions of inner products via embeddings into Hilbert spaces. While polynomials with nonnegative coefficients admit a single feature map realization, this approach fails when negative coefficients are present. To address this limitation, we introduce a two-map framework. Specifically, for any analytic function $f(x) = \sum_{k=0}^{\infty} a_k x^k$ with global convergence, we construct a Hilbert space $\mathcal{H}$ and transformations $\Phi, \Psi: \mathbb{R}^n \to \mathcal{H}$ such that $$\langle \Phi(u), \Psi(v) \rangle = f(\langle u, v \rangle) \quad \text{for all } u, v \in \mathbb{R}^n.$$

1. Real analytic function

We start with the following problem:

Let $P(x)$ be a polynomial with nonnegative coefficients, then there exists a Hilbert space $\mathcal{H}$ and a transformation $\Phi : \mathbb{R}^n \to \mathcal{H}$ such that: $$\langle \Phi(u), \Phi(v) \rangle = P(\langle u, v \rangle) \quad \text{for all } u, v \in \mathbb{R^n}.$$ But our task can be impossible if $P$ has a negative coefficient since $\langle \Phi(u), \Phi(v) \rangle$ is always nonnegative. Therefore, a natural question is what if we consider two transformations $\Phi, \Psi: \mathbb{R}^n \to \mathcal{H}?$ This leads us to a more general problem about analytic function as follow.

Consider a function $$ f(x) = \sum_{k=0}^{\infty} a_k x^k $$ where the series absolutely converges for all $x \in \mathbb{R}$. There exist a Hilbert space $\mathcal{H}$ and transformations $\Phi, \Psi : \mathbb{R}^n \to \mathcal{H}$ such that $$ \langle \Phi(u), \Psi(v) \rangle = f(\langle u, v \rangle) \quad \text{for all } u, v \in \mathbb{R}^n. $$ Also, for any unit vector $u$, we have $$ \|\Phi(u)\|^2_\mathcal{H} = \|\Psi(u)\|^2_\mathcal{H} =\sum _{k=0}^{\infty} |a_k| .$$ Proof.

For each $k \ge 0$, let $V_k = \mathbb{R}^{n^k}$ and define $$ H = \bigoplus_{k=0}^{\infty} V_k = \left\{ (w_0, w_1, \dots) : w_k \in V_k,\ \sum_{k=0}^{\infty} \|w_k\|^2 < \infty \right\} $$

with inner product $$ \langle w, z \rangle_\mathcal{H} = \sum_{k=0}^{\infty} \langle w_k, z_k \rangle_{V_k}. $$

Consider $s_k = \operatorname{sign}(a_k)$ and write $a_k = s_k |a_k|$. For $u \in \mathbb{R}^n$, define $$ \Phi(u) = (\sqrt{|a_0|} u^{\otimes 0}, \sqrt{|a_1|} u^{\otimes 1}, \dots), $$

$$ \Psi(v) = (s_0 \sqrt{|a_0|} v^{\otimes 0}, s_1 \sqrt{|a_1|} v^{\otimes 1}, \dots). $$

Then we prove that $\Phi(u), \Psi(v) \in \mathcal{H}$. Indeed, we have: $$ \left\|\sqrt{|a_k|} u^{\otimes k}\right\|^2 = |a_k| \left\|u^{\otimes k}\right\|^2 = |a_k| \|u\|^{2k}. $$

Thus, $$ \sum_{k=0}^{\infty}\left\|\sqrt{|a_k|} u^{\otimes k}\right\|^2 = \sum_{k=0}^{\infty} |a_k| \|u\|^{2k} $$ converges, so $\Phi(u) \in \mathcal{H}$. Similarly, $\Psi(v) \in \mathcal{H}$.

On the other hand, $$ \begin{aligned} \langle \Phi(u), \Psi(v) \rangle_\mathcal{H} &= \sum_{k=0}^{\infty} \left\langle \sqrt{|a_k|} u^{\otimes k}, s_k \sqrt{|a_k|} v^{\otimes k} \right\rangle_{V_k} \\ &= \sum_{k=0}^{\infty} s_k |a_k| \langle u^{\otimes k}, v^{\otimes k} \rangle\\ &= \sum_{k=0}^{\infty} a_k \langle u, v \rangle^k \\ &= f(\langle u, v \rangle). \end{aligned} $$

Note. Norm when $u$ is a unit vector: If $\|u\| = 1$, then $$ \|u^{\otimes k}\|^2 = \|u\|^{2k} = 1. $$

Hence, $$\|\Phi(u)\|_\mathcal{H}^2 = \sum _{k=0}^{\infty} |a_k| \| u^{\otimes k} \|^2 = \sum _{k=0}^{\infty} |a_k|.$$

Example

Sine. Let $c > 0$. Then function $ f(x) = \sin(cx) $ is real analytic $$ \sin(cx) = cx - \frac{(cx)^3}{3!} + \frac{(cx)^5}{5!} - \ldots $$

Thus, there exists a Hilbert space $\mathcal{H}$ and transformation $\Phi, \Psi: \mathbb{R}^n \to \mathcal{H}$ such that $$ \langle \Phi(u), \Psi(v) \rangle = \sin(c \langle u, v \rangle)\quad \text{for all } u, v \in \mathbb{R}^n. $$

Also, $\Phi$ and $\Psi$ map unit vectors to unit vectors if $$ 1 = c + \frac{c^3}{3!} + \frac{c^5}{5!} + \ldots = \frac{e^c - e^{-c}}{2}. $$

Hence, $$ c = \ln(1 + \sqrt{2}). $$

2. Kernels and feature maps

Given a function of two variables $ K : \mathcal{X} \times \mathcal{X} \to \mathbb{R},$ on some set sets $\mathcal{X}$, when can we find a Hilbert space $\mathcal{H}$ and a transformation $\Phi : \mathcal{X} \to \mathcal{H}$ so that

$$ \langle \Phi(u), \Phi(v) \rangle = K(u,v)^{\textcolor{gray} 1} \quad \text{for all } u, v \in \mathcal{X}? $$

The answer is given by Mercer theorem² 2. Functions of positive and negative type and their connection with the theory of integral equations (Mercer, 1909). and, more precisely, Moore-Aronszajn theorem:

$-$ The necessary and sufficient condition: $K$ is a positive semidefinite kernel, i.e., for any $u_1, \dots, u_N \in \mathcal{X}$, the matrix $ (K(u_i, u_j))_{i,j=1}^N $ is symmetric and positive semidefinite.

$-$ Transformation $\Phi$ is called a feature map.

$-$ The Hilbert space $\mathcal{H}$ is called a reproducing kernel Hilbert space³ (RKHS). 3. Reproducing Kernel Hilbert Space, Mercer’s Theorem, Eigenfunctions, Nyström Method, and Use of Kernels in Machine Learning: Tutorial and Survey (Ghojogh et al, 2021).

$-$ Popular positive semidefinite kernels in machine learning include the Gaussian and polynomial kernels given by: $$ K(u,v) = \exp \left( -\frac{\|u - v\|^2}{2\sigma^2} \right), \quad K(u,v) = (\langle u, v \rangle + r)^k, \quad u, v \in \mathbb{R}^n. $$ where $\sigma > 0, r > 0,$ and $k \in \mathbb{N}$ are parameters.

References

[1] Vershynin, R. (2026). High-dimensional probability: An introduction with applications in data science (2nd ed.). Cambridge University Press.