Let $M \in \mathscr{M}_{N,d}(\mathbb{R})$ defined by $M_{i,j} = g_{j}(i)$, $q \in \Sigma_{N}$, and for all $\theta \in \mathbb{R}^{d}$, $f(\theta) = M\theta$, $Z(\theta) = \sum_{i=1}^{N} e^{f_{i}(\theta)}$, $$p(\theta) = \left(\frac{e^{f_{1}(\theta)}}{Z(\theta)}, \ldots, \frac{e^{f_{N}(\theta)}}{Z(\theta)}\right) \in \Sigma_{N},$$ and $L(\theta) = \ln(Z(\theta)) - q^{T} M\theta$. We assume that $\ker \widetilde{M} = \{0\}$ and that the function $L$ has a global minimum attained at $\theta_{*}$. We denote $$\Sigma_{N}(\bar{g}, g) = \left\{p \in \Sigma_{N} \mid \sum_{i=1}^{N} p_{i} g_{k}(i) = \bar{g}_{k}, 1 \leqslant k \leqslant d\right\}.$$ (a) Show that $H_{N}(p(\theta_{*})) \geqslant H_{N}(q)$ and then that $H_{N}(p(\theta_{*}))$ is the maximum value of $H_{N}$ on $\Sigma_{N}(\bar{g}, g)$. (b) Show that $p(\theta_{*})$ is the unique point of $\Sigma_{N}(\bar{g}, g)$ at which $H_{N}$ attains its maximum.
Let $M \in \mathscr{M}_{N,d}(\mathbb{R})$ defined by $M_{i,j} = g_{j}(i)$, $q \in \Sigma_{N}$, and for all $\theta \in \mathbb{R}^{d}$, $f(\theta) = M\theta$, $Z(\theta) = \sum_{i=1}^{N} e^{f_{i}(\theta)}$,
$$p(\theta) = \left(\frac{e^{f_{1}(\theta)}}{Z(\theta)}, \ldots, \frac{e^{f_{N}(\theta)}}{Z(\theta)}\right) \in \Sigma_{N},$$
and $L(\theta) = \ln(Z(\theta)) - q^{T} M\theta$. We assume that $\ker \widetilde{M} = \{0\}$ and that the function $L$ has a global minimum attained at $\theta_{*}$. We denote
$$\Sigma_{N}(\bar{g}, g) = \left\{p \in \Sigma_{N} \mid \sum_{i=1}^{N} p_{i} g_{k}(i) = \bar{g}_{k}, 1 \leqslant k \leqslant d\right\}.$$
(a) Show that $H_{N}(p(\theta_{*})) \geqslant H_{N}(q)$ and then that $H_{N}(p(\theta_{*}))$ is the maximum value of $H_{N}$ on $\Sigma_{N}(\bar{g}, g)$.
(b) Show that $p(\theta_{*})$ is the unique point of $\Sigma_{N}(\bar{g}, g)$ at which $H_{N}$ attains its maximum.