Show that for all $f \in \mathscr { C } _ { b } ^ { 2 }$, $f$ admits an entropy relative to $\mu$ and that $$\operatorname { Ent } _ { \mu } ( f ) \leqslant \int \left| f ^ { \prime } ( x ) \right| ^ { 2 } \mu ( x ) d x$$ You may consider the family of functions defined by $f _ { \delta } = \delta + f ^ { 2 }$ for $\delta > 0$.

Let $m$ be a measure. We assume that there exists a constant $C > 0$ such that, if $f : \mathbb { R } \rightarrow \mathbb { R }$ is of class $\mathscr { C } ^ { 1 }$ and of bounded derivative $f ^ { \prime }$, then $f$ admits an entropy relative to $m$ and $$\operatorname { Ent } _ { m } ( f ) \leqslant C \int \left| f ^ { \prime } ( x ) \right| ^ { 2 } m ( x ) d x \tag{1}$$ Show that $\int \left( 1 + | x | + x ^ { 2 } \right) m ( x ) d x < + \infty$.

Let $m$ be a measure. We assume that there exists a constant $C > 0$ such that, if $f : \mathbb { R } \rightarrow \mathbb { R }$ is of class $\mathscr { C } ^ { 1 }$ and of bounded derivative $f ^ { \prime }$, then $f$ admits an entropy relative to $m$ and $$\operatorname { Ent } _ { m } ( f ) \leqslant C \int \left| f ^ { \prime } ( x ) \right| ^ { 2 } m ( x ) d x \tag{1}$$ Let $f \in \mathscr { C } _ { b } ^ { 1 }$. We wish to show that $f$ admits a variance relative to $m$ and that $$\operatorname { Var } _ { m } ( f ) \leqslant \frac { C } { 2 } \int \left| f ^ { \prime } ( x ) \right| ^ { 2 } m ( x ) d x \tag{2}$$
10a. Show that $fm$ and $f ^ { 2 } m$ are integrable, and that it suffices to show (2) in the case where we additionally have $\int f ( x ) m ( x ) d x = 0$ and $\int f ( x ) ^ { 2 } m ( x ) d x = 1$.
10b. Under the hypotheses of the previous question, show (2). You may apply (1) to the family of functions $f _ { \varepsilon } = 1 + \varepsilon f$ for $\varepsilon > 0$.

Let $m$ be a measure. We assume that there exists a constant $C > 0$ such that, if $f : \mathbb { R } \rightarrow \mathbb { R }$ is of class $\mathscr { C } ^ { 1 }$ and of bounded derivative $f ^ { \prime }$, then $f$ admits an entropy relative to $m$ and $$\operatorname { Ent } _ { m } ( f ) \leqslant C \int \left| f ^ { \prime } ( x ) \right| ^ { 2 } m ( x ) d x \tag{1}$$ Let $f$ be a function in $\mathscr { C } _ { b } ^ { 1 }$, such that for all $x \in \mathbb { R }$, we have $\left| f ^ { \prime } ( x ) \right| \leqslant 1$. We denote, for $\lambda \in \mathbb { R }$, $$H ( \lambda ) = \int e ^ { \lambda f ( x ) } m ( x ) d x$$ We admit that $H$ is of class $\mathscr { C } ^ { 1 }$ and that we obtain an expression of $H ^ { \prime } ( \lambda )$ by differentiating under the integral sign in the usual manner.
11a. Show that for all $\lambda \in \mathbb { R }$, $$\lambda H ^ { \prime } ( \lambda ) - H ( \lambda ) \ln H ( \lambda ) \leqslant \frac { C \lambda ^ { 2 } } { 4 } H ( \lambda )$$
11b. Deduce that for $\lambda \geqslant 0$, $$\int e ^ { \lambda f ( x ) } m ( x ) d x \leqslant \exp \left( \lambda \int f ( x ) m ( x ) d x + \frac { C \lambda ^ { 2 } } { 4 } \right) \tag{3}$$ You may study the function $\lambda \mapsto \frac { 1 } { \lambda } \ln H ( \lambda )$.

Let $X$ be a finite set and $p = (p_x)_{x \in X}$ a probability distribution on $X$. We assume that $p$ charges all points of $X$: $p_x > 0$ for all $x \in X$. We call entropy of $p$ the quantity $$H(p) = -\sum_{x \in X} p_x \ln(p_x)$$ We consider the set $Q_X = \{\boldsymbol{q} = (q_x)_{x \in X} \in \mathbb{R}^X \mid \forall x \in X, q_x \geq 0\}$. For all $\boldsymbol{q}, \boldsymbol{q}' \in Q_X$ such that $q_x' > 0$ for all $x \in X$, we define: $$\mathrm{KL}(\boldsymbol{q}, \boldsymbol{q}') = \sum_{x \in X} \varphi(q_x / q_x') q_x'$$ with $\varphi : \mathbb{R}_+ \rightarrow \mathbb{R}$ defined by $\varphi(x) = x \log(x) - x + 1$ for $x > 0$ and extended to 0 by continuity.
(a) Specify $\varphi(0)$.
(b) Verify that $\varphi$ is continuous, strictly convex, positive and that $\varphi(x) = 0$ if and only if $x = 1$.
(c) Show that $Q_X$ is convex and that $\boldsymbol{q} \mapsto \mathrm{KL}(\boldsymbol{q}, \boldsymbol{q}')$ is strictly convex, positive and vanishes if and only if $q = q'$.

We consider $\alpha = (\alpha_i)_{i \in I} \in (\mathbb{R}_+^*)^I$ and $\beta = (\beta_j)_{j \in J} \in (\mathbb{R}_+^*)^J$ such that $\sum_{i \in I} \alpha_i = \sum_{j \in J} \beta_j = 1$. We denote by $\boldsymbol{p}$ the element of $F(\alpha, \beta)$ defined by $p_{ij} = \alpha_i \beta_j > 0$ for all $(i,j) \in I \times J$. Let $C = (C_{ij})_{(i,j) \in I \times J} \in \mathbb{R}_+^{I \times J}$ and $\epsilon > 0$. We consider $J_\epsilon : Q \rightarrow \mathbb{R}$ defined by $$J_\epsilon(\boldsymbol{q}) = \sum_{ij} q_{ij} C_{ij} + \epsilon \operatorname{KL}(\boldsymbol{q}, \boldsymbol{p})$$ and $\boldsymbol{q}(\epsilon)$ the unique minimizer of $J_\epsilon$ on $F(\alpha, \beta)$.
(a) Verify that $q(\epsilon)_{ij} > 0$ for all $(i,j) \in I \times J$ (Hint: One may reason by contradiction and consider for all $t \in ]0,1[$ $\boldsymbol{q}(\epsilon, t) = (1-t)\boldsymbol{q}(\epsilon) + t\boldsymbol{p}$ then observe the behavior of $\varphi(x)$ near $x = 0$).
(b) Show that this is no longer true if we assume that $\epsilon = 0$.

We define $Q_{>0} = (\mathbb{R}_+^*)^{I \times J}$ and $\mathscr{L} : Q_{>0} \times (\mathbb{R}^I \times \mathbb{R}^J) \rightarrow \mathbb{R}$ defined by $$\mathscr{L}(\boldsymbol{q}, (f, g)) = J_\epsilon(\boldsymbol{q}) + \sum_{i \in I} f_i \left(\alpha_i - \sum_{j \in J} q_{ij}\right) + \sum_{j \in J} g_j \left(\beta_j - \sum_{i \in I} q_{ij}\right).$$ (a) Verify that $Q_{>0}$ is an open convex set of $\mathbb{R}^{I \times J}$.
(b) Show that there exists $(f(\epsilon), g(\epsilon)) \in \mathbb{R}^I \times \mathbb{R}^J$ such that $\mathscr{L}(q(\epsilon), (f(\epsilon), g(\epsilon)))$ is a saddle point of $\mathscr{L}$. (Hint: One may identify $\mathbb{R}^{I \times J}$ with $\mathbb{R}^n$ and $\mathbb{R}^I \times \mathbb{R}^J$ with $\mathbb{R}^m$, for $n$ the cardinality of $I \times J$ and $m$ the sum of the cardinalities of $I$ and $J$, then use question 3 of part I.)

For $t \in \mathbf{R}_{+}$, we set $S(t) = \operatorname{Ent}_{\varphi}\left(P_{t}(f)\right)$. Justify that $S(t)$ is well defined.
Here $f$ is an element of $C^{2}(\mathbf{R})$ with strictly positive values such that the functions $f, f^{\prime}, f^{\prime\prime}$ and $\frac{f^{\prime 2}}{f}$ have slow growth, and $\int_{-\infty}^{+\infty} f(x) \varphi(x) \mathrm{d}x = 1$.

By admitting that the result of question 7 is valid for the functions $P_{t}(f)$ and $1 + \ln\left(P_{t}(f)\right)$, show that
$$\forall t \in \mathbf{R}_{+}^{*}, \quad -S^{\prime}(t) = \mathrm{e}^{-2t} \int_{-\infty}^{+\infty} \frac{P_{t}\left(f^{\prime}\right)(x)^{2}}{P_{t}(f)(x)} \varphi(x) \mathrm{d}x$$

Establish the following inequality
$$\operatorname{Ent}_{\varphi}(f) \leq \frac{1}{2} \int_{-\infty}^{+\infty} \frac{f^{\prime 2}(x)}{f(x)} \varphi(x) \mathrm{d}x$$

Justify that the quantity $\operatorname{Ent}_{\varphi}(g)$ is well defined for all $g \in C^0(\mathbf{R}) \cap CL(\mathbf{R})$ with strictly positive values such that $\int_{-\infty}^{+\infty} g(x)\varphi(x)\,\mathrm{d}x = 1$, where $$\operatorname{Ent}_{\varphi}(g) = \int_{-\infty}^{+\infty} \ln(g(x))\,g(x)\,\varphi(x)\,\mathrm{d}x.$$
Hint: You may use question 11.

For $t \in \mathbf{R}_+$, we set $S(t) = \operatorname{Ent}_{\varphi}(P_t(f))$. Justify that $S(t)$ is well defined.

By admitting that the result of question 7 is valid for the functions $P_t(f)$ and $1 + \ln(P_t(f))$, show that $$\forall t \in \mathbf{R}_+^*, \quad -S'(t) = \mathrm{e}^{-2t}\int_{-\infty}^{+\infty} \frac{P_t(f')(x)^2}{P_t(f)(x)}\,\varphi(x)\,\mathrm{d}x.$$

Establish the following inequality $$\operatorname{Ent}_{\varphi}(f) \leq \frac{1}{2}\int_{-\infty}^{+\infty} \frac{f'^2(x)}{f(x)}\,\varphi(x)\,\mathrm{d}x.$$