Let $n$ be a strictly positive integer. We consider a family $(X_{k})_{1 \leqslant k \leqslant n}$ of $n$ random variables taking values in $\{1, \ldots, N\}$, pairwise independent and identically distributed, defined on a probability space $(\Omega, \mathscr{A}, \mathbf{P})$. We further assume that $\mathbf{P}(X_{1} = i) = p_{i}$ and that $p_{i} > 0$ for all $i \in \{1, \ldots, N\}$. Show that for all $\epsilon > 0$, we have $\mathbf{P}\left(\left|\frac{1}{n} \ln\left(\prod_{k=1}^{n} p_{X_{k}}\right) + H_{N}(p)\right| \geqslant \epsilon\right)$ tends to 0 as $n$ tends to infinity. Where $H_{N}(p) = \sum_{i=1}^{N} \varphi(p_{i})$ and $\varphi(t) = \begin{cases} 0 & \text{if } t = 0 \\ -t \ln(t) & \text{otherwise.} \end{cases}$
Let $n$ be a strictly positive integer. We consider a family $(X_{k})_{1 \leqslant k \leqslant n}$ of $n$ random variables taking values in $\{1, \ldots, N\}$, pairwise independent and identically distributed, defined on a probability space $(\Omega, \mathscr{A}, \mathbf{P})$. We further assume that $\mathbf{P}(X_{1} = i) = p_{i}$ and that $p_{i} > 0$ for all $i \in \{1, \ldots, N\}$. Show that for all $\epsilon > 0$, we have $\mathbf{P}\left(\left|\frac{1}{n} \ln\left(\prod_{k=1}^{n} p_{X_{k}}\right) + H_{N}(p)\right| \geqslant \epsilon\right)$ tends to 0 as $n$ tends to infinity.
Where $H_{N}(p) = \sum_{i=1}^{N} \varphi(p_{i})$ and $\varphi(t) = \begin{cases} 0 & \text{if } t = 0 \\ -t \ln(t) & \text{otherwise.} \end{cases}$