Consider to solve the following simultaneous linear equation:
$$A x = b$$
where $\boldsymbol { A } \in \mathcal { R } ^ { m \times n } , \boldsymbol { b } \in \mathcal { R } ^ { m }$ are a constant matrix and a vector, and $\boldsymbol { x } \in \mathcal { R } ^ { n }$ is an unknown vector. Answer the following questions.
(1) An $m \times ( n + 1 )$ matrix $\overline { \boldsymbol { A } } = ( \boldsymbol { A } \mid \boldsymbol { b } )$ is made by adding a column vector after the last column of matrix $\boldsymbol { A }$. In the case of $\boldsymbol { A } = \left( \begin{array} { c c c } 1 & 0 & - 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \end{array} \right)$ and $\boldsymbol { b } = \left( \begin{array} { c } 2 \\ 4 \\ 2 \end{array} \right)$, $\overline { \boldsymbol { A } } = \left( \begin{array} { c c c c } 1 & 0 & - 1 & 2 \\ 1 & 1 & 0 & 4 \\ 0 & 1 & 1 & 2 \end{array} \right)$ is obtained. Let the $i$-th column vector of the matrix $\overline { \boldsymbol { A } }$ be $\boldsymbol { a } _ { i } ( i = 1,2,3,4 )$.
(i) Find the maximum number of linearly independent vectors among $\boldsymbol { a } _ { 1 } , \boldsymbol { a } _ { 2 }$ and $\boldsymbol { a } _ { 3 }$.
(ii) Show that $a _ { 4 }$ can be represented as a linear sum of $a _ { 1 } , a _ { 2 }$ and $a _ { 3 }$, by obtaining scalars $x _ { 1 }$ and $x _ { 2 }$ that satisfy $\boldsymbol { a } _ { 4 } = x _ { 1 } \boldsymbol { a } _ { 1 } + x _ { 2 } \boldsymbol { a } _ { 2 } + \boldsymbol { a } _ { 3 }$.
(iii) Find the maximum number of linearly independent vectors among $\boldsymbol { a } _ { 1 } , \boldsymbol { a } _ { 2 } , \boldsymbol { a } _ { 3 }$ and $a _ { 4 }$.
(2) Show that the solution of the simultaneous linear equation exists when $\operatorname { rank } \overline { \boldsymbol { A } } = \operatorname { rank } \boldsymbol { A }$, for arbitrary $m , n , \boldsymbol { A }$ and $\boldsymbol { b }$.
(3) There is no solution when $\operatorname { rank } \overline { \boldsymbol { A } } > \operatorname { rank } \boldsymbol { A }$. When $m > n , \operatorname { rank } \boldsymbol { A } = n$ and $\operatorname { rank } \overline { \boldsymbol { A } } > \operatorname { rank } \boldsymbol { A }$, obtain $\boldsymbol { x }$ that minimizes the squared norm of the difference between the left hand side and the right hand side of the simultaneous linear equation, namely $\| \boldsymbol { b } - \boldsymbol { A } \boldsymbol { x } \| ^ { 2 }$.
(4) When $m < n$ and $\operatorname { rank } \boldsymbol { A } = m$, there exist multiple solutions for the simultaneous linear equation for arbitrary $\boldsymbol { b }$. Obtain $\boldsymbol { x }$ that minimizes $\| \boldsymbol { x } \| ^ { 2 }$ among them, by adopting the method of Lagrange multipliers and using the simultaneous linear equation as the constraint condition.
(5) Show that there exists a unique $\boldsymbol { P } \in \mathcal { R } ^ { n \times m }$ that satisfies the following four equations for arbitrary $m , n$ and $\boldsymbol { A }$.
$$\begin{array} { r } \boldsymbol { A P A } = \boldsymbol { A } \\ \boldsymbol { P A P } = \boldsymbol { P } \\ ( \boldsymbol { A P } ) ^ { T } = \boldsymbol { A P } \\ ( \boldsymbol { P A } ) ^ { T } = \boldsymbol { P A } \end{array}$$
(6) Show that both $\boldsymbol { x }$ obtained in (3) and $\boldsymbol { x }$ obtained in (4) are represented in the form of $x = P b$.

Let $f _ { 1 }$ be a positive constant function on $[ 0,1 ]$ with $f _ { 1 } ( x ) = c$, and let $p$ and $q$ be positive real numbers with $1 / p + 1 / q = 1$. Moreover, let $\left\{ f _ { n } \right\}$ be the sequence of functions on $[ 0,1 ]$ defined by
$$f _ { n + 1 } ( x ) = p \int _ { 0 } ^ { x } \left( f _ { n } ( t ) \right) ^ { 1 / q } \mathrm {~d} t \quad ( n = 1,2 , \ldots )$$
Answer the following questions.
(1) Let $\left\{ a _ { n } \right\}$ and $\left\{ c _ { n } \right\}$ be the sequences of real numbers defined by $a _ { 1 } = 0 , c _ { 1 } = c$ and
$$\begin{aligned} & a _ { n + 1 } = q ^ { - 1 } a _ { n } + 1 \quad ( n = 1,2 , \ldots ) \\ & c _ { n + 1 } = \frac { p \left( c _ { n } \right) ^ { 1 / q } } { a _ { n + 1 } } \quad ( n = 1,2 , \ldots ) \end{aligned}$$
Show that $f _ { n } ( x ) = c _ { n } x ^ { a _ { n } }$.
(2) Let $g _ { n }$ be the function on $[ 0,1 ]$ defined by $g _ { n } ( x ) = x ^ { a _ { n } } - x ^ { p }$ for $n \geq 2$. Noting that $a _ { n } \geq 1$ holds true for $n \geq 2$, show that $g _ { n }$ attains its maximum at a point $x = x _ { n }$, and find the value of $x _ { n }$.
(3) Show that $\lim _ { n \rightarrow \infty } g _ { n } ( x ) = 0$ for any $x \in [ 0,1 ]$.
(4) Let $d _ { n }$ be defined by $d _ { n } = \left( c _ { n } \right) ^ { q ^ { n } }$. Show that $d _ { n + 1 } / d _ { n }$ converges to a finite positive value as $n \rightarrow \infty$. You may use the fact that $\lim _ { t \rightarrow \infty } ( 1 - 1 / t ) ^ { t } = 1 / \mathrm { e }$.
(5) Find the value of $\lim _ { n \rightarrow \infty } c _ { n }$.
(6) Show that $\lim _ { n \rightarrow \infty } f _ { n } ( x ) = x ^ { p }$ for any $x \in [ 0,1 ]$.

Let $z _ { n }$ and $w _ { n } ( n = 0,1,2 , \ldots )$ be complex numbers. Consider a bag that contains two red cards and one white card. First, take one card from the bag and return it to the bag. $z _ { k + 1 } ( k = 0,1,2 , \ldots )$ is generated in the following manner based on the color of the card taken.
$$z _ { k + 1 } = \begin{cases} i z _ { k } & \text { if a red card was taken, } \\ - i z _ { k } & \text { if a white card was taken. } \end{cases}$$
Next, take one card from the bag again and return it to the bag. $w _ { k + 1 }$ is generated in the following manner based on the color of the card taken.
$$w _ { k + 1 } = \begin{cases} - i w _ { k } & \text { if a red card was taken, } \\ i w _ { k } & \text { if a white card was taken. } \end{cases}$$
Here, each card is independently taken with equal probability. The initial state is $z _ { 0 } = 1$ and $w _ { 0 } = 1$. Thus, $z _ { n } , w _ { n }$ are the values after repeating the series of the above two operations $n$ times starting from the state of $z _ { 0 } = 1$ and $w _ { 0 } = 1$. Here, $i$ is the imaginary unit.
Answer the following questions.
(1) Show that $\operatorname { Re } \left( z _ { n } \right) = 0$ if $n$ is odd, and that $\operatorname { Im } \left( z _ { n } \right) = 0$ if $n$ is even. Here, $\operatorname { Re } ( z )$ and $\operatorname { Im } ( z )$ represent the real part and the imaginary part of $z$ respectively.
(2) Let $P _ { n }$ be the probability of $z _ { n } = 1$, and $Q _ { n }$ be the probability of $z _ { n } = i$. Find recurrence equations for $P _ { n }$ and $Q _ { n }$.
(3) Find the probabilities of $z _ { n } = 1 , z _ { n } = i , z _ { n } = - 1$, and $z _ { n } = - i$ respectively.
(4) Show that the expected value of $z _ { n }$ is $( i / 3 ) ^ { n }$.
(5) Find the probability of $z _ { n } = w _ { n }$.
(6) Find the expected value of $z _ { n } + w _ { n }$.
(7) Find the expected value of $z _ { n } w _ { n }$.