Consider to solve the following simultaneous linear equation:
$$A x = b$$
where $\boldsymbol { A } \in \mathcal { R } ^ { m \times n } , \boldsymbol { b } \in \mathcal { R } ^ { m }$ are a constant matrix and a vector, and $\boldsymbol { x } \in \mathcal { R } ^ { n }$ is an unknown vector. Answer the following questions.
(1) An $m \times ( n + 1 )$ matrix $\overline { \boldsymbol { A } } = ( \boldsymbol { A } \mid \boldsymbol { b } )$ is made by adding a column vector after the last column of matrix $\boldsymbol { A }$. In the case of $\boldsymbol { A } = \left( \begin{array} { c c c } 1 & 0 & - 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \end{array} \right)$ and $\boldsymbol { b } = \left( \begin{array} { c } 2 \\ 4 \\ 2 \end{array} \right)$, $\overline { \boldsymbol { A } } = \left( \begin{array} { c c c c } 1 & 0 & - 1 & 2 \\ 1 & 1 & 0 & 4 \\ 0 & 1 & 1 & 2 \end{array} \right)$ is obtained. Let the $i$-th column vector of the matrix $\overline { \boldsymbol { A } }$ be $\boldsymbol { a } _ { i } ( i = 1,2,3,4 )$.
(i) Find the maximum number of linearly independent vectors among $\boldsymbol { a } _ { 1 } , \boldsymbol { a } _ { 2 }$ and $\boldsymbol { a } _ { 3 }$.
(ii) Show that $a _ { 4 }$ can be represented as a linear sum of $a _ { 1 } , a _ { 2 }$ and $a _ { 3 }$, by obtaining scalars $x _ { 1 }$ and $x _ { 2 }$ that satisfy $\boldsymbol { a } _ { 4 } = x _ { 1 } \boldsymbol { a } _ { 1 } + x _ { 2 } \boldsymbol { a } _ { 2 } + \boldsymbol { a } _ { 3 }$.
(iii) Find the maximum number of linearly independent vectors among $\boldsymbol { a } _ { 1 } , \boldsymbol { a } _ { 2 } , \boldsymbol { a } _ { 3 }$ and $a _ { 4 }$.
(2) Show that the solution of the simultaneous linear equation exists when $\operatorname { rank } \overline { \boldsymbol { A } } = \operatorname { rank } \boldsymbol { A }$, for arbitrary $m , n , \boldsymbol { A }$ and $\boldsymbol { b }$.
(3) There is no solution when $\operatorname { rank } \overline { \boldsymbol { A } } > \operatorname { rank } \boldsymbol { A }$. When $m > n , \operatorname { rank } \boldsymbol { A } = n$ and $\operatorname { rank } \overline { \boldsymbol { A } } > \operatorname { rank } \boldsymbol { A }$, obtain $\boldsymbol { x }$ that minimizes the squared norm of the difference between the left hand side and the right hand side of the simultaneous linear equation, namely $\| \boldsymbol { b } - \boldsymbol { A } \boldsymbol { x } \| ^ { 2 }$.
(4) When $m < n$ and $\operatorname { rank } \boldsymbol { A } = m$, there exist multiple solutions for the simultaneous linear equation for arbitrary $\boldsymbol { b }$. Obtain $\boldsymbol { x }$ that minimizes $\| \boldsymbol { x } \| ^ { 2 }$ among them, by adopting the method of Lagrange multipliers and using the simultaneous linear equation as the constraint condition.
(5) Show that there exists a unique $\boldsymbol { P } \in \mathcal { R } ^ { n \times m }$ that satisfies the following four equations for arbitrary $m , n$ and $\boldsymbol { A }$.
$$\begin{array} { r } \boldsymbol { A P A } = \boldsymbol { A } \\ \boldsymbol { P A P } = \boldsymbol { P } \\ ( \boldsymbol { A P } ) ^ { T } = \boldsymbol { A P } \\ ( \boldsymbol { P A } ) ^ { T } = \boldsymbol { P A } \end{array}$$
(6) Show that both $\boldsymbol { x }$ obtained in (3) and $\boldsymbol { x }$ obtained in (4) are represented in the form of $x = P b$.