多元第九章习题

1.设$X=(X_1,X_2{)}^{\prime }$的协方差矩阵,$\mathbf{\Sigma }=(\begin{array}{cc}1& 4\\ 4& 100\end{array})$，试从协方差矩阵$\Sigma$和相关系数矩阵$R$出发求总体主成分，并加以比较

A<-matrix(c(1,4,4,100),nrow = 2)
 
ev=eigen(A)
 
val=round(ev$val,2)
vec=round(ev$val,3)
 
round(val/sum(val),4)
round(cumsum(val)/sum(val),4)

得到特征值和特征向量分别为

$$\begin{array}{c}\begin{array}{rlr}& {\lambda }_1=100.161,& a_1^{{}^{\prime }}=(0.040,-0.999)\\ & {\lambda }_2=0.839,& a_2^{{}^{\prime }}=(0.9991,0.0403)\end{array}\end{array}$$

第一主成分的贡献率为0.04030552

对于相关系数矩阵，只是多了一步标准化的过程：

$$\begin{array}{c}\begin{array}{rlr}& {\lambda }_1=1.4,& a_1^{{}^{\prime }}=(0.707,0.707)\\ & {\lambda }_2=0.6,& a_2^{{}^{\prime }}=(-0.707,0.707)\end{array}\end{array}$$

第一主成分的贡献率为0.7

2.设$\mathbf{X}=(X_1,X_2{)}^{\prime }\sim N_2(0,\mathbf{\Sigma })$ ，协方差矩阵$\mathbf{\Sigma }=\left(\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right)$，其中$\rho$ 为$X_1$和$X_2$的相关系数($\rho >0$) （1）试从$\Sigma$出发求的两个总体主成分; （2）求$X$的等概密度椭圆的主轴方向； （3）试问当$\rho$取多大时才能使第一主成分的贡献率达95%以上

(1)解：由习题一习题可知

$$\begin{array}{c}\begin{array}{r}{Z}_1=\frac{1}{\sqrt{2}}{X}_1+\frac{1}{\sqrt{2}}{X}_2\\ Z_2=\frac{1}{\sqrt{2}}{X}_1-\frac{1}{\sqrt{2}}{X}_2\end{array}\end{array}$$

(2)以$a_i$为坐标轴，则可以得到对应的椭圆方程 $\frac{z_1^2}{1+p}+\frac{z_2^2}{1-p}=c^2$ 则可知主轴方向就是以$a_i$为坐标轴的方向

(3)

$$\begin{array}{c}\begin{array}{r}\frac{1+p}{1+p+1-p}\ge 0.95\\ p\ge 0.9\end{array}\end{array}$$

3.设$p$元总体$X$的协方差矩阵为

$\mathbf{\Sigma }={\sigma }^2\left(\begin{array}{cccc}1& \rho & \cdots & \rho \\ \rho & 1& \cdots & \rho \\ ⋮& ⋮& & ⋮\\ \rho & \rho & \cdots & 1\end{array}\right)(0<\rho ⩽1).$

(1)试证明总体的第一主成分$Z_1=\frac{1}{\sqrt{p}}(X_1+X_2+\cdots +X_p);$ (2)试求第一主成分的贡献率

(1)由第一章习题可知

$$Q=\left(\begin{array}{cccc}\frac{1}{\sqrt{p}}& \frac{1}{\sqrt{2}}& \cdots & \frac{1}{\sqrt{p(p-1)}}\\ ⋮& -\frac{1}{\sqrt{2}}& \cdots & ⋮\\ ⋮& 0& \cdots & ⋮\\ ⋮& ⋮& \cdots & ⋮\\ \frac{1}{\sqrt{p}}& 0& \cdots & \frac{-(p-1)}{\sqrt{(p-1)p}}\end{array}\right)$$ $$\Lambda =\left(\begin{array}{cccc}1+(p-1)\rho & 0& \cdots & 0\\ 0& 1-\rho & \cdots & 0\\ ⋮& ⋮& & 0\\ 0& 0& \cdots & 1-\rho \end{array}\right)$$ $$\left(\begin{array}{cccc}1& \rho & \cdots & \rho \\ \rho & 1& \cdots & \rho \\ ⋮& ⋮& & ⋮\\ \rho & \rho & \cdots & 1\end{array}\right)=Q\Lambda Q^{{}^{\prime }}$$

由主成分定义可知$Z_1=\frac{1}{\sqrt{p}}(X_1+X_2+\cdots +X_p);$

(2)第一主成分贡献率： 由贡献率定义可得 $\frac{1+(p-1)\rho }{p}$

4.设总体$\mathbf{X}=(X_1,\cdots ,X_p{)}^{\prime }\sim N_p(\mathbf{\mu },\mathbf{\Sigma })(\mathbf{\Sigma }>0)$,等概率密度椭球为$(X-\mu {)}^{\prime }{\Sigma }^{-1}(X-\mu )=C^2$ $(C$为常数).试问椭球的主轴方向是什么？

对于协方差矩阵$\Sigma >0$，存在正交矩阵$Q$ ${\mathbf{Q}}^{\prime }\mathbf{\Sigma Q}=\left(\begin{array}{cccc}{\lambda }_1& & & \\ & {\lambda }_2& & \\ & & \ddots & \\ & & & {\lambda }_p\end{array}\right)$ 则其中$Q中第i列对应第i个主轴方向$

5.设三元总体$X$的协方差矩阵为$\Sigma =\left(\begin{array}{ccc}4& 0& 0\\ 0& 4& 0\\ 0& 0& 2\end{array}\right)$,试求总体主成分.

只需取$Z_1=X_1,Z_2=X_2,Z_3=X_3$即可，更一般的证明见题9

6.设三元总体$X$的协方差矩阵为$\Sigma =\left(\begin{array}{ccc}{\sigma }^2& \rho {\sigma }^2& 0\\ \rho {\sigma }^2& {\sigma }^2& \rho {\sigma }^2\\ 0& \rho {\sigma }^2& {\sigma }^2\end{array}\right)$,试求总体主成分，并计算每个主成分解释的方差比例(|$\rho |⩽1/\sqrt{2})$

求得矩阵特征多项式为

$$(\lambda -1)[(\lambda -1{)}^2-2{\rho }^2]$$

特征值特征向量：

$$\begin{array}{c}\begin{array}{rlr}& {\lambda }_1=1+\sqrt{2}\rho ,& a_1^{{}^{\prime }}=(\frac{1}{2},\frac{1}{\sqrt{2}},\frac{1}{2})\\ & {\lambda }_2=1,& a_2^{{}^{\prime }}=(\frac{1}{\sqrt{2}},0,-\frac{1}{\sqrt{2}})\\ & {\lambda }_3=1-\sqrt{2}\rho ，& a_3^{{}^{\prime }}=(\frac{1}{2},-\frac{1}{\sqrt{2}},\frac{1}{2})\end{array}\end{array}$$

则可得到三个主成分

$$\begin{array}{c}\begin{array}{rl}& Z_1=\frac{1}{2}{X}_1+\frac{1}{\sqrt{2}}{X}_2+\frac{1}{2}{X}_3\\ & Z_2=\frac{1}{\sqrt{2}}{X}_1-\frac{1}{\sqrt{2}}{X}_3\\ & Z_3=\frac{1}{2}{X}_1-\frac{1}{\sqrt{2}}{X}_2+\frac{1}{2}{X}_3\end{array}\end{array}$$

再由公式: $\mathrm{Var}(Z_i)={\mathbf{a}}_i^{\prime }\mathbf{\Sigma }{\mathbf{a}}_i={\lambda }_i,i=1,2,3,$可知方差所占比例就是贡献率 则可知贡献率分别为${\lambda }_1/3,{\lambda }_2/3,{\lambda }_3/3$

7.设 4 维随机向量$X$的协方差矩阵是

$\mathbf{\Sigma }=(\begin{array}{cccc}{\sigma }^2& {\sigma }_{12}& {\sigma }_{13}& {\sigma }_{14}\\ {\sigma }_{12}& {\sigma }^2& {\sigma }_{14}& {\sigma }_{13}\\ {\sigma }_{13}& {\sigma }_{14}& {\sigma }^2& {\sigma }_{12}\\ {\sigma }_{14}& {\sigma }_{13}& {\sigma }_{12}& {\sigma }^2\end{array}),$

其中${\sigma }_{1}2⩾{\sigma }_{13}⩾{\sigma }_{14}⩾0$, ${\sigma }^2+{\sigma }_{14}⩾{\sigma }_{12}+{\sigma }_{13}.$试求$X$的主成分.

与前面同理，计算略

8.已知总体$\mathbf{X}=(X_1,\cdots ,X_p{)}^{\prime }$的$n$次观测数据阵为$\mathbf{X}=(x_{ij}{)}_{n\times p}.$设$Z_i=a_i^{\prime }X$是$\mathbf{X}$的前$m$个样本主成分，其中$j=1,\cdots ,m$,且$m<p.$设变量$X_j$与$Z_1,\cdots ,Z_m$的回归模型为

$X_j=b_{j1}{Z}_1+\cdots +b_{jm}{Z}_m+{\epsilon }_j\stackrel{\mathrm{def}}{=}{b}_j^{\prime }Z+{\epsilon }_j,j=1,\cdots ,p.$

(1)试求参数$b_j$的最小二乘估计${\hat{\mathbf{b}}}_j(j=1,\cdots ,p);$ (2)求$X_j$回归方程的回归平方和$U_j$、残差平方和$Q_j$,以及判定系数$R_j^2(j=1,\cdots ,p).$ （1）${\hat{\mathbf{b}}}_j=(Z^{{}^{\prime }}Z{)}^{-1}Z{}^{\prime }{X}_j$ （2）$U_j={\sum }_{i=1}^n({\hat{x}}_{ji}-{\bar{x}}_j{)}^2$ 其中${\hat{x}}_{ji}=Z_{1h}{\hat{\mathbf{b}}}_j(Z_{1h}代表Z矩阵的第一行)，{\bar{x}}_j=\frac{1}{n}{1}_n^{{}^{\prime }}{X}_j$

$$Q_j=\sum _{i=1}^n({\hat{x}}_{ji}-{\overline{x}}_j{)}^2$$ $$R^2=\frac{SSR}{SST}=\frac{U_j}{U_j+Q_j}$$

9.设$\mathbf{X}=(X_1,\cdots ,X_p{)}^{\prime }\sim N_p(\mathbf{\mu },\mathbf{\Sigma }),\mathbf{\Sigma }$有一个$p$重特征值${\lambda }_1$,即$\mathbf{\Sigma }={\lambda }_1{\mathbf{I}}_p.$ 给定观测值$x_i=(x_{i1},\cdots ,x_{ip}{)}^{\prime }(i=1,\cdots ,n)$, (1)试证明：${\lambda }_1$的极大似然估计是

${\hat{\lambda }}_1=\frac{1}{pn}{\sum }_{k=1}^p{\sum }_{i=1}^n(x_{ik}-{\overline{x}}_k{)}^2,\text{其中}{\overline{x}}_k=\frac{1}{n}{\sum }_{i=1}^n{x}_{ik};$

(2)试证明：$\mathbf{X}{\text{ 的主成分由 B}}^{\prime }\mathbf{X}$给出，其中$\mathbf{B}$是任何$p$阶正交矩阵.

(1)有极大似然估计的似然函数给出： $L(\overline{\mathbf{x}},\mathbf{\Sigma })=\frac{1}{(2\pi {)}^{np/2}|\mathbf{\Sigma }{|}^{n/2}}\exp \left[-\frac{1}{2}\mathrm{tr}({\mathbf{\Sigma }}^{-1}\mathbf{V})\right].$

将$\mathbf{\Sigma }={\lambda }_1{\mathbf{I}}_p$带入似然函数可得： $L(\overline{\mathbf{x}},\mathbf{\Sigma })=\frac{1}{(2\pi {)}^{np/2}|{\mathbf{\lambda }}_1{|}^{np/2}}\exp \left[-\frac{1}{2{\lambda }_1}\mathrm{tr}(\mathbf{V})\right].$ 对似然函数求导并让导数等于零可得：

$$exp\{-\frac{tr(V)}{2{\lambda }_1}\}(\frac{1}{{\lambda }_1}{)}^{\frac{np}{2}-1}(\frac{np}{2}-\frac{tr(V)}{2{\lambda }_1})=0$$

则可知

$${\lambda }_1=\frac{1}{np}tr(V)$$

$\text{V}={\sum }^n(x_i-\overline{x})(x_i-\overline{x}{)}^{\prime }$ $\text{tr(V)}={\sum }^n(x_i-\overline{x}{)}^{\prime }(x_i-\overline{x})={\sum }_{k=1}^p{\sum }_{i=1}^n(x_{ik}-{\overline{x}}_k{)}^2$ 故原式得证 (2)利用Lagrange求极值： $\begin{array}{rl}L({\mathbf{a}}_1)& =\mathrm{Var}(Z_1)-\lambda ({\mathbf{a}}_1^{\prime }{\mathbf{a}}_1-1)\\ & ={\mathbf{a}}_1^{\prime }\mathbf{\Sigma }{\mathbf{a}}_1-\lambda ({\mathbf{a}}_1^{\prime }{\mathbf{I}}_p{\mathbf{a}}_1-1).\end{array}$ $\{\begin{array}{l}\frac{\partial L({\mathbf{a}}_1)}{\partial {\mathbf{a}}_1}=2(\mathbf{\Sigma }-\lambda {\mathbf{I}}_p){\mathbf{a}}_1=0,\\ \frac{\partial L({\mathbf{a}}_1)}{\partial \lambda }={\mathbf{a}}_1^{\prime }{\mathbf{a}}_1-1=0.\end{array}$ 由偏导数可知，主需要取$\lambda ={\lambda }_1$即可满足约束1，则对于矩阵$a$满足正交情况都能够满足方程

10.若随机变量$X=(X_1,\cdots ,X_p{)}^{\prime }$的协方差矩阵是非负定矩阵$\Sigma$,随机变量$Y=$ $(Y_1,\cdots ,Y_p{)}^{\prime }$的协方差矩阵是$\Sigma +{\sigma }^2{\mathbf{I}}_p$,则${\mathbf{L}}^{\prime }X$是$X$的主成分的充要条件是${\mathbf{L}}^{\prime }Y$是$Y$的 主成分，其中 L 是正交矩阵.

必要性：若${\mathbf{L}}^{\prime }X$是$X$的主成分，则可知

$$\{\begin{array}{l}\frac{\partial L({\mathbf{L}}_1)}{\partial {\mathbf{L}}_1}=2(\mathbf{\Sigma }-\lambda {\mathbf{I}}_p){\mathbf{L}}_1=0,\\ \frac{\partial L({\mathbf{a}}_1)}{\partial \lambda }={\mathbf{L}}_1^{\prime }{\mathbf{L}}_1-1=0.\end{array}$$

对于Y而言有方程组

$$\{\begin{array}{l}\frac{\partial L({\mathbf{L}}_1)}{\partial {\mathbf{L}}_1}=2(\mathbf{\Sigma }+{\sigma }^2{I}_p-\lambda {\mathbf{I}}_p){\mathbf{L}}_1=0,\\ \frac{\partial L({\mathbf{a}}_1)}{\partial \lambda }={\mathbf{L}}_1^{\prime }{\mathbf{L}}_1-1=0.\end{array}$$

此时只需要取$\lambda =\lambda +{\sigma }^2$即可，不会影响$L$的结构

充分性同理可得。