Liangliang's Blog

最小二乘目标函数为凸函数

证明:

\[ \left\|\mathbf{M} \left(\frac{\bm{s}_1 + \bm{s}_2}{2}\right) - \bm{c}\right\|_2^2 \leq \frac{\|\mathbf{M} \bm{s}_1 - \bm{c}\|_2^2 + \|\mathbf{M} \bm{s}_2 - \bm{c}\|_2^2}{2}. \]

注意到

\[ \begin{split} \left\|\mathbf{M} \left(\frac{\bm{s}_1 + \bm{s}_2}{2}\right) - \bm{c}\right\|_2^2 &= \left(\mathbf{M} \left(\frac{\bm{s}_1 + \bm{s}_2}{2}\right) - \bm{c}\right)^{\mathrm{T}} \left(\mathbf{M} \left(\frac{\bm{s}_1 + \bm{s}_2}{2}\right) - \bm{c}\right) \\ &= \left(\frac{\mathbf{M} \bm{s}_1 + \mathbf{M} \bm{s}_2}{2} - \bm{c}\right)^{\mathrm{T}} \left(\frac{\mathbf{M} \bm{s}_1 + \mathbf{M} \bm{s}_2}{2} - \bm{c}\right) \\ &= \frac{1}{4} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_1 + \frac{1}{4} \boldsymbol{s}_2 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 + \frac{1}{2} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 - (\boldsymbol{s}_1 + \boldsymbol{s}_2)^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} + \bm{c}^{\mathrm{T}} \bm{c} \\ \end{split} \]

\[ \begin{split} \|\mathbf{M} \bm{s}_1 - \bm{c}\|_2^2 + \|\mathbf{M} \bm{s}_2 - \bm{c}\|_2^2 &= (\mathbf{M} \bm{s}_1 - \bm{c})^{\mathrm{T}} (\mathbf{M} \bm{s}_1 - \bm{c}) + (\mathbf{M} \bm{s}_2 - \bm{c})^{\mathrm{T}} (\mathbf{M} \bm{s}_2 - \bm{c}) \\ &= \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_1 + \boldsymbol{s}_2 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 - 2 \boldsymbol{s}_1^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} - 2 \boldsymbol{s}_2^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} + 2 \bm{c}^{\mathrm{T}} \bm{c}. \end{split} \]

因此只需证明

\[ \begin{split} & \frac{1}{4} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_1 + \frac{1}{4} \boldsymbol{s}_2 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 + \frac{1}{2} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 - (\boldsymbol{s}_1 + \boldsymbol{s}_2)^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} + \bm{c}^{\mathrm{T}} \bm{c} \\ \leq & \frac{1}{2}\boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_1 + \frac{1}{2} \boldsymbol{s}_2 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 - \boldsymbol{s}_1^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} - \boldsymbol{s}_2^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \bm{c} + \bm{c}^{\mathrm{T}} \bm{c}. \end{split} \]

消除公共项,只需证明

\[ \begin{split} \frac{1}{4} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_1 + \frac{1}{4} \boldsymbol{s}_2 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 - \frac{1}{2} \boldsymbol{s}_1 \mathbf{M}^{\mathrm{T}} \mathbf{M} \boldsymbol{s}_2 \geq 0. \end{split} \]

上面的不等式可以写成

\[ (\boldsymbol{s}_1 - \boldsymbol{s}_2)^{\mathrm{T}} \mathbf{M}^{\mathrm{T}} \mathbf{M} (\boldsymbol{s}_1 - \boldsymbol{s}_2) \geq 0. \]

也即

\[ \left( \mathbf{M} (\boldsymbol{s}_1 -\boldsymbol{s}_2)\right)^{\mathrm{T}} \left( \mathbf{M} (\boldsymbol{s}_1 -\boldsymbol{s}_2)\right) \geq 0. \]

得证。