PCA：把資料壓縮時保留了什麼？

樣本共變異數 C = Xcᵀ Xc / (n−1)
方向u上的樣本變異 = uᵀ C u，限制 ||u||=1

Xc = U Σ Vᵀ
取前k個方向Vk：Z = Xc Vk
重建：X̂ = Z Vkᵀ + μ

C = [[10/3, 2],
     [2, 10/3]]

第一方向 u1 = (1,1)/√2，變異 λ1=16/3
第二方向 u2 = (1,−1)/√2，變異 λ2=4/3
第一PC解釋比例 = (16/3)/(20/3) = 0.8

python -m pip install numpy==1.26.3 scikit-learn==1.5.1
python pca_projection.py

import numpy as np
import sklearn
from sklearn.decomposition import PCA
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from threadpoolctl import threadpool_limits

X = np.array([[12.0, 22.0], [8.0, 18.0],
              [11.0, 19.0], [9.0, 21.0]])
centered = X - X.mean(axis=0)
covariance = centered.T @ centered / (len(X) - 1)
u = np.array([1.0, 1.0]) / np.sqrt(2.0)
scores = centered @ u
reconstructed = scores[:, None] * u + X.mean(axis=0)
with threadpool_limits(limits=1):
    full = PCA(n_components=2, svd_solver="full").fit(X)
    one = PCA(n_components=1, svd_solver="full").fit(X)
    one_reconstructed = one.inverse_transform(one.transform(X))
    print(f"numpy={np.__version__} sklearn={sklearn.__version__}")
    print(f"toy mean={X.mean(axis=0).tolist()}")
    print("toy covariance=" + np.array2string(covariance, precision=6))
    print("toy explained_variance=" +
          ",".join(f"{v:.6f}" for v in full.explained_variance_))
    print("toy explained_ratio=" +
          ",".join(f"{v:.6f}" for v in full.explained_variance_ratio_))
    print("toy scores_on_positive_diagonal=" +
          ",".join(f"{v:.6f}" for v in scores))
    print(f"toy reconstructed={reconstructed.round(6).tolist()}")
    print(f"toy rank1_SSE={np.square(X - one_reconstructed).sum():.6f}")
    print(f"toy entrywise_MSE={np.square(X - one_reconstructed).mean():.6f}")
    print(f"toy direct_and_sklearn_reconstruction_match="
          f"{np.allclose(reconstructed, one_reconstructed)}")

    # All pixel features share the same original intensity units.
    # Labels are used only for a stratified split, never for fitting PCA.
    digits, labels = load_digits(return_X_y=True)
    train, test = train_test_split(
        np.arange(len(digits)), test_size=0.25,
        stratify=labels, random_state=42,
    )
    model = PCA(n_components=0.95, svd_solver="full").fit(digits[train])
    train_back = model.inverse_transform(model.transform(digits[train]))
    test_back = model.inverse_transform(model.transform(digits[test]))
    print(f"digits train={len(train)} test={len(test)} original_features=64 "
          f"retained_components={model.n_components_}")
    print(f"digits train_explained_ratio_sum="
          f"{model.explained_variance_ratio_.sum():.6f}")
    print(f"digits train_pixel_MSE={np.square(digits[train] - train_back).mean():.6f}")
    print(f"digits test_pixel_MSE={np.square(digits[test] - test_back).mean():.6f}")
    print(f"digits mean_fit_on_train_only="
          f"{np.allclose(model.mean_, digits[train].mean(axis=0))}")

BSD 3-Clause License

Copyright (c) 2007-2026 The scikit-learn developers.
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its
  contributors may be used to endorse or promote products derived from
  this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

New BSD License

Copyright (c) 2007–2019 The scikit-learn developers.
All rights reserved.


Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

  a. Redistributions of source code must retain the above copyright notice,
     this list of conditions and the following disclaimer.
  b. Redistributions in binary form must reproduce the above copyright
     notice, this list of conditions and the following disclaimer in the
     documentation and/or other materials provided with the distribution.
  c. Neither the name of the Scikit-learn Developers  nor the names of
     its contributors may be used to endorse or promote products
     derived from this software without specific prior written
     permission.


THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE.