NOTES¶
ICA — Independent Component Analysis
Blind source separation from mixed observations. Uses the FastICA algorithm.
Algorithm¶
FastICA Fixed-Point Iteration¶
Preprocessing (mandatory):
- Centering: \(\mathbf{x}_c = \mathbf{x} - E[\mathbf{x}]\)
- Whitening: Eigendecompose covariance → \(\tilde{\mathbf{x}} = E D^{-½} E^T \mathbf{x}_c\)
- Decorrelates and normalizes variance
- Reduces the problem from general to orthogonal matrices
Core iteration: For each source \(i = 1, \dots, m\):
\[ \mathbf{w}i^+ = E[\tilde{\mathbf{x}} \cdot g(\mathbf{w}i^T \tilde{\mathbf{x}})] - E[g'(\mathbf{w}i^T \tilde{\mathbf{x}})] \cdot \mathbf{w}i \]
Normalize, then Gram-Schmidt orthogonalize. Convergence: \(1 - |\mathbf{w}i^{(k)T} \cdot \mathbf{w}i^{(k-1)}| < \text{tolerance}\)
Nonlinearity \(g(u)\) Selection:
| Function | Formula | Source type |
|---|---|---|
tanh (default) | \(\tanh(\alpha u)\) | Super-Gaussian (speech, seismic) |
cube | \(u^3\) | Sub-Gaussian |
gauss | \(u \cdot e{-u2/2}\) | Symmetric |
skew | \(u^2\) | Skewed |
ICA vs PCA
| ICA | PCA | |
|---|---|---|
| Goal | Statistical independence | Decorrelation |
| Statistics | Higher-order | 2nd-order only |
| Output order | Ambiguous | Sorted by variance |
Inherent ambiguities
ICA cannot determine the scale or order of separated sources. This is a property of the model, not a bug.
Requirements
Sources ≤ sensors; sources must be non-Gaussian (at most one Gaussian); \(> 100\) samples recommended.
API Reference (C++ class)¶
tiny::ICA ica;
ica.set_max_iterations(1000);
ica.set_tolerance(1e-6f);
ica.set_nonlinearity(tiny::ICANonlinearity::TANH);
// One-shot decomposition
ICADecomposition result = ica.decompose(mixed_signals, num_sources);
// result.sources — separated source signals
// result.mixing_matrix — mixing matrix A
// result.unmixing_matrix — unmixing matrix W
// Reconstruction from sources
Mat reconstructed = ICADecomposition::reconstruct(sources, mixing_matrix);
// Apply trained model to new data
Mat separated = ICADecomposition::apply(decomposition, new_mixed_signals);