Sage-Tips

Which is better PCA or ICA?

As PCA considers second order moments only it lacks information on higher order statistics. Independent Component Analysis (ICA) is a technique data analysis accounting for higher order statistics. ICA is a generalisation of PCA. Moreover, PCA can be used as preproces- sing step in some ICA algorithm.

When can or should PCA be used?

PCA should be used mainly for variables which are strongly correlated. If the relationship is weak between variables, PCA does not work well to reduce data. Refer to the correlation matrix to determine. In general, if most of the correlation coefficients are smaller than 0.3, PCA will not help.

Are ICA components orthogonal?

ICA finds directions in the feature space corresponding to projections with high non-Gaussianity. not necessarily orthogonal in the original feature space, but orthogonal in the whitened feature space.

Where can we apply PCA?

PCA technique is particularly useful in processing data where multi-colinearity exists between the features/variables. PCA can be used when the dimensions of the input features are high (e.g. a lot of variables). PCA can be also used for denoising and data compression.

What are the applications of PCA?

Applications of Principal Component Analysis. PCA is predominantly used as a dimensionality reduction technique in domains like facial recognition, computer vision and image compression. It is also used for finding patterns in data of high dimension in the field of finance, data mining, bioinformatics, psychology, etc.

Why do we need kernel PCA?

It does an excellent job for datasets, which are linearly separable. But, if we use it to non-linear datasets, we might get a result which may not be the optimal dimensionality reduction. Kernel PCA uses a kernel function to project dataset into a higher dimensional feature space, where it is linearly separable.

What is the difference between ICA and PCA?

Differences between ICA and PCA  PCA removes correlations, but not higher order dependence ICA removes correlations and higher order dependence  PCA: some components are more important than others (recall eigenvalues) ICA: all components are equally important PCA: vectors are orthogonal (recall eigenvectors of covariance matrix)

What is the PCA method used for?

The PCA method can only be applied to numerical data, both for initial data analysis and for feature (dimension) reduction. It can help in obtaining the first approximate outlines of regularities underlying a given phenomenon.

How does PCA affect my data?

Applying PCA to your data has the only effect of rotating the original coordinate axes. It is a linear transformation, exactly like for example Fourier transform. Therefore as such it can really not do anything to your data. However, data represented in the new PCA space has some interesting properties.

Do I need to perform dimension reduction with PCA?

Whether you should perform dimension reduction or not is highly context dependent and it is based on your modeling assumptions and data distribution. Applying PCA to your data has the only effect of rotating the original coordinate axes. It is a linear transformation, exactly like for example Fourier transform.