The principal component analysis (PCA) is widely used to reduce the dimensionality of a dataset to its essentialcomponents. To perform PCA, the covariance matrix is constructed and its eigenvalues and eigenvectors are computed. In practical numerical applications, the tail of the sorted eigenvalues is sometimes found to contain negative eigenvalues, which are prohibited mathematically and are a pure consequence of finite-accuracy numerics. The present study suggests that in the case of a many-body dynamical system, the spurious negative eigenvalues of the covariance matrix may in fact be related to the frozen degrees of freedom in the system. Here, we outline the mathematical connection between the eigenvalues of the covariance matrix and the frozen degrees of freedom and validate the connection through two case studies: a model system of coupled harmonic oscillators and a molecular dynamics simulation of a small protein in solution.