Manifold Factor Analysis: Nonlinear Dimension Reduction with Statistical Guarantees

Abstract

Classical factor analysis assumes observed variables are linear combinations of latent factors plus isotropic noise, implicitly restricting the latent structure to an affine subspace. This paper relaxes the linearity assumption by modeling the latent variable mapping as a smooth embedding from R^m to R^p, such that observations concentrate near an unknown m-dimensional submanifold of the observation space. We establish three main results. First, we prove local identifiability of the model parameters under a full-rank Jacobian condition and unit-variance latent normalization, resolving rotational indeterminacy in the nonlinear setting. Second, we derive a maximum marginal likelihood estimator and prove its √n-consistency and asymptotic normality under regularity conditions that accommodate heteroskedastic measurement errors. Third, we develop a two-stage procedure that combines kernel spectral analysis for intrinsic dimension estimation with likelihood- based manifold reconstruction, and prove that the estimated dimension is consistent under a spectral gap condition. Simulation studies demonstrate that the proposed estimator achieves 40–60% lower RMSE than linear factor analysis under quadratic and multiplicative nonlinearities, with coverage probabilities approaching the nominal 95% level at n = 1000. Empirical analyses of marketing, banking, and Parkinson's telemonitoring datasets show that the recovered latent manifolds have 1–2 fewer dimensions than linear PCA suggests, improve cross-validated prediction error by 15–30%, and yield clinically interpretable latent factors in the biomedical application.