Zafer Doğan
AbstractContext— Many machine learning and signal processing methods assume Gaussian data because it simplifies analysis and often works reasonably well in practice. But real data are rarely Gaussian. They can have heavy tails, skewness, or unusually large kurtosis, and these differences matter. In fact, non-Gaussianity is exactly what makes problems like Independent Component Analysis (ICA) identifiable in the first place. What’s less clear is how these distributional differences affect learning dynamics, particularly in high-dimensional online settings. Recent work has shown that online ICA can be described using low-dimensional deterministic equations in certain scaling limits. However, most of these analyses assume a fixed source distribution and do not explore what happens when higher-order moments vary systematically. As a result, we still do not fully understand how changes in kurtosis or tail behavior influence stability, convergence speed, or sensitivity to the learning rate.Objective— This work quantifies how controlled changes in higher-order moment structure shape the macroscopic dynamics of high-dimensional online ICA. We analyze how moment variations interact with initialization and learning rate to determine stability, convergence, and learning speed.Method— We analyze a high-dimensional online ICA model using a McKean-type scaling limit, which leads to a deterministic ordinary differential equation (ODE) for the alignment order parameter. To keep the analysis tractable, we focus on the cubic nonlinearity. We then introduce a moment-controlled data model in which the source distribution is constructed as a weighted mixture of two non-Gaussian random variables. This setup preserves zero mean and unit variance while allowing continuous control over the fourth and sixth moments through a single parameter. In the scaling limit, the resulting ODE has explicit drift and diffusion terms that depend directly on these moments. This makes it possible to carry out a detailed phase-plane and stability analysis and to clearly see how higher-order statistics shape the learning dynamics. We validate the theoretical predictions through simulations of both the limiting ODE and the corresponding finite-dimensional algorithm.Results— The dynamics exhibit moment-induced phase transitions. Increasing fourth and sixth moments shrink the basin of attraction of informative solutions, raise initialization thresholds, and reduce the admissible learning-rate range. The drift term is governed by excess kurtosis, while diffusion grows with higher-order moments, producing a trade-off between signal strength and stochastic damping. Simulations confirm slower convergence, increased sensitivity to initialization, and reduced robustness at large-moment regimes.Conclusion— Non-Gaussianity is necessary for identifiability but can destabilize high-dimensional online learning. Our analysis reveals a fundamental trade-off between statistical richness and algorithmic stability in online ICA. The proposed framework directly links higher-order data statistics to learning dynamics and provides a principled basis for adaptive step-size selection, robust initialization, and extensions to alternative nonlinearities and multi-source settings.