Noise Robust Evolving Hierarchical Stochastic Configuration Architecture With Multi-Level Sparse Representation

Neural architecture search (NAS) can automatically design deep neural network architectures according to optimization objectives to improve model performance. Aiming at solving regression problems with different levels of potential noise, in this paper, we present a noise robust evolving hierarchical stochastic configuration architecture with multi-level sparse representation, named as RESCA. Firstly, to learn high-level representation information, we introduce a graph embedding sparse autoencoder based on stochastic configuration algorithm, then we stacked multi-sparse autoencoders to extract more efficient features. Subsequently, as$\ell _{1}$norm loss functions are more robust than$l_{2}$norm loss functions to tackle noisy data, we adopt$\ell _{1}$norm loss with elastic net regularization method as the loss function of deep stochastic configuration networks. Furthermore, to achieve an optimal regression performance and simplify network structure, swarm intelligence-based optimization algorithm is employed to search the neural architecture of RESCA. Finally, experimental results over several regression datasets with outliers demonstrate that, RESCA perform favorably with better robustness with respect to the previous robust stochastic configuration networks.