Deep reinforcement learning-based energy management for design and control of off-grid renewable microgrids with dual-battery storage

Meeting the growing global electricity demand in remote and off-grid regions requires cost-effective and reliable power solutions that overcome the intermittency of renewable energy sources. This paper presents a comprehensive techno-economic optimization framework for the design and operation of off-grid hybrid renewable energy systems (HRES) integrating photovoltaic (PV), wind turbine, biomass generator, diesel backup, and a dual-chemistry hybrid battery energy storage system (HBESS) combining lithium-ion and nickel-iron batteries. A detailed mathematical modeling approach is employed to capture the nonlinear dynamics, stochastic renewable behavior, battery degradation, and temperature-adjusted component efficiencies. The system is formulated as a multi-objective mixed-integer nonlinear programming problem targeting the minimization of life cycle cost (LCC), levelized cost of energy (LCOE), and CO 2 emissions while satisfying reliability constraints such as loss of power supply probability (LPSP < 0.01). To solve the optimization problem, advanced metaheuristic algorithms—Particle Swarm Optimization (PSO), Genetic Algorithms (GA), Grey Wolf Optimizer (GWO), and Differential Evolution (DE), and Salp Swarm Algorithm (SSA)—and a Deep Q-Network (DQN)-based reinforcement learning energy management strategy are implemented and benchmarked. The proposed DQN-based controller demonstrates superior performance over conventional rule-based and static dispatch methods by maintaining more stable battery state-of-charge (SOC) profiles, reducing degradation, and enabling intelligent real-time decision-making. Simulation results based on realistic meteorological and demand profiles reveal that the integrated DQN and HBESS strategy reduces total LCC by over 20%, CO 2 emissions by up to 30%, and battery degradation costs by over 10% compared to baseline systems. The Salp Swarm Algorithm (SSA) achieves the fastest convergence and the highest-quality Pareto-optimal solutions among all metaheuristics evaluated. Sensitivity analysis identifies diesel price and interest rate as the most influential parameters on LCOE, while load shifting through aggressive demand-side management further minimizes battery usage, operating costs, and emissions. The proposed framework not only addresses key challenges in off-grid microgrid design but also provides a scalable and robust pathway for sustainable rural electrification using hybrid storage and intelligent control.