Seyedeh Maryam Mousavi, Saeid Khasi, Apostolos Kantzas
Geothermal energy offers a continuous and low-carbon source of heat, yet its efficiency is limited by the thermal performance of the working fluid. This study develops and validates a finite-element model for a coaxial closed-loop geothermal system employing surface-modified nanofluids. Water-based nanofluids containing graphene and aluminum nanoparticles functionalized with APTES or SDS were examined over volume fractions of 10 -5 –10 -2 . The model couples fluid flow and heat transfer equations using the SST turbulence framework and performs sensitivity analyses on flow rate, nanoparticle concentration, and inlet temperature.Results show that nanofluids significantly improve outlet temperature, energy recovery, and overall thermal performance compared with water. Graphene/APTES nanofluid achieved a 42 % increase in outlet temperature and a 26 % enhancement in energy recovery at 0.01 m s -1 , while graphene–aluminum hybrid nanofluid provided an optimal balance between heat-transfer gain and pressure loss, improving energy recovery by 31 % and reducing friction factor by over 20 %. Surface modification with APTES and SDS lowered viscosity by 6–8 %, contributing to reduced pumping power. Model predictions agreed closely with experimental literature data, yielding an RMSE < 13 % for generic properties and < 1 % when experimental values were applied. These results confirm the robustness of the model and the potential of surface-engineered hybrid nanofluids to enhance geothermal energy extraction efficiency in Alberta-type formations.