Integrating structural and semantic signals in text-attributed graphs with BiGTex

Text-attributed graphs (TAGs) pose unique challenges for representation learning by requiring models to capture both the semantic richness of node-associated texts and the structural dependencies of the graph. While graph neural networks (GNNs) effectively model topological information, they are limited in handling unstructured textual data. Conversely, large language models (LLMs) excel in text understanding but lack awareness of graph structure.To address this gap, we propose BiGTex (Bidirectional Graph-Text), a hybrid architecture that tightly integrates GNNs and LLMs through stacked Graph-Text Fusion Units. Each unit enables bidirectional interaction between textual and structural representations, allowing text to guide structural reasoning and graph topology to refine textual interpretation. The model employs parameter-efficient fine-tuning using LoRA, keeping the LLM frozen while adapting to task-specific signals.Comprehensive experiments on multiple TAG benchmarks demonstrate that BiGTex achieves state-of-the-art performance in node classification and effectively generalizes to link prediction. An ablation study confirms the critical role of bidirectional attention in enhancing representational quality.Overall, this work contributes: (i) a novel hybrid architecture integrating pre-trained language models with GNNs via dual attention; (ii) a mechanism that injects structural tokens into the LLM and refines textual embeddings through cross-attention; and (iii) empirical evidence of substantial performance gains, including a 14.2% improvement on the ogbn-Arxiv dataset, demonstrating the model’s robustness and practical utility in real-world graph-based tasks.​