Enhanced Tomlinson-Harashima precoding and adapted likelihood-based radius-directed equalization for transmitter-side bandwidth-limited faster-than-Nyquist systems over dynamic channels.

The proliferation of bandwidth-intensive applications, such as AI cloud computing, has necessitated high-baud-rate transmission. However, this progression is often impeded by the transmitter-side bandwidth bottleneck, particularly in cost-sensitive short-reach scenarios. Although adaptive equalizers (AEQs) can mitigate such a bandwidth bottleneck, they do so at the cost of introducing equalization-enhanced high-frequency noise (EEHN), which severely degrades system performance. Tomlinson-Harashima precoding (THP) has emerged as a promising solution to counter this effect. However, its practical deployment is hindered by two critical limitations that this paper aims to resolve. First, conventional THP suffers from residual EEHN due to its inability to mitigate precursor inter-symbol interference (ISI). Second, its associated AEQ, which depends on training sequences for convergence, demonstrates unstable tracking performance over practical dynamic channels characterized by fast rotation of state of polarization (RSOP). To overcome these challenges, we propose an enhanced THP scheme at the transmitter to suppress the EEHN and a likelihood-based selection (LBS) radius-directed equalization (RDE) scheme adapted for THP-expanded constellations at the receiver (Rx) to ensure stable operation over dynamic channels. The effectiveness of the proposed scheme is validated through simulations and experiments. For a transmitter with a 3 dB bandwidth constrained to one-third of the baud rate, the proposed enhanced THP achieves approximately 1 dB sensitivity gain over conventional THP under static channel conditions. Under dynamic channels with a 500 krad/s RSOP, the AEQ relying solely on training sequences fails to converge, whereas the AEQ additionally incorporating payload symbols for RDE updates also suffers from notable degradation owing to frequent decision errors. In contrast, the proposed scheme operates reliably up to 1000 krad/s. At the 500 krad/s and 1000 krad/s RSOP speeds, it achieves a sensitivity improvement of over 1.5 dB and 3.8 dB, respectively. The proposed scheme offers a comprehensive solution for enhancing the performance of transmitter-side bandwidth-limited faster-than-Nyquist systems over practical dynamic channels.