mmSeg: Leveraging mmWave Radar for Fine-grained Human Semantic Segmentation

Human semantic segmentation facilitates the recognition of different parts of the human body and is essential for applications such as sports analysis and fall detection. To integrate human semantic segmentation into the domain of radio front-end sensing, this article introduces mmSeg, an innovative system that leverages commercial millimeter-wave radar for human semantic segmentation. However, the inherent propagation characteristics of mmWave signals often result in highly sparse point clouds with limited semantic information and the entanglement of temporal-topological features, making human semantic segmentation a challenging task. To address these challenges, mmSeg (i) first introduces a radar cross-section (RCS) calculation method suitable for commercial millimeter-wave radar to enhance the semantic information of radar point clouds at a coarse granularity; (ii) further designs a temporal-topological decoupling network to obtain the fine-grained human semantic segmentation results; (iii) constructs an efficient loss function for end-to-end training, based on an adjacency matrix graph to improve the segmentation performance. We evaluate mmSeg on our self-built millimeter-wave dataset HSS and a public dataset MM-Fi. mmSeg achieves an average point cloud segmentation accuracy of 87.74% on the HSS dataset and 84.18% on the MM-Fi dataset, outperforming the existing methods in both cases.