Research on Multi-sensor Fusion 3D Object Detection via PMTV-RCNN for Quadruped Robot in Railway Maintenance

Abstract

Accurate 3D object detection is critical for quadruped robots in railway maintenance. Single LiDAR suffers from sparse point clouds and lack of texture information, while cameras are sensitive to lighting conditions. This paper proposes a multi-modal fusion algorithm PMTV-RCNN that deeply integrates LiDAR point clouds and camera images. The algorithm consists of three key modules: Voxel Transformer for efficient voxel feature extraction, adaptive key point selection to highlight discriminative features, and multi-feature aggregation network for weighted fusion of color and geometric information. Experiments on KITTI and railway field datasets demonstrate that PMTV-RCNN achieves significant improvements over baseline PV-RCNN, with AP gains of 2.5%, 18.06%, and 12.11% for railway equipment/vehicles, pedestrians, and foreign objects under medium difficulty. Field experiments on a quadruped robot verify its robustness in complex railway scenarios.