Multi-view image fusion for three-dimensional (3D) object detection in agricultural scenarios has received limited research attention. To address this issue, we propose Tractor-RoadBEV, a multi-stage multi-view perception framework. The proposed framework first performs two-dimensional (2D) object detection and mask enhancement to extract object-aware features from images. The multi-view features are then transformed into a unified bird’s eye view (BEV) representation through a depth mapping module and a spatial unification module. Subsequently, a temporal fusion module integrates the current BEV feature map with historical BEV features to enhance temporal consistency. Finally, the fused BEV feature map is decoded to obtain 3D object detection results. Experimental results on the constructed tractor road dataset show that TractorRoadBEV achieves F1 scores of 0.8307 and 0.8166 for person detection and vehicle detection, respectively, which significantly outperforms the classical monocular detection method Deep3DBox. In addition, the backbone network of TractorRoadBEV demonstrates strong generalization capability in BEV scene segmentation, achieving an intersection over union (IoU) of 0.602, which is 0.151 higher than that of the classical lift-splat-shoot (LSS) method. These results indicate that the proposed method provides a low-cost, efficient, and scalable perception solution for autonomous agricultural machinery in tractor road environments.