NIRPed

The device is fixed onto the vehicle roof with four magnets; the camera is mounted above the LiDAR via a ring bracket, and two jigs allow fine adjustment of the camera’s pitch and yaw angles to align with the LiDAR’s field of view.2. Data Collection and ProcessingData were collected from 2018 to 2021 in Changsha (urban) and Shaoyang (suburban/campus) areas of Hunan Province, China. To ensure diversity, the collection covered hot/cold seasons, sunny/rainy weather, and various scenes including urban streets, university campuses, and suburban roads.

The vehicle was driven at normal speeds to account for the effect of speed on imaging quality. The camera operated at 3 Hz and the LiDAR at 10 Hz. LiDAR point cloud packets were acquired in real‑time via UDP socket protocol, parsed into 3D coordinates and distance values, and saved as pickle files.

Both images and point cloud files were time‑stamped to the microsecond for subsequent spatiotemporal matching. The pickle file with the smallest time difference from a given image was selected as the best temporal match; all distance points were then projected onto the corresponding image for spatial fusion. Multiple distance points often fell on the same pedestrian; the minimum distance among them was extracted as the ground‑truth distance and annotated in the dataset, consistent with the principle of ensuring driving safety.

A total of 29 data sequences (13 hours total duration) were obtained, yielding 142,000 images and 432,000 point cloud files.3. Dataset Content and Annotation SpecificationsNIRPed is divided into training, validation, and testing subsets at a ratio of 4:3:3. The training subset contains 64,000 pedestrian annotations, the validation set 43,000, and the test set 39,000.

All images were manually annotated using MATLAB’s Ground‑Truth Labeler graphical interface. Each image is labelled with four environmental attributes: season (cold/hot), weather (clear/rainy), city (Changsha/Shaoyang), and scene (urban/campus/suburban). For each pedestrian‑related target (pedestrian, cyclist, motorcyclist), a full bounding box (enclosing the entire target) and a visible bounding box (enclosing only the visible region) are provided, along with attributes for distance, occlusion rate, and difficulty level.

Crowded pedestrians are annotated as “people”; persons inside vehicles or requiring contextual information from adjacent frames are marked as “ignore areas”; human figures or characters in posters/pictures are annotated as “background”. Annotations are provided in MS‑COCO JSON format, compatible with standard object‑detection frameworks. The annotations for the training and validation sets are publicly released, while those for the test set are withheld to ensure fair evaluation.4.

Data Gaps and Error

Notes

At present, the dataset mainly covers sunny and rainy conditions; severe weather such as snow and fog has not yet been included. Some imaging challenges remain in nighttime NIR scenes, including reflections from NIR light, occlusions, motion blur caused by long exposure and relative motion between the ego‑vehicle and targets, especially in extreme rainy or foggy conditions. Regarding distance accuracy, the LiDAR provides a precision of 2 cm; during fusion, multiple distance points on the same target may exist, and selecting the minimum value as the ground truth may introduce a negligible bias, but this does not significantly affect the overall distance distribution.The dataset is freely available for non‑commercial academic research.

The complete data can be accessed via ScienceDB. Users are requested to cite the related paper when using the dataset: Xiaobiao Dai, Junping Hu, Chunlei Luo, Houcine Zerfa, Hai Zhang, and Yuxia Duan, "NIRPed: A Novel Benchmark for Nighttime Pedestrian and Its Distance Joint Detection," in IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 7, pp. 6932-6942, July 2023, doi: 10.1109/TITS.2023.3257079.