3D Point-Cloud Processing Using Panoramic Images for Object Detection
DOI:
https://doi.org/10.32628/CSEIT2410318Keywords:
3D Point Cloud, Object Detection, Mobile Measurement System, Panoramic ImageAbstract
The Remote sensing application plays a major role in real-world critical application projects. The research introduces a novel approach, "3D Point-Cloud Processing Using Panoramic Images for Object Detection," aimed at enhancing the interpretability of laser point clouds through the integration of color information derived from panoramic images. Focusing on the context of Mobile Measurement Systems (MMS), where various digital cameras are utilized, the work addresses the challenges associated with processing panoramic images offering a 360-degree view angle. The core objective is to develop a robust method for generating color point clouds by establishing a mathematical correspondence between panoramic images and laser point clouds. The collinear principle of three points guides the fusion process, involving the center of the omnidirectional multi-camera system, the image point on the sphere, and the object point. Through comprehensive experimental validation, the work confirms the accuracy of the proposed algorithm and formulas, showcasing its effectiveness in generating color point clouds within MMS. This research contributes to the present development of 3D point-cloud processing, introducing a contemporary methodology for improved object detection through the fusion of panoramic images and laser point clouds.
Downloads
References
Zeng, Fanyang, and Ruofei Zhong. "The algorithm to generate color point-cloud with the registration between panoramic image and laser point-cloud." IOP Conference Series: Earth and Environmental Science. Vol. 17. No. 1. IOP Publishing, 2014.
Jung, Kyungmin, Thomas Hitchcox, and James Richard Forbes. "Performance evaluation of 3D keypoint detectors and descriptors on colored point clouds in subsea environments." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.
Wang, Fei, et al. “PCCN-RE: Point cloud colorization network based on relevance embedding.” IET Comput. Vis. 16 (2022): 632-643.
Hua, Lei, et al. “CPC-GSCT: Visual quality assessment for colored point cloud based on geometric segmentation and color transformation.” IET Image Process. 16 (2021): 1083-1095. DOI: https://doi.org/10.1049/ipr2.12211
Hitchcox, Thomas, and James Richard Forbes. “A Point Cloud Registration Pipeline using Gaussian Process Regression for Bathymetric SLAM*.” 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020): 4615-4622. DOI: https://doi.org/10.1109/IROS45743.2020.9340944
Krivokuća, Maja, et al. “Compression of Plenoptic Point Cloud Attributes Using 6-D Point Clouds and 6-D Transforms.” IEEE Transactions on Multimedia 25 (2023): 593-607. DOI: https://doi.org/10.1109/TMM.2021.3129341
Zerman, Emin, et al. “Textured Mesh vs Coloured Point Cloud: A Subjective Study for Volumetric Video Compression.” 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) (2020): 1-6. DOI: https://doi.org/10.1109/QoMEX48832.2020.9123137
Guo, S.; Li, J.; Lai, Z.; Meng, X.; Han, S. CT- block: A novel local global 2021.
He, K.Point cloud transformer: A deep learning framework for 3D point cloud analysis. arXiv 2018, arXiv:1812.04419.
Jianqing Zhang, Li Pan and Shugen Wang 2003 Photogrammetry (Wuhan China: Wuhan University Press) pp 26-28
Wang, Fei et al. “PCCN-RE: Point cloud colourisation network based on relevance embedding.” IET Comput. Vis. 16 (2022): DOI: https://doi.org/10.1049/cvi2.12112
Zhang, M.; You, H.; Kadam, P. Pointhop: An explainable machine learning method for point cloud classification. IEEE Trans. Multimed. 2020, 22, 1744–1755 DOI: https://doi.org/10.1109/TMM.2019.2963592
Guo, S.; Li, J.; Lai, Z.; Meng, X.; Han, S. CT-block: A novel local and global features extractor for point cloud. arXiv 2021, arXiv:2111.15400 DOI: https://doi.org/10.1016/j.neucom.2022.09.056
Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G.; Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036.
Vaswani, A.; Shazeer, N.; Parmar, N. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 5998–6008.
Kaul, C.; Pears, N.; Manandhar, S. FatNet: A feature-attentive network for 3D point cloud processing. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2020; pp. 7211–7218. DOI: https://doi.org/10.1109/ICPR48806.2021.9412731
Fanyang Zeng and Ruofei Zhong 2014 IOP Conf. Ser.: Earth Environ. Sci. 17 012160
Voumard, Jérémie, et al. “Pros and Cons of Structure for Motion Embarked on a Vehicle to Survey Slopes along Transportation Lines Using 3D Georeferenced and Coloured Point Clouds.” Remote. Sens. 10 (2018): 1732. DOI: https://doi.org/10.3390/rs10111732
Yanfang Hou, Zetian Ye and Rong Yong 2011 Research on Integrated Processing of Vehicle-borne Array CCD Images and Laser Point Cloud Based on POS Data Remote Sensing Information 4 76-79
Changqiang Yang, Zetian Ye and Ruofei Zhong 2010 Integration for vehicle-borne laser point cloud and CCD image by time and space match Science of Surveying and Mapping 35(2) 32-33
Jung, Kyungmin, Thomas Hitchcox, and James Richard Forbes. "Performance evaluation of 3d keypoint detectors and descriptors on coloured point clouds in subsea environments." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023. DOI: https://doi.org/10.1109/ICRA48891.2023.10160348
Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G.; Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036.
T. Shinohara, H. Xiu, and M. Matsuoka, "Point2color: 3D Point Cloud Colorization Using a Conditional Generative Network and Differentiable Rendering for Airborne LiDAR," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, 2021, pp. 1062-1071. DOI: https://doi.org/10.1109/CVPRW53098.2021.00117
S. Ikeda, T. Sato and N. Yokoya 2003 A calibration method for an omnidirectional multi-camera system IS&T/SPIE’s 15th Annual Symposium on Electronic Imaging pp 499-507 DOI: https://doi.org/10.1117/12.474073
Zetian Ye, Xianlin Liu and Ruofei Zhong 2008 Board multi-sensor integrated fast data acquisition and processing system research Chinese scientific and technological achievements 12 46-48
Naroditsky O, Patterson A and Daniilidis K 2011 Auto-matic alignment of a camera with a line scan LIDAR system Proceedings IEEE International Conference on Robotics and Automation pp 3429-34 DOI: https://doi.org/10.1109/ICRA.2011.5980513
Youmei Han, Liuzao Wang and Ruofei Zhong 2010 The Calibration of the Line Scan Camera Based on Laser Scanner Acta Geodaetica et Cartographica Sinica 6 631-5
Xu, J.; Yao, C.; Ma, H.; Qian, C.; Wang, J. Automatic Point Cloud Colorization of GroundBased LiDAR Data Using Video Imagery without Position and Orientation System. Remote Sens. 2023, 15, 2658. DOI: https://doi.org/10.3390/rs15102658
Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G. Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036 DOI: https://doi.org/10.3390/rs14041036
Vechersky, Pavel, et al. “Colourising Point Clouds Using Independent Cameras.” IEEE Robotics and Automation Letters 3 (2018): 3575-3582. DOI: https://doi.org/10.1109/LRA.2018.2854290
Fanyang Zeng and Ruofei Zhong 2014 IOP Conf. Ser.: Earth Environ. Sci. 17 012160 DOI: https://doi.org/10.1088/1755-1315/17/1/012160
Downloads
Published
Issue
Section
License
Copyright (c) 2024 International Journal of Scientific Research in Computer Science, Engineering and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
