3D Point-Cloud Processing Using Panoramic Images for Object Detection

Authors

  • Lokesh M R Department of computer science and Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India Author
  • Anushitha K Department of computer science and Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India Author
  • Ashok D Department of computer science and Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India Author
  • Deepak Raj K Department of computer science and Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India Author
  • Harshitha K Department of computer science and Engineering, Vivekananda College of Engineering and Technology, Puttur, Karnataka, India Author

DOI:

https://doi.org/10.32628/CSEIT2410318

Keywords:

3D Point Cloud, Object Detection, Mobile Measurement System, Panoramic Image

Abstract

The Remote sensing application plays a major role in real-world critical application projects. The research introduces a novel approach, "3D Point-Cloud Processing Using Panoramic Images for Object Detection," aimed at enhancing the interpretability of laser point clouds through the integration of color information derived from panoramic images. Focusing on the context of Mobile Measurement Systems (MMS), where various digital cameras are utilized, the work addresses the challenges associated with processing panoramic images offering a 360-degree view angle. The core objective is to develop a robust method for generating color point clouds by establishing a mathematical correspondence between panoramic images and laser point clouds. The collinear principle of three points guides the fusion process, involving the center of the omnidirectional multi-camera system, the image point on the sphere, and the object point. Through comprehensive experimental validation, the work confirms the accuracy of the proposed algorithm and formulas, showcasing its effectiveness in generating color point clouds within MMS. This research contributes to the present development of 3D point-cloud processing, introducing a contemporary methodology for improved object detection through the fusion of panoramic images and laser point clouds.

Downloads

Download data is not yet available.

References

Zeng, Fanyang, and Ruofei Zhong. "The algorithm to generate color point-cloud with the registration between panoramic image and laser point-cloud." IOP Conference Series: Earth and Environmental Science. Vol. 17. No. 1. IOP Publishing, 2014.

Jung, Kyungmin, Thomas Hitchcox, and James Richard Forbes. "Performance evaluation of 3D keypoint detectors and descriptors on colored point clouds in subsea environments." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

Wang, Fei, et al. “PCCN-RE: Point cloud colorization network based on relevance embedding.” IET Comput. Vis. 16 (2022): 632-643.

Hua, Lei, et al. “CPC-GSCT: Visual quality assessment for colored point cloud based on geometric segmentation and color transformation.” IET Image Process. 16 (2021): 1083-1095. DOI: https://doi.org/10.1049/ipr2.12211

Hitchcox, Thomas, and James Richard Forbes. “A Point Cloud Registration Pipeline using Gaussian Process Regression for Bathymetric SLAM*.” 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020): 4615-4622. DOI: https://doi.org/10.1109/IROS45743.2020.9340944

Krivokuća, Maja, et al. “Compression of Plenoptic Point Cloud Attributes Using 6-D Point Clouds and 6-D Transforms.” IEEE Transactions on Multimedia 25 (2023): 593-607. DOI: https://doi.org/10.1109/TMM.2021.3129341

Zerman, Emin, et al. “Textured Mesh vs Coloured Point Cloud: A Subjective Study for Volumetric Video Compression.” 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) (2020): 1-6. DOI: https://doi.org/10.1109/QoMEX48832.2020.9123137

Guo, S.; Li, J.; Lai, Z.; Meng, X.; Han, S. CT- block: A novel local global 2021.

He, K.Point cloud transformer: A deep learning framework for 3D point cloud analysis. arXiv 2018, arXiv:1812.04419.

Jianqing Zhang, Li Pan and Shugen Wang 2003 Photogrammetry (Wuhan China: Wuhan University Press) pp 26-28

Wang, Fei et al. “PCCN-RE: Point cloud colourisation network based on relevance embedding.” IET Comput. Vis. 16 (2022): DOI: https://doi.org/10.1049/cvi2.12112

Zhang, M.; You, H.; Kadam, P. Pointhop: An explainable machine learning method for point cloud classification. IEEE Trans. Multimed. 2020, 22, 1744–1755 DOI: https://doi.org/10.1109/TMM.2019.2963592

Guo, S.; Li, J.; Lai, Z.; Meng, X.; Han, S. CT-block: A novel local and global features extractor for point cloud. arXiv 2021, arXiv:2111.15400 DOI: https://doi.org/10.1016/j.neucom.2022.09.056

Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G.; Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036.

Vaswani, A.; Shazeer, N.; Parmar, N. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 5998–6008.

Kaul, C.; Pears, N.; Manandhar, S. FatNet: A feature-attentive network for 3D point cloud processing. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2020; pp. 7211–7218. DOI: https://doi.org/10.1109/ICPR48806.2021.9412731

Fanyang Zeng and Ruofei Zhong 2014 IOP Conf. Ser.: Earth Environ. Sci. 17 012160

Voumard, Jérémie, et al. “Pros and Cons of Structure for Motion Embarked on a Vehicle to Survey Slopes along Transportation Lines Using 3D Georeferenced and Coloured Point Clouds.” Remote. Sens. 10 (2018): 1732. DOI: https://doi.org/10.3390/rs10111732

Yanfang Hou, Zetian Ye and Rong Yong 2011 Research on Integrated Processing of Vehicle-borne Array CCD Images and Laser Point Cloud Based on POS Data Remote Sensing Information 4 76-79

Changqiang Yang, Zetian Ye and Ruofei Zhong 2010 Integration for vehicle-borne laser point cloud and CCD image by time and space match Science of Surveying and Mapping 35(2) 32-33

Jung, Kyungmin, Thomas Hitchcox, and James Richard Forbes. "Performance evaluation of 3d keypoint detectors and descriptors on coloured point clouds in subsea environments." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023. DOI: https://doi.org/10.1109/ICRA48891.2023.10160348

Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G.; Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036.

T. Shinohara, H. Xiu, and M. Matsuoka, "Point2color: 3D Point Cloud Colorization Using a Conditional Generative Network and Differentiable Rendering for Airborne LiDAR," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, 2021, pp. 1062-1071. DOI: https://doi.org/10.1109/CVPRW53098.2021.00117

S. Ikeda, T. Sato and N. Yokoya 2003 A calibration method for an omnidirectional multi-camera system IS&T/SPIE’s 15th Annual Symposium on Electronic Imaging pp 499-507 DOI: https://doi.org/10.1117/12.474073

Zetian Ye, Xianlin Liu and Ruofei Zhong 2008 Board multi-sensor integrated fast data acquisition and processing system research Chinese scientific and technological achievements 12 46-48

Naroditsky O, Patterson A and Daniilidis K 2011 Auto-matic alignment of a camera with a line scan LIDAR system Proceedings IEEE International Conference on Robotics and Automation pp 3429-34 DOI: https://doi.org/10.1109/ICRA.2011.5980513

Youmei Han, Liuzao Wang and Ruofei Zhong 2010 The Calibration of the Line Scan Camera Based on Laser Scanner Acta Geodaetica et Cartographica Sinica 6 631-5

Xu, J.; Yao, C.; Ma, H.; Qian, C.; Wang, J. Automatic Point Cloud Colorization of GroundBased LiDAR Data Using Video Imagery without Position and Orientation System. Remote Sens. 2023, 15, 2658. DOI: https://doi.org/10.3390/rs15102658

Jing, W.; Zhang, W.; Li, L.; Di, D.; Chen, G. Wang, J. AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens. 2022, 14, 1036 DOI: https://doi.org/10.3390/rs14041036

Vechersky, Pavel, et al. “Colourising Point Clouds Using Independent Cameras.” IEEE Robotics and Automation Letters 3 (2018): 3575-3582. DOI: https://doi.org/10.1109/LRA.2018.2854290

Fanyang Zeng and Ruofei Zhong 2014 IOP Conf. Ser.: Earth Environ. Sci. 17 012160 DOI: https://doi.org/10.1088/1755-1315/17/1/012160

Downloads

Published

15-05-2024

Issue

Section

Research Articles

How to Cite

[1]
Lokesh M R, Anushitha K, Ashok D, Deepak Raj K, and Harshitha K, “3D Point-Cloud Processing Using Panoramic Images for Object Detection”, Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol, vol. 10, no. 3, pp. 186–198, May 2024, doi: 10.32628/CSEIT2410318.

Similar Articles

1-10 of 210

You may also start an advanced similarity search for this article.