$\newcommand{\d}{\mathrm{d}} \newcommand{\bff}{\boldsymbol{f}} \newcommand{\bfg}{\boldsymbol{g}} \newcommand{\bfp}{\boldsymbol{p}} \newcommand{\bfq}{\boldsymbol{q}} \newcommand{\bfx}{\boldsymbol{x}} \newcommand{\bfu}{\boldsymbol{u}} \newcommand{\bfv}{\boldsymbol{v}} \newcommand{\bfA}{\boldsymbol{A}} \newcommand{\bfB}{\boldsymbol{B}} \newcommand{\bfC}{\boldsymbol{C}} \newcommand{\bfM}{\boldsymbol{M}} \newcommand{\bfJ}{\boldsymbol{J}} \newcommand{\bfR}{\boldsymbol{R}} \newcommand{\bfT}{\boldsymbol{T}} \newcommand{\bfomega}{\boldsymbol{\omega}} \newcommand{\bftau}{\boldsymbol{\tau}}$

Processing 3D point clouds using PCL

Contrary to conventional 2D cameras, which provide 2D images of the world, 3D cameras provide 3D information in the form of point clouds. A point cloud is a collection of points described by their X-Y-Z coordinates.

PCL is a good library that provides a number of functionalities to manipulate point clouds. Unfortunately, contrary to OpenCV, the Python bindings to PCL are very limited. This tutorial will therefore deal with the C++ library.

Since there is already an excellent PCL tutorial, we shall not duplicate the effort here. The reader is advised to go through the whole tutorial, with particular attention to the following sections:

1. Basic usage;

2. I/O;

3. Filtering;

4. Features;

5. Recognition;

6. Registration.

Many robotic applications require finding the pose (rotation and translation) of an object in a scene. The reader is advised to go through the pose estimation example in the PCL tutorial to familiarize with that task. Note that, depending on the models and scene at hand, some parameter values might need to be adjusted. Please play around with them to see how they influence the final result.