mirrors/opencv_contrib
1
0
Fork 0
Code Pull Requests Packages Releases Activity

vendor: OpenCV 5.0.0 snapshot at 755e50675d97db9b7d449d8bd6b09888646f6c6e

Browse Source
This commit is contained in:
Gitea Mirror Bot
2026-08-22 00:11:13 +08:00
commit 12022378a3
3872 changed files with 2513409 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/structured_light/tutorials/capture_pattern/capture_pattern.markdown
+130
View File
@@ -0,0 +1,130 @@
Capture Gray code pattern tutorial {#tutorial_capture_graycode_pattern}
=============
Goal
----
In this tutorial you will learn how to use the *GrayCodePattern* class to:
- Generate a Gray code pattern.
- Project the Gray code pattern.
- Capture the projected Gray code pattern.
It is important to underline that *GrayCodePattern* class actually implements the 3DUNDERWORLD algorithm described in @cite UNDERWORLD , which is based on a stereo approach: we need to capture the projected pattern at the same time from two different views if we want to reconstruct the 3D model of the scanned object. Thus, an acquisition set consists of the images captured by each camera for each image in the pattern sequence.
Code
----
@include structured_light/samples/cap_pattern.cpp
Explanation
-----------
First of all the pattern images to project must be generated. Since the number of images is a function of the projector's resolution, *GrayCodePattern* class parameters must be set with our projector's width and height. In this way the *generate* method can be called: it fills a vector of Mat with the computed pattern images:
@code{.cpp}
structured_light::GrayCodePattern::Params params;
....
params.width = parser.get<int>( 1 );
params.height = parser.get<int>( 2 );
....
// Set up GraycodePattern with params
Ptr<structured_light::GrayCodePattern> graycode = structured_light::GrayCodePattern::create( params );
// Storage for pattern
vector<Mat> pattern;
graycode->generate( pattern );
@endcode
For example, using the default projector resolution (1024 x 768), 40 images have to be projected: 20 for regular color pattern (10 images for the columns sequence and 10 for the rows one) and 20 for the color-inverted pattern, where the inverted pattern images are images with the same structure as the original but with inverted colors. This provides an effective method for easily determining the intensity value of each pixel when it is lit (highest value) and when it is not lit (lowest value) during the decoding step.
Subsequently, to identify shadow regions, the regions of two images where the pixels are not lit by projector's light and thus where there is not code information, the 3DUNDERWORLD algorithm computes a shadow mask for the two cameras views, starting from a white and a black images captured by each camera. So two additional images need to be projected and captured with both cameras:
@code{.cpp}
// Generate the all-white and all-black images needed for shadows mask computation
Mat white;
Mat black;
graycode->getImagesForShadowMasks( black, white );
pattern.push_back( white );
pattern.push_back( black );
@endcode
Thus, the final projection sequence is projected as follows: first the column and its inverted sequence, then the row and its inverted sequence and finally the white and black images.
Once the pattern images have been generated, they must be projected using the full screen option: the images must fill all the projection area, otherwise the projector full resolution is not exploited, a condition on which is based 3DUNDERWORLD implementation.
@code{.cpp}
// Setting pattern window on second monitor (the projector's one)
namedWindow( "Pattern Window", WINDOW_NORMAL );
resizeWindow( "Pattern Window", params.width, params.height );
moveWindow( "Pattern Window", params.width + 316, -20 );
setWindowProperty( "Pattern Window", WND_PROP_FULLSCREEN, WINDOW_FULLSCREEN );
@endcode
At this point the images can be captured with our digital cameras, using libgphoto2 library, recently included in OpenCV: remember to turn on gPhoto2 option in Cmake.list when building OpenCV.
@code{.cpp}
// Open camera number 1, using libgphoto2
VideoCapture cap1( CAP_GPHOTO2 );
if( !cap1.isOpened() )
{
// check if cam1 opened
cout << "cam1 not opened!" << endl;
help();
return -1;
}
// Open camera number 2
VideoCapture cap2( 1 );
if( !cap2.isOpened() )
{
// check if cam2 opened
cout << "cam2 not opened!" << endl;
help();
return -1;
}
@endcode
The two cameras must work at the same resolution and must have autofocus option disabled, maintaining the same focus during all acquisition. The projector can be positioned in the middle of the cameras.
However, before to proceed with pattern acquisition, the cameras must be calibrated. Once the calibration is performed, there should be no movement of the cameras, otherwise a new calibration will be needed.
After having connected the cameras and the projector to the computer, cap_pattern demo can be launched giving as parameters the path where to save the images, and the projector's width and height, taking care to use the same focus and cameras settings of calibration.
At this point, to acquire the images with both cameras, the user can press any key.
@code{.cpp}
// Turning off autofocus
cap1.set( CAP_PROP_SETTINGS, 1 );
cap2.set( CAP_PROP_SETTINGS, 1 );
int i = 0;
while( i < (int) pattern.size() )
{
cout << "Waiting to save image number " << i + 1 << endl << "Press any key to acquire the photo" << endl;
imshow( "Pattern Window", pattern[i] );
Mat frame1;
Mat frame2;
cap1 >> frame1; // get a new frame from camera 1
cap2 >> frame2; // get a new frame from camera 2
...
}
@endcode
If the captured images are good (the user must take care that the projected pattern is viewed from the two cameras), the user can save them pressing the enter key, otherwise pressing any other key he can take another shot.
@code{.cpp}
// Pressing enter, it saves the output
if( key == 13 )
{
ostringstream name;
name << i + 1;
save1 = imwrite( path + "pattern_cam1_im" + name.str() + ".png", frame1 );
save2 = imwrite( path + "pattern_cam2_im" + name.str() + ".png", frame2 );
if( ( save1 ) && ( save2 ) )
{
cout << "pattern cam1 and cam2 images number " << i + 1 << " saved" << endl << endl;
i++;
}
else
{
cout << "pattern cam1 and cam2 images number " << i + 1 << " NOT saved" << endl << endl << "Retry, check the path"<< endl << endl;
}
}
@endcode
The acquistion ends when all the pattern images have saved for both cameras. Then the user can reconstruct the 3D model of the captured scene using the *decode* method of *GrayCodePattern* class (see next tutorial).
modules/structured_light/tutorials/capturesinpattern/capturesinpattern.markdown
+207
View File
@@ -0,0 +1,207 @@
Capture Sinusoidal pattern tutorial {#tutorial_capture_sinusoidal_pattern}
=============
Goal
----
In this tutorial, you will learn how to use the sinusoidal pattern class to:
- Generate sinusoidal patterns.
- Project the generated patterns.
- Capture the projected patterns.
- Compute a wrapped phase map from these patterns using three different algorithms (Fourier Transform Profilometry, Phase Shifting Profilometry, Fourier-assisted Phase Shifting Profilometry)
- Unwrap the previous phase map.
Code
----
@include structured_light/samples/capsinpattern.cpp
Expalantion
-----------
First, the sinusoidal patterns must be generated. *SinusoidalPattern* class parameters have to be set by the user:
- projector width and height
- number of periods in the patterns
- set cross markers in the patterns (used to convert relative phase map to absolute phase map)
- patterns direction (horizontal or vertical)
- phase shift value (usually set to 2pi/3 to enable a cyclical system)
- number of pixels between two consecutive markers on the same row/column
- id of the method used to compute the phase map (FTP = 0, PSP = 1, FAPS = 2)
The user can also choose to save the patterns and the phase map.
@code{.cpp}
structured_light::SinusoidalPattern::Params params;
params.width = parser.get<int>(0);
params.height = parser.get<int>(1);
params.nbrOfPeriods = parser.get<int>(2);
params.setMarkers = parser.get<bool>(3);
params.horizontal = parser.get<bool>(4);
params.methodId = parser.get<int>(5);
params.shiftValue = static_cast<float>(2 * CV_PI / 3);
params.nbrOfPixelsBetweenMarkers = 70;
String outputPatternPath = parser.get<String>(6);
String outputWrappedPhasePath = parser.get<String>(7);
String outputUnwrappedPhasePath = parser.get<String>(8);
Ptr<structured_light::SinusoidalPattern> sinus = structured_light::SinusoidalPattern::create(params);
// Storage for patterns
vector<Mat> patterns;
//Generate sinusoidal patterns
sinus->generate(patterns);
@endcode
The number of patterns is always equal to three, no matter the method used to compute the phase map. Those three patterns are projected in a loop which is fine since the system is cyclical.
Once the patterns have been generated, the camera is opened and the patterns are projected, using fullscreen resolution. In this tutorial, a prosilica camera is used to capture gray images. When the first pattern is displayed by the projector, the user can press any key to start the projection sequence.
@code{.cpp}
VideoCapture cap(CAP_PVAPI);
if( !cap.isOpened() )
{
cout << "Camera could not be opened" << endl;
return -1;
}
cap.set(CAP_PROP_PVAPI_PIXELFORMAT, CAP_PVAPI_PIXELFORMAT_MONO8);
namedWindow("pattern", WINDOW_NORMAL);
setWindowProperty("pattern", WND_PROP_FULLSCREEN, WINDOW_FULLSCREEN);
imshow("pattern", patterns[0]);
cout << "Press any key when ready" << endl;
waitKey(0);
@endcode
In this tutorial, 30 images are projected so, each of the three patterns is projected ten times.
The "while" loop takes care of the projection process. The captured images are stored in a vector of Mat. There is a 30 ms delay between two successive captures.
When the projection is done, the user has to press "Enter" to start computing the phase maps.
@code{.cpp}
int nbrOfImages = 30;
int count = 0;
vector<Mat> img(nbrOfImages);
Size camSize(-1, -1);
while( count < nbrOfImages )
{
for(int i = 0; i < (int)patterns.size(); ++i )
{
imshow("pattern", patterns[i]);
waitKey(30);
cap >> img[count];
count += 1;
}
}
cout << "press enter when ready" << endl;
bool loop = true;
while ( loop )
{
char c = waitKey(0);
if( c == 10 )
{
loop = false;
}
}
@endcode
The phase maps are ready to be computed according to the selected method.
For FTP, a phase map is computed for each projected pattern, but we need to compute the shadow mask from three successive patterns, as explained in @cite faps. Therefore, three patterns are set in a vector called captures. Care is taken to fill this vector with three patterns, especially when we reach the last captures. The unwrapping algorithm needs to know the size of the captured images so, we make sure to give it to the "unwrapPhaseMap" method.
The phase maps are converted to 8-bit images in order to save them as png.
@code{.cpp}
switch(params.methodId)
{
case structured_light::FTP:
for( int i = 0; i < nbrOfImages; ++i )
{
/*We need three images to compute the shadow mask, as described in the reference paper
* even if the phase map is computed from one pattern only
*/
vector<Mat> captures;
if( i == nbrOfImages - 2 )
{
captures.push_back(img[i]);
captures.push_back(img[i-1]);
captures.push_back(img[i+1]);
}
else if( i == nbrOfImages - 1 )
{
captures.push_back(img[i]);
captures.push_back(img[i-1]);
captures.push_back(img[i-2]);
}
else
{
captures.push_back(img[i]);
captures.push_back(img[i+1]);
captures.push_back(img[i+2]);
}
sinus->computePhaseMap(captures, wrappedPhaseMap, shadowMask);
if( camSize.height == -1 )
{
camSize.height = img[i].rows;
camSize.width = img[i].cols;
}
sinus->unwrapPhaseMap(wrappedPhaseMap, unwrappedPhaseMap, camSize, shadowMask);
unwrappedPhaseMap.convertTo(unwrappedPhaseMap8, CV_8U, 1, 128);
wrappedPhaseMap.convertTo(wrappedPhaseMap8, CV_8U, 255, 128);
if( !outputUnwrappedPhasePath.empty() )
{
ostringstream name;
name << i;
imwrite(outputUnwrappedPhasePath + "_FTP_" + name.str() + ".png", unwrappedPhaseMap8);
}
if( !outputWrappedPhasePath.empty() )
{
ostringstream name;
name << i;
imwrite(outputWrappedPhasePath + "_FTP_" + name.str() + ".png", wrappedPhaseMap8);
}
}
break;
@endcode
For PSP and FAPS, three projected images are used to compute a single phase map. These three images are set in "captures", a vector working as a FIFO.Here again, phase maps are converted to 8-bit images in order to save them as png.
@code{.cpp}
case structured_light::PSP:
case structured_light::FAPS:
for( int i = 0; i < nbrOfImages - 2; ++i )
{
vector<Mat> captures;
captures.push_back(img[i]);
captures.push_back(img[i+1]);
captures.push_back(img[i+2]);
sinus->computePhaseMap(captures, wrappedPhaseMap, shadowMask);
if( camSize.height == -1 )
{
camSize.height = img[i].rows;
camSize.width = img[i].cols;
}
sinus->unwrapPhaseMap(wrappedPhaseMap, unwrappedPhaseMap, camSize, shadowMask);
unwrappedPhaseMap.convertTo(unwrappedPhaseMap8, CV_8U, 1, 128);
wrappedPhaseMap.convertTo(wrappedPhaseMap8, CV_8U, 255, 128);
if( !outputUnwrappedPhasePath.empty() )
{
ostringstream name;
name << i;
if( params.methodId == structured_light::PSP )
imwrite(outputUnwrappedPhasePath + "_PSP_" + name.str() + ".png", unwrappedPhaseMap8);
else
imwrite(outputUnwrappedPhasePath + "_FAPS_" + name.str() + ".png", unwrappedPhaseMap8);
}
if( !outputWrappedPhasePath.empty() )
{
ostringstream name;
name << i;
if( params.methodId == structured_light::PSP )
imwrite(outputWrappedPhasePath + "_PSP_" + name.str() + ".png", wrappedPhaseMap8);
else
imwrite(outputWrappedPhasePath + "_FAPS_" + name.str() + ".png", wrappedPhaseMap8);
}
}
break;
@endcode
modules/structured_light/tutorials/decode_pattern/tutorial_decode_pattern.markdown
+196
View File
@@ -0,0 +1,196 @@
Decode Gray code pattern tutorial {#tutorial_decode_graycode_pattern}
=============
Goal
----
In this tutorial you will learn how to use the *GrayCodePattern* class to:
- Decode a previously acquired Gray code pattern.
- Generate a disparity map.
- Generate a pointcloud.
Code
----
@include structured_light/samples/pointcloud.cpp
Explanation
-----------
First of all the needed parameters must be passed to the program.
The first is the name list of previously acquired pattern images, stored in a .yaml file organized as below:
@code{.cpp}
%YAML:1.0
cam1:
- "/data/pattern_cam1_im1.png"
- "/data/pattern_cam1_im2.png"
..............
- "/data/pattern_cam1_im42.png"
- "/data/pattern_cam1_im43.png"
- "/data/pattern_cam1_im44.png"
cam2:
- "/data/pattern_cam2_im1.png"
- "/data/pattern_cam2_im2.png"
..............
- "/data/pattern_cam2_im42.png"
- "/data/pattern_cam2_im43.png"
- "/data/pattern_cam2_im44.png"
@endcode
For example, the dataset used for this tutorial has been acquired using a projector with a resolution of 1280x800, so 42 pattern images (from number 1 to 42) + 1 white (number 43) and 1 black (number 44) were captured with both the two cameras.
Then the cameras calibration parameters, stored in another .yml file, together with the width and the height of the projector used to project the pattern, and, optionally, the values of white and black tresholds, must be passed to the tutorial program.
In this way, *GrayCodePattern* class parameters can be set up with the width and the height of the projector used during the pattern acquisition and a pointer to a GrayCodePattern object can be created:
@code{.cpp}
structured_light::GrayCodePattern::Params params;
....
params.width = parser.get<int>( 2 );
params.height = parser.get<int>( 3 );
....
// Set up GraycodePattern with params
Ptr<structured_light::GrayCodePattern> graycode = structured_light::GrayCodePattern::create( params );
@endcode
If the white and black thresholds are passed as parameters (these thresholds influence the number of decoded pixels), their values can be set, otherwise the algorithm will use the default values.
@code{.cpp}
size_t white_thresh = 0;
size_t black_thresh = 0;
if( argc == 7 )
{
// If passed, setting the white and black threshold, otherwise using default values
white_thresh = parser.get<size_t>( 4 );
black_thresh = parser.get<size_t>( 5 );
graycode->setWhiteThreshold( white_thresh );
graycode->setBlackThreshold( black_thresh );
}
@endcode
At this point, to use the *decode* method of *GrayCodePattern* class, the acquired pattern images must be stored in a vector of vector of Mat.
The external vector has a size of two because two are the cameras: the first vector stores the pattern images captured from the left camera, the second those acquired from the right one. The number of pattern images is obviously the same for both cameras and can be retrieved using the getNumberOfPatternImages() method:
@code{.cpp}
size_t numberOfPatternImages = graycode->getNumberOfPatternImages();
vector<vector<Mat> > captured_pattern;
captured_pattern.resize( 2 );
captured_pattern[0].resize( numberOfPatternImages );
captured_pattern[1].resize( numberOfPatternImages );
.....
for( size_t i = 0; i < numberOfPatternImages; i++ )
{
captured_pattern[0][i] = imread( imagelist[i], IMREAD_GRAYSCALE );
captured_pattern[1][i] = imread( imagelist[i + numberOfPatternImages + 2], IMREAD_GRAYSCALE );
......
}
@endcode
As regards the black and white images, they must be stored in two different vectors of Mat:
@code{.cpp}
vector<Mat> blackImages;
vector<Mat> whiteImages;
blackImages.resize( 2 );
whiteImages.resize( 2 );
// Loading images (all white + all black) needed for shadows computation
cvtColor( color, whiteImages[0], COLOR_RGB2GRAY );
whiteImages[1] = imread( imagelist[2 * numberOfPatternImages + 2], IMREAD_GRAYSCALE );
blackImages[0] = imread( imagelist[numberOfPatternImages + 1], IMREAD_GRAYSCALE );
blackImages[1] = imread( imagelist[2 * numberOfPatternImages + 2 + 1], IMREAD_GRAYSCALE );
@endcode
It is important to underline that all the images, the pattern ones, black and white, must be loaded as grayscale images and rectified before being passed to decode method:
@code{.cpp}
// Stereo rectify
cout << "Rectifying images..." << endl;
Mat R1, R2, P1, P2, Q;
Rect validRoi[2];
stereoRectify( cam1intrinsics, cam1distCoeffs, cam2intrinsics, cam2distCoeffs, imagesSize, R, T, R1, R2, P1, P2, Q, 0,
-1, imagesSize, &validRoi[0], &validRoi[1] );
Mat map1x, map1y, map2x, map2y;
initUndistortRectifyMap( cam1intrinsics, cam1distCoeffs, R1, P1, imagesSize, CV_32FC1, map1x, map1y );
initUndistortRectifyMap( cam2intrinsics, cam2distCoeffs, R2, P2, imagesSize, CV_32FC1, map2x, map2y );
........
for( size_t i = 0; i < numberOfPatternImages; i++ )
{
........
remap( captured_pattern[1][i], captured_pattern[1][i], map1x, map1y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
remap( captured_pattern[0][i], captured_pattern[0][i], map2x, map2y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
}
........
remap( color, color, map2x, map2y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
remap( whiteImages[0], whiteImages[0], map2x, map2y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
remap( whiteImages[1], whiteImages[1], map1x, map1y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
remap( blackImages[0], blackImages[0], map2x, map2y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
remap( blackImages[1], blackImages[1], map1x, map1y, INTER_NEAREST, BORDER_CONSTANT, Scalar() );
@endcode
In this way the *decode* method can be called to decode the pattern and to generate the corresponding disparity map, computed on the first camera (left):
@code{.cpp}
Mat disparityMap;
bool decoded = graycode->decode(captured_pattern, disparityMap, blackImages, whiteImages,
structured_light::DECODE_3D_UNDERWORLD);
@endcode
To better visualize the result, a colormap is applied to the computed disparity:
@code{.cpp}
double min;
double max;
minMaxIdx(disparityMap, &min, &max);
Mat cm_disp, scaledDisparityMap;
cout << "disp min " << min << endl << "disp max " << max << endl;
convertScaleAbs( disparityMap, scaledDisparityMap, 255 / ( max - min ) );
applyColorMap( scaledDisparityMap, cm_disp, COLORMAP_JET );
// Show the result
resize( cm_disp, cm_disp, Size( 640, 480 ) );
imshow( "cm disparity m", cm_disp )
@endcode
![](pics/cm_disparity.png)
At this point the point cloud can be generated using the reprojectImageTo3D method, taking care to convert the computed disparity in a CV_32FC1 Mat (decode method computes a CV_64FC1 disparity map):
@code{.cpp}
Mat pointcloud;
disparityMap.convertTo( disparityMap, CV_32FC1 );
reprojectImageTo3D( disparityMap, pointcloud, Q, true, -1 );
@endcode
Then a mask to remove the unwanted background is computed:
@code{.cpp}
Mat dst, thresholded_disp;
threshold( scaledDisparityMap, thresholded_disp, 0, 255, THRESH_OTSU + THRESH_BINARY );
resize( thresholded_disp, dst, Size( 640, 480 ) );
imshow( "threshold disp otsu", dst );
@endcode
![](pics/threshold_disp.png)
The white image of cam1 was previously loaded also as a color image, in order to map the color of the object on its reconstructed pointcloud:
@code{.cpp}
Mat color = imread( imagelist[numberOfPatternImages], IMREAD_COLOR );
@endcode
The background renoval mask is thus applied to the point cloud and to the color image:
@code{.cpp}
Mat pointcloud_tresh, color_tresh;
pointcloud.copyTo(pointcloud_tresh, thresholded_disp);
color.copyTo(color_tresh, thresholded_disp);
@endcode
Finally the computed point cloud of the scanned object can be visualized on viz:
@code{.cpp}
viz::Viz3d myWindow( "Point cloud with color");
myWindow.setBackgroundMeshLab();
myWindow.showWidget( "coosys", viz::WCoordinateSystem());
myWindow.showWidget( "pointcloud", viz::WCloud( pointcloud_tresh, color_tresh ) );
myWindow.showWidget( "text2d", viz::WText( "Point cloud", Point(20, 20), 20, viz::Color::green() ) );
myWindow.spin();
@endcode
![](pics/plane_viz.png)
modules/structured_light/tutorials/structured_light.markdown
+26
View File
@@ -0,0 +1,26 @@
Structured Light tutorials {#tutorial_structured_light}
=============================================================
- @subpage tutorial_capture_graycode_pattern
_Compatibility:_ \> OpenCV 3.0.0
_Author:_ Roberta Ravanelli
You will learn how to acquire a dataset using *GrayCodePattern* class.
- @subpage tutorial_decode_graycode_pattern
_Compatibility:_ \> OpenCV 3.0.0
_Author:_ Roberta Ravanelli
You will learn how to decode a previously acquired Gray code pattern, generating a pointcloud.
- @subpage tutorial_capture_sinusoidal_pattern
_Compatibility:_ \> OpenCV 3.0.0
_Author:_ Ambroise Moreau
You will learn how to compute phase maps using *SinusoidalPattern* class.