vendor: OpenCV 5.0.0 snapshot at 755e50675d97db9b7d449d8bd6b09888646f6c6e

2026-08-22 00:11:13 +08:00
commit 12022378a3
3872 changed files with 2513409 additions and 0 deletions
@@ -0,0 +1,91 @@
+Face landmark detection in an image {#tutorial_face_landmark_detection_in_an_image}
+===================================
+
+![](images/facereg.jpg)
+
+This application lets you detect landmarks of detected faces in an image. You can detect landmarks of all the faces found in an image
+and use them further in various applications like face swapping, face averaging etc.
+This functionality is now available in OpenCV.
+
+```
+// Command to be typed for running the sample
+./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image=/path_to_image/image.jpg
+```
+### Description of command parameters {tutorial_face_training_parameters}
+
+> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
+> * **image** i : (REQUIRED) A path to image in which face landmarks have to be detected.[example - /data/image.jpg]
+> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
+
+Understanding code
+------------------
+
+![](images/d.png)
+
+This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
+
+``` c++
+CascadeClassifier face_cascade;
+face_cascade.load(cascade_name);
+
+Mat img = imread(image);
+Ptr<Facemark> facemark = createFacemarkKazemi());
+facemark->loadModel(filename);
+cout<<"Loaded model"<<endl;
+```
+
+The above code creates a CascadeClassifier to detect face regions, and an instance of the face landmark detection class.
+We need to load a pretrained model for face landmark detection, and a cascade file for the face detection.
+It also loads the image in which landmarks have to be detected.
+
+
+``` c++
+vector<Rect> faces;
+resize(img,img,Size(460,460),0,0,INTER_LINEAR_EXACT);
+
+Mat gray;
+std::vector<Rect> faces;
+if(img.channels()>1){
+    cvtColor(img.getMat(),gray,COLOR_BGR2GRAY);
+}
+else{
+    gray = img.getMat().clone();
+}
+equalizeHist( gray, gray );
+
+face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
+```
+
+After doing some preprocessing, we first have to detect possible face regions (which will be stored in a `vector<Rect>`.
+Also, the image is resized to a smaller size as processing speed is faster with small images.
+
+
+``` c++
+vector< vector<Point2f> > shapes;
+
+if (facemark->fit(img,faces,shapes))
+{
+    for ( size_t i = 0; i < faces.size(); i++ )
+    {
+        cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
+    }
+    for (unsigned long i=0;i<faces.size();i++){
+        for(unsigned long k=0;k<shapes[i].size();k++)
+            cv::circle(img,shapes[i][k],5,cv::Scalar(0,0,255),FILLED);
+    }
+    namedWindow("Detected_shape");
+    imshow("Detected_shape",img);
+    waitKey(0);
+}
+```
+
+
+It then creates a vector of vector to store shapes for each face detected.
+The above code calls the function fit to get shapes of all detected faces in the image
+and then draws the rectangles bounding the faces and marks the desired landmarks.
+
+### Detection Results
+
+![](ab.jpg)
+
+![](ab-1.jpg)
@@ -0,0 +1,177 @@
+![](face_landmark/images/2.jpg)
+
+Training face landmark detector{#tutorial_face_training_face_landmark_detector}
+==============================
+
+This application helps to train your own face landmark detector. You can train your own face landmark detection by just providing the paths for
+directory containing the images and files containing their corresponding face landmarks. As this landmark detector was originally trained on
+[HELEN dataset](http://www.ifp.illinois.edu/~vuongle2/helen/), the training follows the format of data provided in HELEN dataset.
+
+The dataset consists of .txt files whose first line contains the image name which then follows the annotations.
+The format of the file containing annotations should be of following format :
+>       /directory/images/abc.jpg
+>       123.45,345.65
+>       321.67,543.89
+>       .... , ....
+>       .... , ....
+The above format is similar to HELEN dataset which is used for training the model.
+
+```
+// Command to be typed for running the sample
+./sample_train_landmark_detector -annotations=/home/sukhad/Downloads/code/trainset/ -config=config.xml -face_cascade=lbpcascadefrontalface.xml -model=trained_model.dat -width=460 -height=460
+```
+
+## Description of command parameters
+
+> * **annotations** a : (REQUIRED) Path to annotations txt file [example - /data/annotations.txt]
+> * **config** c : (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/config.xml]
+> * **model** m :  (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/model.dat]
+> * **width** w : (OPTIONAL)  The width which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
+> * **height** h : (OPTIONAL) The height which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
+> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a detector.
+
+## Description of training parameters
+
+
+The configuration file described above which is used while training contains the training parameters which are required for training.
+
+**The description of parameters is as follows :**
+
+1. **Cascade depth :** This stores the depth of cascade of regressors used for training.
+2. **Tree depth :** This stores the depth of trees created as weak learners during gradient boosting.
+3. **Number of trees per cascade level :** This stores number of trees required per cascade level.
+4. **Learning rate :** This stores the learning rate for gradient boosting.This is required to prevent overfitting using shrinkage.
+5. **Oversampling amount :** This stores the oversampling amount for the samples.
+6. **Number of test coordinates :** This stores number of test coordinates to be generated as samples to decide for making the split.
+7. **Lambda :** This stores the value used for calculating the probabilty which helps to select closer pixels for making the split.
+8. **Number of test splits :** This stores the number of test splits to be generated before making the best split.
+
+
+To get more detailed description about the training parameters you can refer to the [Research paper](https://pdfs.semanticscholar.org/d78b/6a5b0dcaa81b1faea5fb0000045a62513567.pdf).
+
+## Understanding code
+
+
+![](face_landmark/images/3.jpg)
+
+
+Jumping directly to the code :
+
+``` c++
+CascadeClassifier face_cascade;
+bool myDetector( InputArray image, OutputArray ROIs );
+
+bool myDetector( InputArray image, OutputArray ROIs ){
+    Mat gray;
+    std::vector<Rect> faces;
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
+    }
+    else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+    face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
+    Mat(faces).copyTo(ROIs);
+    return true;
+}
+```
+The facemark API provides the functionality to the user to use their own face detector to be used in training.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
+
+``` c++
+vector<String> filenames;
+glob(directory,filenames);
+```
+The above code creates a vector filenames for storing the names of the .txt files.
+It gets the filenames of the files in the directory.
+
+``` c++
+Mat img = imread(image);
+face_cascade.load(cascade_name);
+FacemarkKazemi::Params params;
+params.configfile = configfile_name;
+Ptr<Facemark> facemark = FacemarkKazemi::create(params);
+facemark->setFaceDetector(myDetector);
+
+```
+The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
+as function pointer to the facemark pointer created for detecting faces while training the model.
+
+``` c++
+vector<String> imagenames;
+vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
+vector<Mat> trainimages;
+loadTrainingData(filenames,trainlandmarks,imagenames);
+for(unsigned long i=0;i<300;i++){
+string imgname = imagenames[i].substr(0, imagenames[i].size()-1);
+string img = directory + string(imgname) + ".jpg";
+Mat src = imread(img);
+if(src.empty()){
+    cerr<<string("Image "+img+" not found\n.")<<endl;
+    continue;
+}
+trainimages.push_back(src);
+Trainlandmarks.push_back(trainlandmarks[i]);
+}
+```
+The above code creates std::vectors to store the images and their corresponding landmarks.
+The above code calls a function loadTrainingData to load the landmarks and the images into their respective vectors.
+
+If the dataset you downloaded is of the following format :
+```
+version: 1
+n_points:  68
+{
+ 115.167660 220.807529
+ 116.164839 245.721357
+ 120.208690 270.389841
+  ...
+}
+This is the example of the dataset available at https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
+
+```
+
+Then skip the above code for loading training data and use the following code. This sample is provided as sampleTrainLandmarkDetector2.cpp
+in the face module in opencv contrib.
+
+``` c++
+std::vector<String> images;
+std::vector<std::vector<Point2f> > facePoints;
+loadTrainingData(imagesList, annotations, images, facePoints, 0.0);
+```
+
+In the above code imagelist and annotations are the file of following format :
+```
+example of contents for images.txt:
+../trainset/image_0001.png
+../trainset/image_0002.png
+example of contents for annotation.txt:
+../trainset/image_0001.pts
+../trainset/image_0002.pts
+```
+
+These symbolize the names of images and their corresponding annotations.
+
+The above code scales images and landmarks as training on images of smaller size takes less time.
+This is because processing larger images requires more time. After scaling data it calculates mean
+shape of the data which is used as initial shape while training.
+
+Finally call the following function to perform training :
+
+``` c++
+facemark->training(Trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
+```
+In the above function scale is passed to scale all images and the corresponding landmarks so that the size of all
+images can be reduced as it takes greater time to process large images.
+This call to the train function trains the model and stores the trained model file with the given
+filename specified.As the training starts successfully you will see something like this :
+![](face_landmark/images/train1.png)
+
+
+**The error rate on trained images depends on the number of images used for training used as follows :**
+
+![](face_landmark/images/train.png)
+
+**The error rate on test images depends on the number of images used for training used as follows :**
+
+![](face_landmark/images/test.png)
@@ -0,0 +1,110 @@
+Face landmark detection in a video{#tutorial_face_landmark_detection_in_video}
+===================================
+
+This application lets you detect landmarks of detected faces in a video.This application first detects faces in a current video frame
+and then finds their facial landmarks. You just have to pass the video as input.
+```
+// Command to be typed for running the sample
+./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -video=/path_to_video/video.avi
+```
+Description of command parameters
+---------------------------------
+
+> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
+> * **video** v : (REQUIRED) A path to video in which face landmarks have to be detected.[example - /data/video.avi]
+> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
+
+### Understanding code
+
+This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
+
+``` c++
+CascadeClassifier face_cascade;
+bool myDetector( InputArray image, OutputArray ROIs );
+
+bool myDetector( InputArray image, OutputArray ROIs ){
+    Mat gray;
+    std::vector<Rect> faces;
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
+    }
+    else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+    face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
+    Mat(faces).copyTo(ROIs);
+    return true;
+}
+```
+The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
+
+``` c++
+VideoCapture cap(video);
+if(!cap.isOpened()){
+	cerr<<"Video cannot be loaded. Give correct path"<<endl;
+	return -1;
+}
+```
+
+The above code creates a video capture object and then loads the video.
+If the video is not loaded properly it prompts the user else the code proceeds.
+
+``` c++
+Mat img = imread(image);
+face_cascade.load(cascade_name);
+FacemarkKazemi::Params params;
+params.configfile = configfile_name;
+Ptr<Facemark> facemark = FacemarkKazemi::create(params);
+facemark->setFaceDetector(myDetector);
+
+```
+The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
+as function pointer to the facemark pointer created for detecting faces.
+``` c++
+vector<Rect> faces;
+vector< vector<Point2f> > shapes;
+Mat img;
+```
+The above code creates a vector to store the detected faces and a vector of vector to store shapes for each
+face detected in the current frame.
+
+``` c++
+while(1){
+	faces.clear();
+	shapes.clear();
+	cap>>img;
+	resize(img,img,Size(600,600),0,0,INTER_LINEAR_EXACT);
+	facemark->getFaces(img,faces);
+	if(faces.size()==0){
+	    cout<<"No faces found in this frame"<<endl;
+	}
+	else{
+	    for( size_t i = 0; i < faces.size(); i++ )
+	    {
+		cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
+	    }
+	    if(facemark->fit(img,faces,shapes))
+	    {
+		for(unsigned long i=0;i<faces.size();i++){
+		    for(unsigned long k=0;k<shapes[i].size();k++)
+		        cv::circle(img,shapes[i][k],3,cv::Scalar(0,0,255),FILLED);
+		}
+	    }
+	}
+	namedWindow("Detected_shape");
+	imshow("Detected_shape",img);
+	if(waitKey(1) >= 0) break;
+}
+```
+
+The above code then reads each frame and detects faces and the landmarks corresponding to each shape detected.
+It then displays the current frame.
+
+After running the above code you will get results something like this
+
+Sample video:
+
+@htmlonly
+<iframe width="560" height="315" src="https://www.youtube.com/embed/ZtaV07T90D8" frameborder="0" allowfullscreen></iframe>
+@endhtmlonly
@@ -0,0 +1,147 @@
+Face swapping using face landmark detection{#tutorial_face_swapping_face_landmark_detection}
+===========================================
+
+This application lets you swap a face in one image with another face in other image. The application first detects faces in both images and finds its landmarks. Then it swaps the face in first image with in another image. You just have to give paths to the images run the application to swap the two faces.
+```
+// Command to be typed for running the sample
+./sample_face_swapping -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image1=/path_to_image/image1.jpg -image2=/path_to_image/image2.jpg
+```
+### Description of command parameters
+
+> * **image1** i1 (REQUIRED) Path to the first image file in which you want to apply swapping.
+> * **image2** i2 (REQUIRED) Path to the second image file in which you want to apply face swapping.
+> * **model** m (REQUIRED) Path to the file containing model to be loaded for face landmark detection.
+> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a face detector.
+
+### Understanding the code
+
+This tutorial will explain the sample code for face swapping using OpenCV. Jumping directly to the code :
+
+``` c++
+CascadeClassifier face_cascade;
+bool myDetector( InputArray image, OutputArray ROIs );
+
+bool myDetector( InputArray image, OutputArray ROIs ){
+    Mat gray;
+    std::vector<Rect> faces;
+    if(image.channels()>1){
+        cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
+    }
+    else{
+        gray = image.getMat().clone();
+    }
+    equalizeHist( gray, gray );
+    face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
+    Mat(faces).copyTo(ROIs);
+    return true;
+}
+```
+The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
+
+
+``` c++
+Mat img = imread(image);
+face_cascade.load(cascade_name);
+FacemarkKazemi::Params params;
+params.configfile = configfile_name;
+Ptr<Facemark> facemark = FacemarkKazemi::create(params);
+facemark->setFaceDetector(myDetector);
+```
+The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
+as function pointer to the facemark pointer created for detecting faces while training the model.
+``` c++
+vector<Rect> faces1,faces2;
+vector< vector<Point2f> > shape1,shape2;
+float ratio1 = (float)img1.cols/(float)img1.rows;
+float ratio2 = (float)img2.cols/(float)img2.rows;
+resize(img1,img1,Size(640*ratio1,640*ratio1),0,0,INTER_LINEAR_EXACT);
+resize(img2,img2,Size(640*ratio2,640*ratio2),0,0,INTER_LINEAR_EXACT);
+Mat img1Warped = img2.clone();
+facemark->getFaces(img1,faces1);
+facemark->getFaces(img2,faces2);
+facemark->fit(img1,faces1,shape1);
+facemark->fit(img2,faces2,shape2);
+
+```
+
+The above code creates vectors to store the detected faces and a vector of vector to store shapes for each
+face detected in both the images.It then detects landmarks of each face detected in both the images.the images are resized
+as it is easier to process small images. The images are resized according their actual ratio.
+
+
+``` c++
+vector<Point2f> boundary_image1;
+vector<Point2f> boundary_image2;
+vector<int> index;
+convexHull(Mat(points2),index, false, false);
+for(size_t i = 0; i < index.size(); i++)
+{
+    boundary_image1.push_back(points1[index[i]]);
+    boundary_image2.push_back(points2[index[i]]);
+}
+```
+
+The above code then finds convex hull to find the boundary points of the face in the image which has to be swapped.
+
+``` c++
+vector< vector<int> > triangles;
+Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
+divideIntoTriangles(rect, boundary_image2, triangles);
+for(size_t i = 0; i < triangles.size(); i++)
+{
+    vector<Point2f> triangle1, triangle2;
+    for(int j = 0; j < 3; j++)
+    {
+        triangle1.push_back(boundary_image1[triangles[i][j]]);
+        triangle2.push_back(boundary_image2[triangles[i][j]]);
+    }
+    warpTriangle(img1, img1Warped, triangle1, triangle2);
+}
+```
+
+Now as we need to warp one face over the other and we need to find affine transform.
+Now as the function in OpenCV to find affine transform requires three set of points to calculate
+the affine matrix. Also we just need to warp the face instead of the surrounding regions. Hence
+we divide the face into triangles so that each triiangle can be easily warped onto the other image.
+
+The function divideIntoTriangles divides the detected faces into triangles.
+The function warpTriangle then warps each triangle of one image to other image  to swap the faces.
+
+``` c++
+vector<Point> hull;
+for(size_t i = 0; i < boundary_image2.size(); i++)
+{
+    Point pt((int)boundary_image2[i].x,(int)boundary_image2[i].y);
+    hull.push_back(pt);
+}
+Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
+fillConvexPoly(mask,&hull[0],(int)hull.size(), Scalar(255,255,255));
+Rect r = boundingRect(boundary_image2);
+Point center = (r.tl() + r.br()) / 2;
+Mat output;
+img1Warped.convertTo(img1Warped, CV_8UC3);
+seamlessClone(img1Warped,img2, mask, center, output, NORMAL_CLONE);
+imshow("Face_Swapped", output);
+```
+
+Even after warping the results somehow look unnatural. Hence to improve the results we apply seamless cloning
+to get the desired results as required.
+
+### Results
+
+Consider two images to be used for face swapping as follows :
+
+First image
+-----------
+
+![](images/227943776_1.jpg)
+
+Second image
+------------
+
+![](images/230501201_1.jpg)
+
+Results after swapping
+----------------------
+
+![](images/face_swapped.jpg)