mirrors/opencv_contrib
1
0
Fork 0
Code Pull Requests Packages Releases Activity

vendor: OpenCV 5.0.0 snapshot at 755e50675d97db9b7d449d8bd6b09888646f6c6e

Browse Source
This commit is contained in:
Gitea Mirror Bot
2026-08-22 00:11:13 +08:00
commit 12022378a3
3872 changed files with 2513409 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/face/CMakeLists.txt
+27
View File
@@ -0,0 +1,27 @@
set(the_description "Face recognition etc")
ocv_define_module(face opencv_core
opencv_imgproc
opencv_xobjdetect
opencv_geometry # estimateAffinePartial2D() (trainFacemark)
opencv_photo # seamlessClone() (face_swap sample)
WRAP python java objc
)
# NOTE: objdetect module is needed for one of the samples
set(__commit_hash "8afa57abc8229d611c4937165d20e2a2d9fc5a12")
set(__file_hash "7505c44ca4eb54b4ab1e4777cb96ac05")
ocv_download(
FILENAME face_landmark_model.dat
HASH ${__file_hash}
URL
"${OPENCV_FACE_ALIGNMENT_URL}"
"$ENV{OPENCV_FACE_ALIGNMENT_URL}"
"https://raw.githubusercontent.com/opencv/opencv_3rdparty/${__commit_hash}/"
DESTINATION_DIR "${CMAKE_BINARY_DIR}/${OPENCV_TEST_DATA_INSTALL_PATH}/cv/face/"
ID "data"
RELATIVE_URL
STATUS res
)
if(NOT res)
message(WARNING "Face: Can't get model file for face alignment.")
endif()
modules/face/README.md
+8
View File
@@ -0,0 +1,8 @@
Face recognition techniques
===========================
Collection of face recognition techniques:
1. Eigen Faces
2. Fisher Faces
3. Local Binary Pattern Histograms
modules/face/doc/changelog.markdown
+75
View File
@@ -0,0 +1,75 @@
Face module changelog {#face_changelog}
=====================
Release 0.05
------------
This library is now included in the official OpenCV distribution (from 2.4 on). The
cv::FaceRecognizer is now an Algorithm, which better fits into the overall OpenCV API.
To reduce the confusion on user side and minimize my work, libfacerec and OpenCV have been
synchronized and are now based on the same interfaces and implementation.
The library now has an extensive documentation:
- The API is explained in detail and with a lot of code examples.
- The face recognition guide I had written for Python and GNU Octave/MATLAB has been adapted to
the new OpenCV C++ cv::FaceRecognizer.
- A tutorial for gender classification with Fisherfaces.
- A tutorial for face recognition in videos (e.g. webcam).
### Release highlights
- There are no single highlights to pick from, this release is a highlight itself.
Release 0.04
------------
This version is fully Windows-compatible and works with OpenCV 2.3.1. Several bugfixes, but none
influenced the recognition rate.
### Release highlights
- A whole lot of exceptions with meaningful error messages.
- A tutorial for Windows users:
[<http://bytefish.de/blog/opencv_visual_studio_and_libfacerec>](http://bytefish.de/blog/opencv_visual_studio_and_libfacerec)
Release 0.03
------------
Reworked the library to provide separate implementations in cpp files, because it's the preferred
way of contributing OpenCV libraries. This means the library is not header-only anymore. Slight API
changes were done, please see the documentation for details.
### Release highlights
- New Unit Tests (for LBP Histograms) make the library more robust.
- Added more documentation.
Release 0.02
------------
Reworked the library to provide separate implementations in cpp files, because it's the preferred
way of contributing OpenCV libraries. This means the library is not header-only anymore. Slight API
changes were done, please see the documentation for details.
### Release highlights
- New Unit Tests (for LBP Histograms) make the library more robust.
- Added a documentation and changelog in reStructuredText.
Release 0.01
------------
Initial release as header-only library.
### Release highlights
- Colormaps for OpenCV to enhance the visualization.
- Face Recognition algorithms implemented:
- Eigenfaces @cite TP91
- Fisherfaces @cite BHK97
- Local Binary Patterns Histograms @cite AHP04
- Added persistence facilities to store the models with a common API.
- Unit Tests (using [gtest](http://code.google.com/p/googletest/)).
- Providing a CMakeLists.txt to enable easy cross-platform building.
modules/face/doc/face.bib
+170
View File
@@ -0,0 +1,170 @@
@incollection{AHP04,
title={Face recognition with local binary patterns},
author={Ahonen, Timo and Hadid, Abdenour and Pietik{\"a}inen, Matti},
booktitle={Computer vision-eccv 2004},
pages={469--481},
year={2004},
publisher={Springer}
}
@article{BHK97,
title={Eigenfaces vs. fisherfaces: Recognition using class specific linear projection},
author={Belhumeur, Peter N. and Hespanha, Jo{\~a}o P and Kriegman, David},
journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
volume={19},
number={7},
pages={711--720},
year={1997},
publisher={IEEE}
}
@inproceedings{Bru92,
title={Face recognition through geometrical features},
author={Brunelli, Roberto and Poggio, Tomaso},
booktitle={Computer Vision—ECCV'92},
pages={792--800},
year={1992},
organization={Springer}
}
@book{Duda01,
title={Pattern classification},
author={Duda, Richard O and Hart, Peter E and Stork, David G},
year={2012},
publisher={John Wiley \& Sons}
}
@article{Fisher36,
title={The use of multiple measurements in taxonomic problems},
author={Fisher, Ronald A},
journal={Annals of eugenics},
volume={7},
number={2},
pages={179--188},
year={1936},
publisher={Wiley Online Library}
}
@article{GBK01,
title={From few to many: Illumination cone models for face recognition under variable lighting and pose},
author={Georghiades, Athinodoros S. and Belhumeur, Peter N. and Kriegman, David},
journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
volume={23},
number={6},
pages={643--660},
year={2001},
publisher={IEEE}
}
@article{Kanade73,
title={Picture processing system by computer complex and recognition of human faces},
author={Kanade, Takeo},
year={1974}
}
@article{KM01,
title={Pca versus lda},
author={Mart{\'\i}nez, Aleix M and Kak, Avinash C},
journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
volume={23},
number={2},
pages={228--233},
year={2001},
publisher={IEEE}
}
@article{Lee05,
title={Acquiring linear subspaces for face recognition under variable lighting},
author={Lee, Kuang-Chih and Ho, Jeffrey and Kriegman, David},
journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
volume={27},
number={5},
pages={684--698},
year={2005},
publisher={IEEE}
}
@incollection{Messer06,
title={Performance characterisation of face recognition algorithms and their sensitivity to severe illumination changes},
author={Messer, Kieron and Kittler, Josef and Short, James and Heusch, Guillaume and Cardinaux, Fabien and Marcel, Sebastien and Rodriguez, Yann and Shan, Shiguang and Su, Yu and Gao, Wen and others},
booktitle={Advances in Biometrics},
pages={1--11},
year={2005},
publisher={Springer}
}
@article{RJ91,
title={Small sample size effects in statistical pattern recognition: Recommendations for practitioners},
author={Raudys, Sarunas J and Jain, Anil K.},
journal={IEEE Transactions on pattern analysis and machine intelligence},
volume={13},
number={3},
pages={252--264},
year={1991},
publisher={IEEE Computer Society}
}
@article{Tan10,
title={Enhanced local texture feature sets for face recognition under difficult lighting conditions},
author={Tan, Xiaoyang and Triggs, Bill},
journal={Image Processing, IEEE Transactions on},
volume={19},
number={6},
pages={1635--1650},
year={2010},
publisher={IEEE}
}
@article{TP91,
title={Eigenfaces for recognition},
author={Turk, Matthew and Pentland, Alex},
journal={Journal of cognitive neuroscience},
volume={3},
number={1},
pages={71--86},
year={1991},
publisher={MIT Press}
}
@article{Tu06,
title={Newborns' face recognition: Role of inner and outer facial features},
author={Turati, Chiara and Macchi Cassia, Viola and Simion, Francesca and Leo, Irene},
journal={Child development},
volume={77},
number={2},
pages={297--311},
year={2006},
publisher={Wiley Online Library}
}
@article{Wiskott97,
title={Face recognition by elastic bunch graph matching},
author={Wiskott, Laurenz and Fellous, J-M and Kuiger, N and Von Der Malsburg, Christoph},
journal={Pattern Analysis and Machine Intelligence, IEEE Transactions on},
volume={19},
number={7},
pages={775--779},
year={1997},
publisher={IEEE}
}
@article{Zhao03,
title={Face recognition: A literature survey},
author={Zhao, Wenyi and Chellappa, Rama and Phillips, P Jonathon and Rosenfeld, Azriel},
journal={Acm Computing Surveys (CSUR)},
volume={35},
number={4},
pages={399--458},
year={2003},
publisher={ACM}
}
@article{Savvides04,
title={Cancelable Biometric Filters for Face Recognition},
author={Savvides, Marios and Kumar, B. V. K. Vijaya and Khosla, P. K. },
journal={Pattern Recognition, International Conference on, vol. 03, no. , pp. 922-925, 2004, },
volume={03},
pages={922-925},
year={2004},
publisher={IEEE}
}
modules/face/include/opencv2/face.hpp
+386
View File
@@ -0,0 +1,386 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#ifndef __OPENCV_FACE_HPP__
#define __OPENCV_FACE_HPP__
/**
@defgroup face Face Analysis
- @ref face_changelog
- @ref tutorial_face_main
*/
#include "opencv2/core.hpp"
#include "face/predict_collector.hpp"
#include <map>
namespace cv { namespace face {
//! @addtogroup face
//! @{
/** @brief Abstract base class for all face recognition models
All face recognition models in OpenCV are derived from the abstract base class FaceRecognizer, which
provides a unified access to all face recongition algorithms in OpenCV.
### Description
I'll go a bit more into detail explaining FaceRecognizer, because it doesn't look like a powerful
interface at first sight. But: Every FaceRecognizer is an Algorithm, so you can easily get/set all
model internals (if allowed by the implementation). Algorithm is a relatively new OpenCV concept,
which is available since the 2.4 release. I suggest you take a look at its description.
Algorithm provides the following features for all derived classes:
- So called "virtual constructor". That is, each Algorithm derivative is registered at program
start and you can get the list of registered algorithms and create instance of a particular
algorithm by its name (see Algorithm::create). If you plan to add your own algorithms, it is
good practice to add a unique prefix to your algorithms to distinguish them from other
algorithms.
- Setting/Retrieving algorithm parameters by name. If you used video capturing functionality from
OpenCV highgui module, you are probably familar with cv::cvSetCaptureProperty,
ocvcvGetCaptureProperty, VideoCapture::set and VideoCapture::get. Algorithm provides similar
method where instead of integer id's you specify the parameter names as text Strings. See
Algorithm::set and Algorithm::get for details.
- Reading and writing parameters from/to XML or YAML files. Every Algorithm derivative can store
all its parameters and then read them back. There is no need to re-implement it each time.
Moreover every FaceRecognizer supports the:
- **Training** of a FaceRecognizer with FaceRecognizer::train on a given set of images (your face
database!).
- **Prediction** of a given sample image, that means a face. The image is given as a Mat.
- **Loading/Saving** the model state from/to a given XML or YAML.
- **Setting/Getting labels info**, that is stored as a string. String labels info is useful for
keeping names of the recognized people.
@note When using the FaceRecognizer interface in combination with Python, please stick to Python 2.
Some underlying scripts like create_csv will not work in other versions, like Python 3. Setting the
Thresholds +++++++++++++++++++++++
Sometimes you run into the situation, when you want to apply a threshold on the prediction. A common
scenario in face recognition is to tell, whether a face belongs to the training dataset or if it is
unknown. You might wonder, why there's no public API in FaceRecognizer to set the threshold for the
prediction, but rest assured: It's supported. It just means there's no generic way in an abstract
class to provide an interface for setting/getting the thresholds of *every possible* FaceRecognizer
algorithm. The appropriate place to set the thresholds is in the constructor of the specific
FaceRecognizer and since every FaceRecognizer is a Algorithm (see above), you can get/set the
thresholds at runtime!
Here is an example of setting a threshold for the Eigenfaces method, when creating the model:
@code
// Let's say we want to keep 10 Eigenfaces and have a threshold value of 10.0
int num_components = 10;
double threshold = 10.0;
// Then if you want to have a cv::FaceRecognizer with a confidence threshold,
// create the concrete implementation with the appropriate parameters:
Ptr<FaceRecognizer> model = EigenFaceRecognizer::create(num_components, threshold);
@endcode
Sometimes it's impossible to train the model, just to experiment with threshold values. Thanks to
Algorithm it's possible to set internal model thresholds during runtime. Let's see how we would
set/get the prediction for the Eigenface model, we've created above:
@code
// The following line reads the threshold from the Eigenfaces model:
double current_threshold = model->getDouble("threshold");
// And this line sets the threshold to 0.0:
model->set("threshold", 0.0);
@endcode
If you've set the threshold to 0.0 as we did above, then:
@code
//
Mat img = imread("person1/3.jpg", IMREAD_GRAYSCALE);
// Get a prediction from the model. Note: We've set a threshold of 0.0 above,
// since the distance is almost always larger than 0.0, you'll get -1 as
// label, which indicates, this face is unknown
int predicted_label = model->predict(img);
// ...
@endcode
is going to yield -1 as predicted label, which states this face is unknown.
### Getting the name of a FaceRecognizer
Since every FaceRecognizer is a Algorithm, you can use Algorithm::name to get the name of a
FaceRecognizer:
@code
// Create a FaceRecognizer:
Ptr<FaceRecognizer> model = EigenFaceRecognizer::create();
// And here's how to get its name:
String name = model->name();
@endcode
*/
class CV_EXPORTS_W FaceRecognizer : public Algorithm
{
public:
/** @brief Trains a FaceRecognizer with given data and associated labels.
@param src The training images, that means the faces you want to learn. The data has to be
given as a vector\<Mat\>.
@param labels The labels corresponding to the images have to be given either as a vector\<int\>
or a Mat of type CV_32SC1.
The following source code snippet shows you how to learn a Fisherfaces model on a given set of
images. The images are read with imread and pushed into a std::vector\<Mat\>. The labels of each
image are stored within a std::vector\<int\> (you could also use a Mat of type CV_32SC1). Think of
the label as the subject (the person) this image belongs to, so same subjects (persons) should have
the same label. For the available FaceRecognizer you don't have to pay any attention to the order of
the labels, just make sure same persons have the same label:
@code
// holds images and labels
vector<Mat> images;
vector<int> labels;
// using Mat of type CV_32SC1
// Mat labels(number_of_samples, 1, CV_32SC1);
// images for first person
images.push_back(imread("person0/0.jpg", IMREAD_GRAYSCALE)); labels.push_back(0);
images.push_back(imread("person0/1.jpg", IMREAD_GRAYSCALE)); labels.push_back(0);
images.push_back(imread("person0/2.jpg", IMREAD_GRAYSCALE)); labels.push_back(0);
// images for second person
images.push_back(imread("person1/0.jpg", IMREAD_GRAYSCALE)); labels.push_back(1);
images.push_back(imread("person1/1.jpg", IMREAD_GRAYSCALE)); labels.push_back(1);
images.push_back(imread("person1/2.jpg", IMREAD_GRAYSCALE)); labels.push_back(1);
@endcode
Now that you have read some images, we can create a new FaceRecognizer. In this example I'll create
a Fisherfaces model and decide to keep all of the possible Fisherfaces:
@code
// Create a new Fisherfaces model and retain all available Fisherfaces,
// this is the most common usage of this specific FaceRecognizer:
//
Ptr<FaceRecognizer> model = FisherFaceRecognizer::create();
@endcode
And finally train it on the given dataset (the face images and labels):
@code
// This is the common interface to train all of the available cv::FaceRecognizer
// implementations:
//
model->train(images, labels);
@endcode
*/
CV_WRAP virtual void train(InputArrayOfArrays src, InputArray labels) = 0;
/** @brief Updates a FaceRecognizer with given data and associated labels.
@param src The training images, that means the faces you want to learn. The data has to be given
as a vector\<Mat\>.
@param labels The labels corresponding to the images have to be given either as a vector\<int\> or
a Mat of type CV_32SC1.
This method updates a (probably trained) FaceRecognizer, but only if the algorithm supports it. The
Local Binary Patterns Histograms (LBPH) recognizer (see createLBPHFaceRecognizer) can be updated.
For the Eigenfaces and Fisherfaces method, this is algorithmically not possible and you have to
re-estimate the model with FaceRecognizer::train. In any case, a call to train empties the existing
model and learns a new model, while update does not delete any model data.
@code
// Create a new LBPH model (it can be updated) and use the default parameters,
// this is the most common usage of this specific FaceRecognizer:
//
Ptr<FaceRecognizer> model = LBPHFaceRecognizer::create();
// This is the common interface to train all of the available cv::FaceRecognizer
// implementations:
//
model->train(images, labels);
// Some containers to hold new image:
vector<Mat> newImages;
vector<int> newLabels;
// You should add some images to the containers:
//
// ...
//
// Now updating the model is as easy as calling:
model->update(newImages,newLabels);
// This will preserve the old model data and extend the existing model
// with the new features extracted from newImages!
@endcode
Calling update on an Eigenfaces model (see EigenFaceRecognizer::create), which doesn't support
updating, will throw an error similar to:
@code
OpenCV Error: The function/feature is not implemented (This FaceRecognizer (FaceRecognizer.Eigenfaces) does not support updating, you have to use FaceRecognizer::train to update it.) in update, file /home/philipp/git/opencv/modules/contrib/src/facerec.cpp, line 305
terminate called after throwing an instance of 'cv::Exception'
@endcode
@note The FaceRecognizer does not store your training images, because this would be very
memory intense and it's not the responsibility of te FaceRecognizer to do so. The caller is
responsible for maintaining the dataset, he want to work with.
*/
CV_WRAP virtual void update(InputArrayOfArrays src, InputArray labels);
/** @overload */
CV_WRAP_AS(predict_label) int predict(InputArray src) const;
/** @brief Predicts a label and associated confidence (e.g. distance) for a given input image.
@param src Sample image to get a prediction from.
@param label The predicted label for the given image.
@param confidence Associated confidence (e.g. distance) for the predicted label.
The suffix const means that prediction does not affect the internal model state, so the method can
be safely called from within different threads.
The following example shows how to get a prediction from a trained model:
@code
using namespace cv;
// Do your initialization here (create the cv::FaceRecognizer model) ...
// ...
// Read in a sample image:
Mat img = imread("person1/3.jpg", IMREAD_GRAYSCALE);
// And get a prediction from the cv::FaceRecognizer:
int predicted = model->predict(img);
@endcode
Or to get a prediction and the associated confidence (e.g. distance):
@code
using namespace cv;
// Do your initialization here (create the cv::FaceRecognizer model) ...
// ...
Mat img = imread("person1/3.jpg", IMREAD_GRAYSCALE);
// Some variables for the predicted label and associated confidence (e.g. distance):
int predicted_label = -1;
double predicted_confidence = 0.0;
// Get the prediction and associated confidence from the model
model->predict(img, predicted_label, predicted_confidence);
@endcode
*/
CV_WRAP void predict(InputArray src, CV_OUT int &label, CV_OUT double &confidence) const;
/** @brief - if implemented - send all result of prediction to collector that can be used for somehow custom result handling
@param src Sample image to get a prediction from.
@param collector User-defined collector object that accepts all results
To implement this method u just have to do same internal cycle as in predict(InputArray src, CV_OUT int &label, CV_OUT double &confidence) but
not try to get "best@ result, just resend it to caller side with given collector
*/
CV_WRAP_AS(predict_collect) virtual void predict(InputArray src, Ptr<PredictCollector> collector) const = 0;
/** @brief Saves a FaceRecognizer and its model state.
Saves this model to a given filename, either as XML or YAML.
@param filename The filename to store this FaceRecognizer to (either XML/YAML).
Every FaceRecognizer overwrites FaceRecognizer::save(FileStorage& fs) to save the internal model
state. FaceRecognizer::save(const String& filename) saves the state of a model to the given
filename.
The suffix const means that prediction does not affect the internal model state, so the method can
be safely called from within different threads.
*/
CV_WRAP virtual void write(const String& filename) const;
/** @brief Loads a FaceRecognizer and its model state.
Loads a persisted model and state from a given XML or YAML file . Every FaceRecognizer has to
overwrite FaceRecognizer::load(FileStorage& fs) to enable loading the model state.
FaceRecognizer::load(FileStorage& fs) in turn gets called by
FaceRecognizer::load(const String& filename), to ease saving a model.
*/
CV_WRAP virtual void read(const String& filename);
/** @overload
Saves this model to a given FileStorage.
@param fs The FileStorage to store this FaceRecognizer to.
*/
virtual void write(FileStorage& fs) const CV_OVERRIDE = 0;
/** @overload */
virtual void read(const FileNode& fn) CV_OVERRIDE = 0;
/** @overload */
virtual bool empty() const CV_OVERRIDE = 0;
/** @brief Sets string info for the specified model's label.
The string info is replaced by the provided value if it was set before for the specified label.
*/
CV_WRAP virtual void setLabelInfo(int label, const String& strInfo);
/** @brief Gets string information by label.
If an unknown label id is provided or there is no label information associated with the specified
label id the method returns an empty string.
*/
CV_WRAP virtual String getLabelInfo(int label) const;
/** @brief Gets vector of labels by string.
The function searches for the labels containing the specified sub-string in the associated string
info.
*/
CV_WRAP virtual std::vector<int> getLabelsByString(const String& str) const;
/** @brief threshold parameter accessor - required for default BestMinDist collector */
virtual double getThreshold() const = 0;
/** @brief Sets threshold of model */
virtual void setThreshold(double val) = 0;
protected:
// Stored pairs "label id - string info"
std::map<int, String> _labelsInfo;
};
//! @}
}}
#include "opencv2/face/facerec.hpp"
#include "opencv2/face/facemark.hpp"
#include "opencv2/face/facemark_train.hpp"
#include "opencv2/face/facemarkLBF.hpp"
#include "opencv2/face/facemarkAAM.hpp"
#include "opencv2/face/face_alignment.hpp"
#include "opencv2/face/mace.hpp"
#endif // __OPENCV_FACE_HPP__
modules/face/include/opencv2/face/bif.hpp
+83
View File
@@ -0,0 +1,83 @@
/*
By downloading, copying, installing or using the software you agree to this license.
If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2000-2015, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2015, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015, OpenCV Foundation, all rights reserved.
Copyright (C) 2015, Itseez Inc., all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are disclaimed.
In no event shall copyright holders or contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#ifndef __OPENCV_BIF_HPP__
#define __OPENCV_BIF_HPP__
#include "opencv2/core.hpp"
namespace cv {
namespace face {
/** Implementation of bio-inspired features (BIF) from the paper:
* Guo, Guodong, et al. "Human age estimation using bio-inspired features."
* Computer Vision and Pattern Recognition, 2009. CVPR 2009.
*/
class CV_EXPORTS_W BIF : public Algorithm {
public:
/** @returns The number of filter bands used for computing BIF. */
CV_WRAP virtual int getNumBands() const = 0;
/** @returns The number of image rotations. */
CV_WRAP virtual int getNumRotations() const = 0;
/** Computes features sby input image.
* @param image Input image (CV_32FC1).
* @param features Feature vector (CV_32FC1).
*/
CV_WRAP virtual void compute(InputArray image,
OutputArray features) const = 0;
/**
* @param num_bands The number of filter bands (<=8) used for computing BIF.
* @param num_rotations The number of image rotations for computing BIF.
* @returns Object for computing BIF.
*/
CV_WRAP static Ptr<BIF> create(int num_bands = 8, int num_rotations = 12);
};
} // namespace cv
} // namespace face
#endif // #ifndef __OPENCV_FACEREC_HPP__
modules/face/include/opencv2/face/face_alignment.hpp
+60
View File
@@ -0,0 +1,60 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_FACE_ALIGNMENT_HPP__
#define __OPENCV_FACE_ALIGNMENT_HPP__
#include "opencv2/face/facemark_train.hpp"
namespace cv{
namespace face{
class CV_EXPORTS_W FacemarkKazemi : public Facemark
{
public:
struct CV_EXPORTS Params
{
/**
* \brief Constructor
*/
Params();
/// cascade_depth This stores the deapth of cascade used for training.
unsigned long cascade_depth;
/// tree_depth This stores the max height of the regression tree built.
unsigned long tree_depth;
/// num_trees_per_cascade_level This stores number of trees fit per cascade level.
unsigned long num_trees_per_cascade_level;
/// learning_rate stores the learning rate in gradient boosting, also referred as shrinkage.
float learning_rate;
/// oversampling_amount stores number of initialisations used to create training samples.
unsigned long oversampling_amount;
/// num_test_coordinates stores number of test coordinates.
unsigned long num_test_coordinates;
/// lambda stores a value to calculate probability of closeness of two coordinates.
float lambda;
/// num_test_splits stores number of random test splits generated.
unsigned long num_test_splits;
/// configfile stores the name of the file containing the values of training parameters
String configfile;
};
static Ptr<FacemarkKazemi> create(const FacemarkKazemi::Params &parameters = FacemarkKazemi::Params());
virtual ~FacemarkKazemi();
/** @brief This function is used to train the model using gradient boosting to get a cascade of regressors
*which can then be used to predict shape.
*@param images A vector of type cv::Mat which stores the images which are used in training samples.
*@param landmarks A vector of vectors of type cv::Point2f which stores the landmarks detected in a particular image.
*@param scale A size of type cv::Size to which all images and landmarks have to be scaled to.
*@param configfile A variable of type std::string which stores the name of the file storing parameters for training the model.
*@param modelFilename A variable of type std::string which stores the name of the trained model file that has to be saved.
*@returns A boolean value. The function returns true if the model is trained properly or false if it is not trained.
*/
virtual bool training(std::vector<Mat>& images, std::vector< std::vector<Point2f> >& landmarks,std::string configfile,Size scale,std::string modelFilename = "face_landmarks.dat")=0;
/// set the custom face detector
virtual bool setFaceDetector(bool(*f)(InputArray , OutputArray, void*), void* userData)=0;
/// get faces using the custom detector
virtual bool getFaces(InputArray image, OutputArray faces)=0;
};
}} // namespace
#endif
modules/face/include/opencv2/face/facemark.hpp
+92
View File
@@ -0,0 +1,92 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#ifndef __OPENCV_FACELANDMARK_HPP__
#define __OPENCV_FACELANDMARK_HPP__
#include "opencv2/core.hpp"
#include <vector>
namespace cv {
namespace face {
//! @addtogroup face
//! @{
/** @brief Abstract base class for all facemark models
To utilize this API in your program, please take a look at the @ref tutorial_table_of_content_facemark
### Description
Facemark is a base class which provides universal access to any specific facemark algorithm.
Therefore, the users should declare a desired algorithm before they can use it in their application.
Here is an example on how to declare a facemark algorithm:
@code
// Using Facemark in your code:
Ptr<Facemark> facemark = createFacemarkLBF();
@endcode
The typical pipeline for facemark detection is as follows:
- Load the trained model using Facemark::loadModel.
- Perform the fitting on an image via Facemark::fit.
*/
class CV_EXPORTS_W Facemark : public virtual Algorithm
{
public:
/** @brief A function to load the trained model before the fitting process.
@param model A string represent the filename of a trained model.
<B>Example of usage</B>
@code
facemark->loadModel("../data/lbf.model");
@endcode
*/
CV_WRAP virtual void loadModel( String model ) = 0;
// virtual void saveModel(String fs)=0;
/** @brief Detect facial landmarks from an image.
@param image Input image.
@param faces Output of the function which represent region of interest of the detected faces.
Each face is stored in cv::Rect container.
@param landmarks The detected landmark points for each faces.
<B>Example of usage</B>
@code
Mat image = imread("image.jpg");
std::vector<Rect> faces;
std::vector<std::vector<Point2f> > landmarks;
facemark->fit(image, faces, landmarks);
@endcode
*/
CV_WRAP virtual bool fit( InputArray image,
InputArray faces,
OutputArrayOfArrays landmarks) = 0;
}; /* Facemark*/
//! construct an AAM facemark detector
CV_EXPORTS_W Ptr<Facemark> createFacemarkAAM();
//! construct an LBF facemark detector
CV_EXPORTS_W Ptr<Facemark> createFacemarkLBF();
//! construct a Kazemi facemark detector
CV_EXPORTS_W Ptr<Facemark> createFacemarkKazemi();
//! @}
} // face
} // cv
#endif //__OPENCV_FACELANDMARK_HPP__
modules/face/include/opencv2/face/facemarkAAM.hpp
+162
View File
@@ -0,0 +1,162 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#ifndef __OPENCV_FACEMARK_AAM_HPP__
#define __OPENCV_FACEMARK_AAM_HPP__
#include "opencv2/face/facemark_train.hpp"
namespace cv {
namespace face {
//! @addtogroup face
//! @{
class CV_EXPORTS_W FacemarkAAM : public FacemarkTrain
{
public:
struct CV_EXPORTS Params
{
/**
* \brief Constructor
*/
Params();
/**
* \brief Read parameters from file, currently unused
*/
void read(const FileNode& /*fn*/);
/**
* \brief Read parameters from file, currently unused
*/
void write(FileStorage& /*fs*/) const;
std::string model_filename;
int m;
int n;
int n_iter;
bool verbose;
bool save_model;
int max_m, max_n, texture_max_m;
std::vector<float>scales;
};
/**
* \brief Optional parameter for fitting process.
*/
struct CV_EXPORTS Config
{
Config( Mat rot = Mat::eye(2,2,CV_32F),
Point2f trans = Point2f(0.0f, 0.0f),
float scaling = 1.0f,
int scale_id=0
);
Mat R;
Point2f t;
float scale;
int model_scale_idx;
};
/**
* \brief Data container for the facemark::getData function
*/
struct CV_EXPORTS Data
{
std::vector<Point2f> s0;
};
/**
* \brief The model of AAM Algorithm
*/
struct CV_EXPORTS Model
{
std::vector<float>scales;
//!< defines the scales considered to build the model
/*warping*/
std::vector<Vec3i> triangles;
//!< each element contains 3 values, represent index of facemarks that construct one triangle (obtained using delaunay triangulation)
struct Texture{
int max_m; //!< unused delete
Rect resolution;
//!< resolution of the current scale
Mat A;
//!< gray values from all face region in the dataset, projected in PCA space
Mat A0;
//!< average of gray values from all face region in the dataset
Mat AA;
//!< gray values from all erorded face region in the dataset, projected in PCA space
Mat AA0;
//!< average of gray values from all erorded face region in the dataset
std::vector<std::vector<Point> > textureIdx;
//!< index for warping of each delaunay triangle region constructed by 3 facemarks
std::vector<Point2f> base_shape;
//!< basic shape, normalized to be fit in an image with current detection resolution
std::vector<int> ind1;
//!< index of pixels for mapping process to obtains the grays values of face region
std::vector<int> ind2;
//!< index of pixels for mapping process to obtains the grays values of eroded face region
};
std::vector<Texture> textures;
//!< a container to holds the texture data for each scale of fitting
/*shape*/
std::vector<Point2f> s0;
//!< the basic shape obtained from training dataset
Mat S,Q;
//!< the encoded shapes from training data
};
//! overload with additional Config structures
virtual bool fitConfig( InputArray image, InputArray roi, OutputArrayOfArrays _landmarks, const std::vector<Config> &runtime_params ) = 0;
//! initializer
static Ptr<FacemarkAAM> create(const FacemarkAAM::Params &parameters = FacemarkAAM::Params() );
virtual ~FacemarkAAM() {}
}; /* AAM */
//! @}
} /* namespace face */
} /* namespace cv */
#endif
modules/face/include/opencv2/face/facemarkLBF.hpp
+120
View File
@@ -0,0 +1,120 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#ifndef __OPENCV_FACEMARK_LBF_HPP__
#define __OPENCV_FACEMARK_LBF_HPP__
#include "opencv2/face/facemark_train.hpp"
namespace cv {
namespace face {
//! @addtogroup face
//! @{
class CV_EXPORTS_W FacemarkLBF : public FacemarkTrain
{
public:
struct CV_EXPORTS Params
{
/**
* \brief Constructor
*/
Params();
double shape_offset;
//!< offset for the loaded face landmark points
String cascade_face;
//!< filename of the face detector model
bool verbose;
//!< show the training print-out
int n_landmarks;
//!< number of landmark points
int initShape_n;
//!< multiplier for augment the training data
int stages_n;
//!< number of refinement stages
int tree_n;
//!< number of tree in the model for each landmark point refinement
int tree_depth;
//!< the depth of decision tree, defines the size of feature
double bagging_overlap;
//!< overlap ratio for training the LBF feature
std::string model_filename;
//!< filename where the trained model will be saved
bool save_model; //!< flag to save the trained model or not
unsigned int seed; //!< seed for shuffling the training data
std::vector<int> feats_m;
std::vector<double> radius_m;
std::vector<int> pupils[2];
//!< index of facemark points on pupils of left and right eye
Rect detectROI;
void read(const FileNode& /*fn*/);
void write(FileStorage& /*fs*/) const;
};
class BBox {
public:
BBox();
~BBox();
BBox(double x, double y, double w, double h);
Mat project(const Mat &shape) const;
Mat reproject(const Mat &shape) const;
double x, y;
double x_center, y_center;
double x_scale, y_scale;
double width, height;
};
static Ptr<FacemarkLBF> create(const FacemarkLBF::Params &parameters = FacemarkLBF::Params() );
virtual ~FacemarkLBF(){};
}; /* LBF */
//! @}
} /* namespace face */
}/* namespace cv */
#endif
modules/face/include/opencv2/face/facemark_train.hpp
+386
View File
@@ -0,0 +1,386 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#ifndef __OPENCV_FACELANDMARKTRAIN_HPP__
#define __OPENCV_FACELANDMARKTRAIN_HPP__
#include "opencv2/face/facemark.hpp"
#include "opencv2/xobjdetect.hpp"
#include <vector>
#include <string>
namespace cv {
namespace face {
//! @addtogroup face
//! @{
typedef bool(*FN_FaceDetector)(InputArray, OutputArray, void* userData);
struct CParams{
String cascade; //!< the face detector
double scaleFactor; //!< Parameter specifying how much the image size is reduced at each image scale.
int minNeighbors; //!< Parameter specifying how many neighbors each candidate rectangle should have to retain it.
Size minSize; //!< Minimum possible object size.
Size maxSize; //!< Maximum possible object size.
CV_EXPORTS CParams(
String cascade_model,
double sf = 1.1,
int minN = 3,
Size minSz = Size(30, 30),
Size maxSz = Size()
);
CascadeClassifier face_cascade;
};
/** @brief Default face detector
This function is mainly utilized by the implementation of a Facemark Algorithm.
End users are advised to use function Facemark::getFaces which can be manually defined
and circumvented to the algorithm by Facemark::setFaceDetector.
@param image The input image to be processed.
@param faces Output of the function which represent region of interest of the detected faces.
Each face is stored in cv::Rect container.
@param params detector parameters
<B>Example of usage</B>
@code
std::vector<cv::Rect> faces;
CParams params("haarcascade_frontalface_alt.xml");
cv::face::getFaces(frame, faces, &params);
for(int j=0;j<faces.size();j++){
cv::rectangle(frame, faces[j], cv::Scalar(255,0,255));
}
cv::imshow("detection", frame);
@endcode
*/
CV_EXPORTS bool getFaces(InputArray image, OutputArray faces, CParams* params);
CV_EXPORTS_W bool getFacesHAAR(InputArray image, OutputArray faces, const String& face_cascade_name);
/** @brief A utility to load list of paths to training image and annotation file.
@param imageList The specified file contains paths to the training images.
@param annotationList The specified file contains paths to the training annotations.
@param images The loaded paths of training images.
@param annotations The loaded paths of annotation files.
Example of usage:
@code
String imageFiles = "images_path.txt";
String ptsFiles = "annotations_path.txt";
std::vector<String> images_train;
std::vector<String> landmarks_train;
loadDatasetList(imageFiles,ptsFiles,images_train,landmarks_train);
@endcode
*/
CV_EXPORTS_W bool loadDatasetList(String imageList,
String annotationList,
std::vector<String> & images,
std::vector<String> & annotations);
/** @brief A utility to load facial landmark dataset from a single file.
@param filename The filename of a file that contains the dataset information.
Each line contains the filename of an image followed by
pairs of x and y values of facial landmarks points separated by a space.
Example
@code
/home/user/ibug/image_003_1.jpg 336.820955 240.864510 334.238298 260.922709 335.266918 ...
/home/user/ibug/image_005_1.jpg 376.158428 230.845712 376.736984 254.924635 383.265403 ...
@endcode
@param images A vector where each element represent the filename of image in the dataset.
Images are not loaded by default to save the memory.
@param facePoints The loaded landmark points for all training data.
@param delim Delimiter between each element, the default value is a whitespace.
@param offset An offset value to adjust the loaded points.
<B>Example of usage</B>
@code
cv::String imageFiles = "../data/images_train.txt";
cv::String ptsFiles = "../data/points_train.txt";
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imageFiles, ptsFiles, images, facePoints, 0.0f);
@endcode
*/
CV_EXPORTS_W bool loadTrainingData( String filename , std::vector<String> & images,
OutputArray facePoints,
char delim = ' ', float offset = 0.0f);
/** @brief A utility to load facial landmark information from the dataset.
@param imageList A file contains the list of image filenames in the training dataset.
@param groundTruth A file contains the list of filenames
where the landmarks points information are stored.
The content in each file should follow the standard format (see face::loadFacePoints).
@param images A vector where each element represent the filename of image in the dataset.
Images are not loaded by default to save the memory.
@param facePoints The loaded landmark points for all training data.
@param offset An offset value to adjust the loaded points.
<B>Example of usage</B>
@code
cv::String imageFiles = "../data/images_train.txt";
cv::String ptsFiles = "../data/points_train.txt";
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imageFiles, ptsFiles, images, facePoints, 0.0f);
@endcode
example of content in the images_train.txt
@code
/home/user/ibug/image_003_1.jpg
/home/user/ibug/image_004_1.jpg
/home/user/ibug/image_005_1.jpg
/home/user/ibug/image_006.jpg
@endcode
example of content in the points_train.txt
@code
/home/user/ibug/image_003_1.pts
/home/user/ibug/image_004_1.pts
/home/user/ibug/image_005_1.pts
/home/user/ibug/image_006.pts
@endcode
*/
CV_EXPORTS_W bool loadTrainingData( String imageList, String groundTruth,
std::vector<String> & images,
OutputArray facePoints,
float offset = 0.0f);
/** @brief This function extracts the data for training from .txt files which contains the corresponding image name and landmarks.
*The first file in each file should give the path of the image whose
*landmarks are being described in the file. Then in the subsequent
*lines there should be coordinates of the landmarks in the image
*i.e each line should be of the form x,y
*where x represents the x coordinate of the landmark and y represents
*the y coordinate of the landmark.
*
*For reference you can see the files as provided in the
*<a href="http://www.ifp.illinois.edu/~vuongle2/helen/">HELEN dataset</a>
*
* @param filename A vector of type cv::String containing name of the .txt files.
* @param trainlandmarks A vector of type cv::Point2f that would store shape or landmarks of all images.
* @param trainimages A vector of type cv::String which stores the name of images whose landmarks are tracked
* @returns A boolean value. It returns true when it reads the data successfully and false otherwise
*/
CV_EXPORTS_W bool loadTrainingData(std::vector<String> filename,std::vector< std::vector<Point2f> >
&trainlandmarks,std::vector<String> & trainimages);
/** @brief A utility to load facial landmark information from a given file.
@param filename The filename of file contains the facial landmarks data.
@param points The loaded facial landmark points.
@param offset An offset value to adjust the loaded points.
<B>Example of usage</B>
@code
std::vector<Point2f> points;
face::loadFacePoints("filename.txt", points, 0.0f);
@endcode
The annotation file should follow the default format which is
@code
version: 1
n_points: 68
{
212.716603 499.771793
230.232816 566.290071
...
}
@endcode
where n_points is the number of points considered
and each point is represented as its position in x and y.
*/
CV_EXPORTS_W bool loadFacePoints( String filename, OutputArray points,
float offset = 0.0f);
/** @brief Utility to draw the detected facial landmark points
@param image The input image to be processed.
@param points Contains the data of points which will be drawn.
@param color The color of points in BGR format represented by cv::Scalar.
<B>Example of usage</B>
@code
std::vector<Rect> faces;
std::vector<std::vector<Point2f> > landmarks;
facemark->getFaces(img, faces);
facemark->fit(img, faces, landmarks);
for(int j=0;j<rects.size();j++){
face::drawFacemarks(frame, landmarks[j], Scalar(0,0,255));
}
@endcode
*/
CV_EXPORTS_W void drawFacemarks( InputOutputArray image, InputArray points,
Scalar color = Scalar(255,0,0));
/** @brief Abstract base class for trainable facemark models
To utilize this API in your program, please take a look at the @ref tutorial_table_of_content_facemark
### Description
The AAM and LBF facemark models in OpenCV are derived from the abstract base class FacemarkTrain, which
provides a unified access to those facemark algorithms in OpenCV.
Here is an example on how to declare facemark algorithm:
@code
// Using Facemark in your code:
Ptr<Facemark> facemark = FacemarkLBF::create();
@endcode
The typical pipeline for facemark detection is listed as follows:
- (Non-mandatory) Set a user defined face detection using FacemarkTrain::setFaceDetector.
The facemark algorithms are designed to fit the facial points into a face.
Therefore, the face information should be provided to the facemark algorithm.
Some algorithms might provides a default face recognition function.
However, the users might prefer to use their own face detector to obtains the best possible detection result.
- (Non-mandatory) Training the model for a specific algorithm using FacemarkTrain::training.
In this case, the model should be automatically saved by the algorithm.
If the user already have a trained model, then this part can be omitted.
- Load the trained model using Facemark::loadModel.
- Perform the fitting via the Facemark::fit.
*/
class CV_EXPORTS_W FacemarkTrain : public Facemark
{
public:
/** @brief Add one training sample to the trainer.
@param image Input image.
@param landmarks The ground-truth of facial landmarks points corresponds to the image.
<B>Example of usage</B>
@code
String imageFiles = "../data/images_train.txt";
String ptsFiles = "../data/points_train.txt";
std::vector<String> images_train;
std::vector<String> landmarks_train;
// load the list of dataset: image paths and landmark file paths
loadDatasetList(imageFiles,ptsFiles,images_train,landmarks_train);
Mat image;
std::vector<Point2f> facial_points;
for(size_t i=0;i<images_train.size();i++){
image = imread(images_train[i].c_str());
loadFacePoints(landmarks_train[i],facial_points);
facemark->addTrainingSample(image, facial_points);
}
@endcode
The contents in the training files should follows the standard format.
Here are examples for the contents in these files.
example of content in the images_train.txt
@code
/home/user/ibug/image_003_1.jpg
/home/user/ibug/image_004_1.jpg
/home/user/ibug/image_005_1.jpg
/home/user/ibug/image_006.jpg
@endcode
example of content in the points_train.txt
@code
/home/user/ibug/image_003_1.pts
/home/user/ibug/image_004_1.pts
/home/user/ibug/image_005_1.pts
/home/user/ibug/image_006.pts
@endcode
*/
virtual bool addTrainingSample(InputArray image, InputArray landmarks)=0;
/** @brief Trains a Facemark algorithm using the given dataset.
Before the training process, training samples should be added to the trainer
using face::addTrainingSample function.
@param parameters Optional extra parameters (algorithm dependent).
<B>Example of usage</B>
@code
FacemarkLBF::Params params;
params.model_filename = "ibug68.model"; // filename to save the trained model
Ptr<Facemark> facemark = FacemarkLBF::create(params);
// add training samples (see Facemark::addTrainingSample)
facemark->training();
@endcode
*/
virtual void training(void* parameters=0)=0;
/** @brief Set a user defined face detector for the Facemark algorithm.
@param detector The user defined face detector function
@param userData Detector parameters
<B>Example of usage</B>
@code
MyDetectorParameters detectorParameters(...);
facemark->setFaceDetector(myDetector, &detectorParameters);
@endcode
Example of a user defined face detector
@code
bool myDetector( InputArray image, OutputArray faces, void* userData)
{
MyDetectorParameters* params = (MyDetectorParameters*)userData;
// -------- do something --------
}
@endcode
TODO Lifetime of detector parameters is uncontrolled. Rework interface design to "Ptr<FaceDetector>".
*/
virtual bool setFaceDetector(FN_FaceDetector detector, void* userData = 0)=0;
/** @brief Detect faces from a given image using default or user defined face detector.
Some Algorithm might not provide a default face detector.
@param image Input image.
@param faces Output of the function which represent region of interest of the detected faces. Each face is stored in cv::Rect container.
<B>Example of usage</B>
@code
std::vector<cv::Rect> faces;
facemark->getFaces(img, faces);
for(int j=0;j<faces.size();j++){
cv::rectangle(img, faces[j], cv::Scalar(255,0,255));
}
@endcode
*/
virtual bool getFaces(InputArray image, OutputArray faces)=0;
/** @brief Get data from an algorithm
@param items The obtained data, algorithm dependent.
<B>Example of usage</B>
@code
Ptr<FacemarkAAM> facemark = FacemarkAAM::create();
facemark->loadModel("AAM.yml");
FacemarkAAM::Data data;
facemark->getData(&data);
std::vector<Point2f> s0 = data.s0;
cout<<s0<<endl;
@endcode
*/
virtual bool getData(void * items=0)=0; // FIXIT
}; /* Facemark*/
//! @}
} /* namespace face */
} /* namespace cv */
#endif //__OPENCV_FACELANDMARKTRAIN_HPP__
modules/face/include/opencv2/face/facerec.hpp
+191
View File
@@ -0,0 +1,191 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
// Copyright (c) 2011,2012. Philipp Wagner <bytefish[at]gmx[dot]de>.
// Third party copyrights are property of their respective owners.
#ifndef __OPENCV_FACEREC_HPP__
#define __OPENCV_FACEREC_HPP__
#include "opencv2/face.hpp"
#include "opencv2/core.hpp"
namespace cv { namespace face {
//! @addtogroup face
//! @{
// base for two classes
class CV_EXPORTS_W BasicFaceRecognizer : public FaceRecognizer
{
public:
/** @see setNumComponents */
CV_WRAP int getNumComponents() const;
/** @copybrief getNumComponents @see getNumComponents */
CV_WRAP void setNumComponents(int val);
/** @see setThreshold */
CV_WRAP double getThreshold() const CV_OVERRIDE;
/** @copybrief getThreshold @see getThreshold */
CV_WRAP void setThreshold(double val) CV_OVERRIDE;
CV_WRAP std::vector<cv::Mat> getProjections() const;
CV_WRAP cv::Mat getLabels() const;
CV_WRAP cv::Mat getEigenValues() const;
CV_WRAP cv::Mat getEigenVectors() const;
CV_WRAP cv::Mat getMean() const;
virtual void read(const FileNode& fn) CV_OVERRIDE;
virtual void write(FileStorage& fs) const CV_OVERRIDE;
virtual bool empty() const CV_OVERRIDE;
using FaceRecognizer::read;
using FaceRecognizer::write;
protected:
int _num_components;
double _threshold;
std::vector<Mat> _projections;
Mat _labels;
Mat _eigenvectors;
Mat _eigenvalues;
Mat _mean;
};
class CV_EXPORTS_W EigenFaceRecognizer : public BasicFaceRecognizer
{
public:
/**
@param num_components The number of components (read: Eigenfaces) kept for this Principal
Component Analysis. As a hint: There's no rule how many components (read: Eigenfaces) should be
kept for good reconstruction capabilities. It is based on your input data, so experiment with the
number. Keeping 80 components should almost always be sufficient.
@param threshold The threshold applied in the prediction.
### Notes:
- Training and prediction must be done on grayscale images, use cvtColor to convert between the
color spaces.
- **THE EIGENFACES METHOD MAKES THE ASSUMPTION, THAT THE TRAINING AND TEST IMAGES ARE OF EQUAL
SIZE.** (caps-lock, because I got so many mails asking for this). You have to make sure your
input data has the correct shape, else a meaningful exception is thrown. Use resize to resize
the images.
- This model does not support updating.
### Model internal data:
- num_components see EigenFaceRecognizer::create.
- threshold see EigenFaceRecognizer::create.
- eigenvalues The eigenvalues for this Principal Component Analysis (ordered descending).
- eigenvectors The eigenvectors for this Principal Component Analysis (ordered by their
eigenvalue).
- mean The sample mean calculated from the training data.
- projections The projections of the training data.
- labels The threshold applied in the prediction. If the distance to the nearest neighbor is
larger than the threshold, this method returns -1.
*/
CV_WRAP static Ptr<EigenFaceRecognizer> create(int num_components = 0, double threshold = DBL_MAX);
};
class CV_EXPORTS_W FisherFaceRecognizer : public BasicFaceRecognizer
{
public:
/**
@param num_components The number of components (read: Fisherfaces) kept for this Linear
Discriminant Analysis with the Fisherfaces criterion. It's useful to keep all components, that
means the number of your classes c (read: subjects, persons you want to recognize). If you leave
this at the default (0) or set it to a value less-equal 0 or greater (c-1), it will be set to the
correct number (c-1) automatically.
@param threshold The threshold applied in the prediction. If the distance to the nearest neighbor
is larger than the threshold, this method returns -1.
### Notes:
- Training and prediction must be done on grayscale images, use cvtColor to convert between the
color spaces.
- **THE FISHERFACES METHOD MAKES THE ASSUMPTION, THAT THE TRAINING AND TEST IMAGES ARE OF EQUAL
SIZE.** (caps-lock, because I got so many mails asking for this). You have to make sure your
input data has the correct shape, else a meaningful exception is thrown. Use resize to resize
the images.
- This model does not support updating.
### Model internal data:
- num_components see FisherFaceRecognizer::create.
- threshold see FisherFaceRecognizer::create.
- eigenvalues The eigenvalues for this Linear Discriminant Analysis (ordered descending).
- eigenvectors The eigenvectors for this Linear Discriminant Analysis (ordered by their
eigenvalue).
- mean The sample mean calculated from the training data.
- projections The projections of the training data.
- labels The labels corresponding to the projections.
*/
CV_WRAP static Ptr<FisherFaceRecognizer> create(int num_components = 0, double threshold = DBL_MAX);
};
class CV_EXPORTS_W LBPHFaceRecognizer : public FaceRecognizer
{
public:
/** @see setGridX */
CV_WRAP virtual int getGridX() const = 0;
/** @copybrief getGridX @see getGridX */
CV_WRAP virtual void setGridX(int val) = 0;
/** @see setGridY */
CV_WRAP virtual int getGridY() const = 0;
/** @copybrief getGridY @see getGridY */
CV_WRAP virtual void setGridY(int val) = 0;
/** @see setRadius */
CV_WRAP virtual int getRadius() const = 0;
/** @copybrief getRadius @see getRadius */
CV_WRAP virtual void setRadius(int val) = 0;
/** @see setNeighbors */
CV_WRAP virtual int getNeighbors() const = 0;
/** @copybrief getNeighbors @see getNeighbors */
CV_WRAP virtual void setNeighbors(int val) = 0;
/** @see setThreshold */
CV_WRAP virtual double getThreshold() const CV_OVERRIDE = 0;
/** @copybrief getThreshold @see getThreshold */
CV_WRAP virtual void setThreshold(double val) CV_OVERRIDE = 0;
CV_WRAP virtual std::vector<cv::Mat> getHistograms() const = 0;
CV_WRAP virtual cv::Mat getLabels() const = 0;
/**
@param radius The radius used for building the Circular Local Binary Pattern. The greater the
radius, the smoother the image but more spatial information you can get.
@param neighbors The number of sample points to build a Circular Local Binary Pattern from. An
appropriate value is to use `8` sample points. Keep in mind: the more sample points you include,
the higher the computational cost.
@param grid_x The number of cells in the horizontal direction, 8 is a common value used in
publications. The more cells, the finer the grid, the higher the dimensionality of the resulting
feature vector.
@param grid_y The number of cells in the vertical direction, 8 is a common value used in
publications. The more cells, the finer the grid, the higher the dimensionality of the resulting
feature vector.
@param threshold The threshold applied in the prediction. If the distance to the nearest neighbor
is larger than the threshold, this method returns -1.
### Notes:
- The Circular Local Binary Patterns (used in training and prediction) expect the data given as
grayscale images, use cvtColor to convert between the color spaces.
- This model supports updating.
### Model internal data:
- radius see LBPHFaceRecognizer::create.
- neighbors see LBPHFaceRecognizer::create.
- grid_x see LLBPHFaceRecognizer::create.
- grid_y see LBPHFaceRecognizer::create.
- threshold see LBPHFaceRecognizer::create.
- histograms Local Binary Patterns Histograms calculated from the given training data (empty if
none was given).
- labels Labels corresponding to the calculated Local Binary Patterns Histograms.
*/
CV_WRAP static Ptr<LBPHFaceRecognizer> create(int radius=1, int neighbors=8, int grid_x=8, int grid_y=8, double threshold = DBL_MAX);
};
//! @}
}} //namespace cv::face
#endif //__OPENCV_FACEREC_HPP__
modules/face/include/opencv2/face/mace.hpp
+114
View File
@@ -0,0 +1,114 @@
// This file is part of the OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __mace_h_onboard__
#define __mace_h_onboard__
#include "opencv2/core.hpp"
namespace cv {
namespace face {
//! @addtogroup face
//! @{
/**
@brief Minimum Average Correlation Energy Filter
useful for authentication with (cancellable) biometrical features.
(does not need many positives to train (10-50), and no negatives at all, also robust to noise/salting)
see also: @cite Savvides04
this implementation is largely based on: https://code.google.com/archive/p/pam-face-authentication (GSOC 2009)
use it like:
@code
Ptr<face::MACE> mace = face::MACE::create(64);
vector<Mat> pos_images = ...
mace->train(pos_images);
Mat query = ...
bool same = mace->same(query);
@endcode
you can also use two-factor authentication, with an additional passphrase:
@code
String owners_passphrase = "ilikehotdogs";
Ptr<face::MACE> mace = face::MACE::create(64);
mace->salt(owners_passphrase);
vector<Mat> pos_images = ...
mace->train(pos_images);
// now, users have to give a valid passphrase, along with the image:
Mat query = ...
cout << "enter passphrase: ";
string pass;
getline(cin, pass);
mace->salt(pass);
bool same = mace->same(query);
@endcode
save/load your model:
@code
Ptr<face::MACE> mace = face::MACE::create(64);
mace->train(pos_images);
mace->save("my_mace.xml");
// later:
Ptr<MACE> reloaded = MACE::load("my_mace.xml");
reloaded->same(some_image);
@endcode
*/
class CV_EXPORTS_W MACE : public cv::Algorithm
{
public:
/**
@brief optionally encrypt images with random convolution
@param passphrase a crc64 random seed will get generated from this
*/
CV_WRAP virtual void salt(const cv::String &passphrase) = 0;
/**
@brief train it on positive features
compute the mace filter: `h = D(-1) * X * (X(+) * D(-1) * X)(-1) * C`
also calculate a minimal threshold for this class, the smallest self-similarity from the train images
@param images a vector<Mat> with the train images
*/
CV_WRAP virtual void train(cv::InputArrayOfArrays images) = 0;
/**
@brief correlate query img and threshold to min class value
@param query a Mat with query image
*/
CV_WRAP virtual bool same(cv::InputArray query) const = 0;
/**
@brief constructor
@param filename build a new MACE instance from a pre-serialized FileStorage
@param objname (optional) top-level node in the FileStorage
*/
CV_WRAP static cv::Ptr<MACE> load(const String &filename, const String &objname=String());
/**
@brief constructor
@param IMGSIZE images will get resized to this (should be an even number)
*/
CV_WRAP static cv::Ptr<MACE> create(int IMGSIZE=64);
};
//! @}
}/* namespace face */
}/* namespace cv */
#endif // __mace_h_onboard__
modules/face/include/opencv2/face/predict_collector.hpp
+127
View File
@@ -0,0 +1,127 @@
/*
By downloading, copying, installing or using the software you agree to this license.
If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2000-2015, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2015, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015, OpenCV Foundation, all rights reserved.
Copyright (C) 2015, Itseez Inc., all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are disclaimed.
In no event shall copyright holders or contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#ifndef __OPENCV_PREDICT_COLLECTOR_HPP__
#define __OPENCV_PREDICT_COLLECTOR_HPP__
#include <vector>
#include <map>
#include <utility>
#include <cfloat>
#include "opencv2/core/base.hpp"
namespace cv {
namespace face {
//! @addtogroup face
//! @{
/** @brief Abstract base class for all strategies of prediction result handling
*/
class CV_EXPORTS_W PredictCollector
{
public:
virtual ~PredictCollector() {}
/** @brief Interface method called by face recognizer before results processing
@param size total size of prediction evaluation that recognizer could perform
*/
virtual void init(size_t size) { CV_UNUSED(size); }
/** @brief Interface method called by face recognizer for each result
@param label current prediction label
@param dist current prediction distance (confidence)
*/
virtual bool collect(int label, double dist) = 0;
};
/** @brief Default predict collector
Trace minimal distance with treshhold checking (that is default behavior for most predict logic)
*/
class CV_EXPORTS_W StandardCollector : public PredictCollector
{
public:
struct PredictResult
{
int label;
double distance;
PredictResult(int label_ = -1, double distance_ = DBL_MAX) : label(label_), distance(distance_) {}
};
protected:
double threshold;
PredictResult minRes;
std::vector<PredictResult> data;
public:
/** @brief Constructor
@param threshold_ set threshold
*/
StandardCollector(double threshold_ = DBL_MAX);
/** @brief overloaded interface method */
void init(size_t size) CV_OVERRIDE;
/** @brief overloaded interface method */
bool collect(int label, double dist) CV_OVERRIDE;
/** @brief Returns label with minimal distance */
CV_WRAP int getMinLabel() const;
/** @brief Returns minimal distance value */
CV_WRAP double getMinDist() const;
/** @brief Return results as vector
@param sorted If set, results will be sorted by distance
Each values is a pair of label and distance.
*/
CV_WRAP std::vector< std::pair<int, double> > getResults(bool sorted = false) const;
/** @brief Return results as map
Labels are keys, values are minimal distances
*/
std::map<int, double> getResultsMap() const;
/** @brief Static constructor
@param threshold set threshold
*/
CV_WRAP static Ptr<StandardCollector> create(double threshold = DBL_MAX);
};
//! @}
}
}
#endif
modules/face/misc/java/gen_dict.json
+16
View File
@@ -0,0 +1,16 @@
{
"const_ignore_list": [
],
"const_private_list" : [
],
"missing_consts" : {
},
"ManualFuncs" : {
},
"func_arg_fix" : {
"fit" : {
"landmarks" : {"ctype" : "vector_vector_Point2f"},
"faces" : {"ctype" : "vector_Rect"}
}
}
}
modules/face/misc/objc/gen_dict.json
+6
View File
@@ -0,0 +1,6 @@
{
"AdditionalImports" : {
"*" : [ "\"face.hpp\"" ],
"BIF" : [ "\"face/bif.hpp\"" ]
}
}
modules/face/samples/CMakeLists.txt
+28
View File
@@ -0,0 +1,28 @@
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
set(name "facerec")
project(facerec_cpp_samples)
#SET(OpenCV_DIR /path/to/your/opencv/installation)
# packages
find_package(OpenCV REQUIRED) # http://opencv.org
# probably you should loop through the sample files here
add_executable(facerec_demo facerec_demo.cpp)
target_link_libraries(facerec_demo opencv_core opencv_face opencv_imgproc opencv_highgui)
add_executable(facerec_video facerec_video.cpp)
target_link_libraries(facerec_video opencv_face opencv_core opencv_imgproc opencv_highgui opencv_xobjdetect opencv_imgproc)
add_executable(facerec_eigenfaces facerec_eigenfaces.cpp)
target_link_libraries(facerec_eigenfaces opencv_face opencv_core opencv_imgproc opencv_highgui)
add_executable(facerec_fisherfaces facerec_fisherfaces.cpp)
target_link_libraries(facerec_fisherfaces opencv_face opencv_core opencv_imgproc opencv_highgui)
add_executable(facerec_lbph facerec_lbph.cpp)
target_link_libraries(facerec_lbph opencv_face opencv_core opencv_imgproc opencv_highgui)
add_executable(mace_webcam mace_webcam.cpp)
target_link_libraries(mace_webcam opencv_face opencv_core opencv_imgproc opencv_highgui opencv_videoio)
modules/face/samples/Facemark.java
+48
View File
@@ -0,0 +1,48 @@
import org.opencv.core.*;
import org.opencv.face.*;
import org.opencv.imgcodecs.*;
import org.opencv.imgproc.*;
import org.opencv.xobjdetect.*;
import java.util.*;
public class Facemark {
static {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
}
public static void main(String[] args) {
if (args.length < 3) {
System.out.println("use: java Facemark [image file] [cascade file] [model file]");
return;
}
// read the image
Mat img = Imgcodecs.imread(args[0]);
// setup face detection
CascadeClassifier cascade = new CascadeClassifier(args[1]);
MatOfRect faces = new MatOfRect();
// detect faces
cascade.detectMultiScale(img, faces);
// setup landmarks detector
Facemark fm = Face.createFacemarkKazemi();
fm.loadModel(args[2]);
// fit landmarks for each found face
ArrayList<MatOfPoint2f> landmarks = new ArrayList<MatOfPoint2f>();
fm.fit(img, faces, landmarks);
// draw them
for (int i=0; i<landmarks.size(); i++) {
MatOfPoint2f lm = landmarks.get(i);
for (int j=0; j<lm.rows(); j++) {
double [] dp = lm.get(j,0);
Point p = new Point(dp[0], dp[1]);
Imgproc.circle(img,p,2,new Scalar(222),1);
}
}
// save result
Imgcodecs.imwrite("landmarks.jpg",img);
}
}
modules/face/samples/etc/at.txt
+400
View File
@@ -0,0 +1,400 @@
/home/philipp/facerec/data/at/s13/2.pgm;12
/home/philipp/facerec/data/at/s13/7.pgm;12
/home/philipp/facerec/data/at/s13/6.pgm;12
/home/philipp/facerec/data/at/s13/9.pgm;12
/home/philipp/facerec/data/at/s13/5.pgm;12
/home/philipp/facerec/data/at/s13/3.pgm;12
/home/philipp/facerec/data/at/s13/4.pgm;12
/home/philipp/facerec/data/at/s13/10.pgm;12
/home/philipp/facerec/data/at/s13/8.pgm;12
/home/philipp/facerec/data/at/s13/1.pgm;12
/home/philipp/facerec/data/at/s17/2.pgm;16
/home/philipp/facerec/data/at/s17/7.pgm;16
/home/philipp/facerec/data/at/s17/6.pgm;16
/home/philipp/facerec/data/at/s17/9.pgm;16
/home/philipp/facerec/data/at/s17/5.pgm;16
/home/philipp/facerec/data/at/s17/3.pgm;16
/home/philipp/facerec/data/at/s17/4.pgm;16
/home/philipp/facerec/data/at/s17/10.pgm;16
/home/philipp/facerec/data/at/s17/8.pgm;16
/home/philipp/facerec/data/at/s17/1.pgm;16
/home/philipp/facerec/data/at/s32/2.pgm;31
/home/philipp/facerec/data/at/s32/7.pgm;31
/home/philipp/facerec/data/at/s32/6.pgm;31
/home/philipp/facerec/data/at/s32/9.pgm;31
/home/philipp/facerec/data/at/s32/5.pgm;31
/home/philipp/facerec/data/at/s32/3.pgm;31
/home/philipp/facerec/data/at/s32/4.pgm;31
/home/philipp/facerec/data/at/s32/10.pgm;31
/home/philipp/facerec/data/at/s32/8.pgm;31
/home/philipp/facerec/data/at/s32/1.pgm;31
/home/philipp/facerec/data/at/s10/2.pgm;9
/home/philipp/facerec/data/at/s10/7.pgm;9
/home/philipp/facerec/data/at/s10/6.pgm;9
/home/philipp/facerec/data/at/s10/9.pgm;9
/home/philipp/facerec/data/at/s10/5.pgm;9
/home/philipp/facerec/data/at/s10/3.pgm;9
/home/philipp/facerec/data/at/s10/4.pgm;9
/home/philipp/facerec/data/at/s10/10.pgm;9
/home/philipp/facerec/data/at/s10/8.pgm;9
/home/philipp/facerec/data/at/s10/1.pgm;9
/home/philipp/facerec/data/at/s27/2.pgm;26
/home/philipp/facerec/data/at/s27/7.pgm;26
/home/philipp/facerec/data/at/s27/6.pgm;26
/home/philipp/facerec/data/at/s27/9.pgm;26
/home/philipp/facerec/data/at/s27/5.pgm;26
/home/philipp/facerec/data/at/s27/3.pgm;26
/home/philipp/facerec/data/at/s27/4.pgm;26
/home/philipp/facerec/data/at/s27/10.pgm;26
/home/philipp/facerec/data/at/s27/8.pgm;26
/home/philipp/facerec/data/at/s27/1.pgm;26
/home/philipp/facerec/data/at/s5/2.pgm;4
/home/philipp/facerec/data/at/s5/7.pgm;4
/home/philipp/facerec/data/at/s5/6.pgm;4
/home/philipp/facerec/data/at/s5/9.pgm;4
/home/philipp/facerec/data/at/s5/5.pgm;4
/home/philipp/facerec/data/at/s5/3.pgm;4
/home/philipp/facerec/data/at/s5/4.pgm;4
/home/philipp/facerec/data/at/s5/10.pgm;4
/home/philipp/facerec/data/at/s5/8.pgm;4
/home/philipp/facerec/data/at/s5/1.pgm;4
/home/philipp/facerec/data/at/s20/2.pgm;19
/home/philipp/facerec/data/at/s20/7.pgm;19
/home/philipp/facerec/data/at/s20/6.pgm;19
/home/philipp/facerec/data/at/s20/9.pgm;19
/home/philipp/facerec/data/at/s20/5.pgm;19
/home/philipp/facerec/data/at/s20/3.pgm;19
/home/philipp/facerec/data/at/s20/4.pgm;19
/home/philipp/facerec/data/at/s20/10.pgm;19
/home/philipp/facerec/data/at/s20/8.pgm;19
/home/philipp/facerec/data/at/s20/1.pgm;19
/home/philipp/facerec/data/at/s30/2.pgm;29
/home/philipp/facerec/data/at/s30/7.pgm;29
/home/philipp/facerec/data/at/s30/6.pgm;29
/home/philipp/facerec/data/at/s30/9.pgm;29
/home/philipp/facerec/data/at/s30/5.pgm;29
/home/philipp/facerec/data/at/s30/3.pgm;29
/home/philipp/facerec/data/at/s30/4.pgm;29
/home/philipp/facerec/data/at/s30/10.pgm;29
/home/philipp/facerec/data/at/s30/8.pgm;29
/home/philipp/facerec/data/at/s30/1.pgm;29
/home/philipp/facerec/data/at/s39/2.pgm;38
/home/philipp/facerec/data/at/s39/7.pgm;38
/home/philipp/facerec/data/at/s39/6.pgm;38
/home/philipp/facerec/data/at/s39/9.pgm;38
/home/philipp/facerec/data/at/s39/5.pgm;38
/home/philipp/facerec/data/at/s39/3.pgm;38
/home/philipp/facerec/data/at/s39/4.pgm;38
/home/philipp/facerec/data/at/s39/10.pgm;38
/home/philipp/facerec/data/at/s39/8.pgm;38
/home/philipp/facerec/data/at/s39/1.pgm;38
/home/philipp/facerec/data/at/s35/2.pgm;34
/home/philipp/facerec/data/at/s35/7.pgm;34
/home/philipp/facerec/data/at/s35/6.pgm;34
/home/philipp/facerec/data/at/s35/9.pgm;34
/home/philipp/facerec/data/at/s35/5.pgm;34
/home/philipp/facerec/data/at/s35/3.pgm;34
/home/philipp/facerec/data/at/s35/4.pgm;34
/home/philipp/facerec/data/at/s35/10.pgm;34
/home/philipp/facerec/data/at/s35/8.pgm;34
/home/philipp/facerec/data/at/s35/1.pgm;34
/home/philipp/facerec/data/at/s23/2.pgm;22
/home/philipp/facerec/data/at/s23/7.pgm;22
/home/philipp/facerec/data/at/s23/6.pgm;22
/home/philipp/facerec/data/at/s23/9.pgm;22
/home/philipp/facerec/data/at/s23/5.pgm;22
/home/philipp/facerec/data/at/s23/3.pgm;22
/home/philipp/facerec/data/at/s23/4.pgm;22
/home/philipp/facerec/data/at/s23/10.pgm;22
/home/philipp/facerec/data/at/s23/8.pgm;22
/home/philipp/facerec/data/at/s23/1.pgm;22
/home/philipp/facerec/data/at/s4/2.pgm;3
/home/philipp/facerec/data/at/s4/7.pgm;3
/home/philipp/facerec/data/at/s4/6.pgm;3
/home/philipp/facerec/data/at/s4/9.pgm;3
/home/philipp/facerec/data/at/s4/5.pgm;3
/home/philipp/facerec/data/at/s4/3.pgm;3
/home/philipp/facerec/data/at/s4/4.pgm;3
/home/philipp/facerec/data/at/s4/10.pgm;3
/home/philipp/facerec/data/at/s4/8.pgm;3
/home/philipp/facerec/data/at/s4/1.pgm;3
/home/philipp/facerec/data/at/s9/2.pgm;8
/home/philipp/facerec/data/at/s9/7.pgm;8
/home/philipp/facerec/data/at/s9/6.pgm;8
/home/philipp/facerec/data/at/s9/9.pgm;8
/home/philipp/facerec/data/at/s9/5.pgm;8
/home/philipp/facerec/data/at/s9/3.pgm;8
/home/philipp/facerec/data/at/s9/4.pgm;8
/home/philipp/facerec/data/at/s9/10.pgm;8
/home/philipp/facerec/data/at/s9/8.pgm;8
/home/philipp/facerec/data/at/s9/1.pgm;8
/home/philipp/facerec/data/at/s37/2.pgm;36
/home/philipp/facerec/data/at/s37/7.pgm;36
/home/philipp/facerec/data/at/s37/6.pgm;36
/home/philipp/facerec/data/at/s37/9.pgm;36
/home/philipp/facerec/data/at/s37/5.pgm;36
/home/philipp/facerec/data/at/s37/3.pgm;36
/home/philipp/facerec/data/at/s37/4.pgm;36
/home/philipp/facerec/data/at/s37/10.pgm;36
/home/philipp/facerec/data/at/s37/8.pgm;36
/home/philipp/facerec/data/at/s37/1.pgm;36
/home/philipp/facerec/data/at/s24/2.pgm;23
/home/philipp/facerec/data/at/s24/7.pgm;23
/home/philipp/facerec/data/at/s24/6.pgm;23
/home/philipp/facerec/data/at/s24/9.pgm;23
/home/philipp/facerec/data/at/s24/5.pgm;23
/home/philipp/facerec/data/at/s24/3.pgm;23
/home/philipp/facerec/data/at/s24/4.pgm;23
/home/philipp/facerec/data/at/s24/10.pgm;23
/home/philipp/facerec/data/at/s24/8.pgm;23
/home/philipp/facerec/data/at/s24/1.pgm;23
/home/philipp/facerec/data/at/s19/2.pgm;18
/home/philipp/facerec/data/at/s19/7.pgm;18
/home/philipp/facerec/data/at/s19/6.pgm;18
/home/philipp/facerec/data/at/s19/9.pgm;18
/home/philipp/facerec/data/at/s19/5.pgm;18
/home/philipp/facerec/data/at/s19/3.pgm;18
/home/philipp/facerec/data/at/s19/4.pgm;18
/home/philipp/facerec/data/at/s19/10.pgm;18
/home/philipp/facerec/data/at/s19/8.pgm;18
/home/philipp/facerec/data/at/s19/1.pgm;18
/home/philipp/facerec/data/at/s8/2.pgm;7
/home/philipp/facerec/data/at/s8/7.pgm;7
/home/philipp/facerec/data/at/s8/6.pgm;7
/home/philipp/facerec/data/at/s8/9.pgm;7
/home/philipp/facerec/data/at/s8/5.pgm;7
/home/philipp/facerec/data/at/s8/3.pgm;7
/home/philipp/facerec/data/at/s8/4.pgm;7
/home/philipp/facerec/data/at/s8/10.pgm;7
/home/philipp/facerec/data/at/s8/8.pgm;7
/home/philipp/facerec/data/at/s8/1.pgm;7
/home/philipp/facerec/data/at/s21/2.pgm;20
/home/philipp/facerec/data/at/s21/7.pgm;20
/home/philipp/facerec/data/at/s21/6.pgm;20
/home/philipp/facerec/data/at/s21/9.pgm;20
/home/philipp/facerec/data/at/s21/5.pgm;20
/home/philipp/facerec/data/at/s21/3.pgm;20
/home/philipp/facerec/data/at/s21/4.pgm;20
/home/philipp/facerec/data/at/s21/10.pgm;20
/home/philipp/facerec/data/at/s21/8.pgm;20
/home/philipp/facerec/data/at/s21/1.pgm;20
/home/philipp/facerec/data/at/s1/2.pgm;0
/home/philipp/facerec/data/at/s1/7.pgm;0
/home/philipp/facerec/data/at/s1/6.pgm;0
/home/philipp/facerec/data/at/s1/9.pgm;0
/home/philipp/facerec/data/at/s1/5.pgm;0
/home/philipp/facerec/data/at/s1/3.pgm;0
/home/philipp/facerec/data/at/s1/4.pgm;0
/home/philipp/facerec/data/at/s1/10.pgm;0
/home/philipp/facerec/data/at/s1/8.pgm;0
/home/philipp/facerec/data/at/s1/1.pgm;0
/home/philipp/facerec/data/at/s7/2.pgm;6
/home/philipp/facerec/data/at/s7/7.pgm;6
/home/philipp/facerec/data/at/s7/6.pgm;6
/home/philipp/facerec/data/at/s7/9.pgm;6
/home/philipp/facerec/data/at/s7/5.pgm;6
/home/philipp/facerec/data/at/s7/3.pgm;6
/home/philipp/facerec/data/at/s7/4.pgm;6
/home/philipp/facerec/data/at/s7/10.pgm;6
/home/philipp/facerec/data/at/s7/8.pgm;6
/home/philipp/facerec/data/at/s7/1.pgm;6
/home/philipp/facerec/data/at/s16/2.pgm;15
/home/philipp/facerec/data/at/s16/7.pgm;15
/home/philipp/facerec/data/at/s16/6.pgm;15
/home/philipp/facerec/data/at/s16/9.pgm;15
/home/philipp/facerec/data/at/s16/5.pgm;15
/home/philipp/facerec/data/at/s16/3.pgm;15
/home/philipp/facerec/data/at/s16/4.pgm;15
/home/philipp/facerec/data/at/s16/10.pgm;15
/home/philipp/facerec/data/at/s16/8.pgm;15
/home/philipp/facerec/data/at/s16/1.pgm;15
/home/philipp/facerec/data/at/s36/2.pgm;35
/home/philipp/facerec/data/at/s36/7.pgm;35
/home/philipp/facerec/data/at/s36/6.pgm;35
/home/philipp/facerec/data/at/s36/9.pgm;35
/home/philipp/facerec/data/at/s36/5.pgm;35
/home/philipp/facerec/data/at/s36/3.pgm;35
/home/philipp/facerec/data/at/s36/4.pgm;35
/home/philipp/facerec/data/at/s36/10.pgm;35
/home/philipp/facerec/data/at/s36/8.pgm;35
/home/philipp/facerec/data/at/s36/1.pgm;35
/home/philipp/facerec/data/at/s25/2.pgm;24
/home/philipp/facerec/data/at/s25/7.pgm;24
/home/philipp/facerec/data/at/s25/6.pgm;24
/home/philipp/facerec/data/at/s25/9.pgm;24
/home/philipp/facerec/data/at/s25/5.pgm;24
/home/philipp/facerec/data/at/s25/3.pgm;24
/home/philipp/facerec/data/at/s25/4.pgm;24
/home/philipp/facerec/data/at/s25/10.pgm;24
/home/philipp/facerec/data/at/s25/8.pgm;24
/home/philipp/facerec/data/at/s25/1.pgm;24
/home/philipp/facerec/data/at/s14/2.pgm;13
/home/philipp/facerec/data/at/s14/7.pgm;13
/home/philipp/facerec/data/at/s14/6.pgm;13
/home/philipp/facerec/data/at/s14/9.pgm;13
/home/philipp/facerec/data/at/s14/5.pgm;13
/home/philipp/facerec/data/at/s14/3.pgm;13
/home/philipp/facerec/data/at/s14/4.pgm;13
/home/philipp/facerec/data/at/s14/10.pgm;13
/home/philipp/facerec/data/at/s14/8.pgm;13
/home/philipp/facerec/data/at/s14/1.pgm;13
/home/philipp/facerec/data/at/s34/2.pgm;33
/home/philipp/facerec/data/at/s34/7.pgm;33
/home/philipp/facerec/data/at/s34/6.pgm;33
/home/philipp/facerec/data/at/s34/9.pgm;33
/home/philipp/facerec/data/at/s34/5.pgm;33
/home/philipp/facerec/data/at/s34/3.pgm;33
/home/philipp/facerec/data/at/s34/4.pgm;33
/home/philipp/facerec/data/at/s34/10.pgm;33
/home/philipp/facerec/data/at/s34/8.pgm;33
/home/philipp/facerec/data/at/s34/1.pgm;33
/home/philipp/facerec/data/at/s11/2.pgm;10
/home/philipp/facerec/data/at/s11/7.pgm;10
/home/philipp/facerec/data/at/s11/6.pgm;10
/home/philipp/facerec/data/at/s11/9.pgm;10
/home/philipp/facerec/data/at/s11/5.pgm;10
/home/philipp/facerec/data/at/s11/3.pgm;10
/home/philipp/facerec/data/at/s11/4.pgm;10
/home/philipp/facerec/data/at/s11/10.pgm;10
/home/philipp/facerec/data/at/s11/8.pgm;10
/home/philipp/facerec/data/at/s11/1.pgm;10
/home/philipp/facerec/data/at/s26/2.pgm;25
/home/philipp/facerec/data/at/s26/7.pgm;25
/home/philipp/facerec/data/at/s26/6.pgm;25
/home/philipp/facerec/data/at/s26/9.pgm;25
/home/philipp/facerec/data/at/s26/5.pgm;25
/home/philipp/facerec/data/at/s26/3.pgm;25
/home/philipp/facerec/data/at/s26/4.pgm;25
/home/philipp/facerec/data/at/s26/10.pgm;25
/home/philipp/facerec/data/at/s26/8.pgm;25
/home/philipp/facerec/data/at/s26/1.pgm;25
/home/philipp/facerec/data/at/s18/2.pgm;17
/home/philipp/facerec/data/at/s18/7.pgm;17
/home/philipp/facerec/data/at/s18/6.pgm;17
/home/philipp/facerec/data/at/s18/9.pgm;17
/home/philipp/facerec/data/at/s18/5.pgm;17
/home/philipp/facerec/data/at/s18/3.pgm;17
/home/philipp/facerec/data/at/s18/4.pgm;17
/home/philipp/facerec/data/at/s18/10.pgm;17
/home/philipp/facerec/data/at/s18/8.pgm;17
/home/philipp/facerec/data/at/s18/1.pgm;17
/home/philipp/facerec/data/at/s29/2.pgm;28
/home/philipp/facerec/data/at/s29/7.pgm;28
/home/philipp/facerec/data/at/s29/6.pgm;28
/home/philipp/facerec/data/at/s29/9.pgm;28
/home/philipp/facerec/data/at/s29/5.pgm;28
/home/philipp/facerec/data/at/s29/3.pgm;28
/home/philipp/facerec/data/at/s29/4.pgm;28
/home/philipp/facerec/data/at/s29/10.pgm;28
/home/philipp/facerec/data/at/s29/8.pgm;28
/home/philipp/facerec/data/at/s29/1.pgm;28
/home/philipp/facerec/data/at/s33/2.pgm;32
/home/philipp/facerec/data/at/s33/7.pgm;32
/home/philipp/facerec/data/at/s33/6.pgm;32
/home/philipp/facerec/data/at/s33/9.pgm;32
/home/philipp/facerec/data/at/s33/5.pgm;32
/home/philipp/facerec/data/at/s33/3.pgm;32
/home/philipp/facerec/data/at/s33/4.pgm;32
/home/philipp/facerec/data/at/s33/10.pgm;32
/home/philipp/facerec/data/at/s33/8.pgm;32
/home/philipp/facerec/data/at/s33/1.pgm;32
/home/philipp/facerec/data/at/s12/2.pgm;11
/home/philipp/facerec/data/at/s12/7.pgm;11
/home/philipp/facerec/data/at/s12/6.pgm;11
/home/philipp/facerec/data/at/s12/9.pgm;11
/home/philipp/facerec/data/at/s12/5.pgm;11
/home/philipp/facerec/data/at/s12/3.pgm;11
/home/philipp/facerec/data/at/s12/4.pgm;11
/home/philipp/facerec/data/at/s12/10.pgm;11
/home/philipp/facerec/data/at/s12/8.pgm;11
/home/philipp/facerec/data/at/s12/1.pgm;11
/home/philipp/facerec/data/at/s6/2.pgm;5
/home/philipp/facerec/data/at/s6/7.pgm;5
/home/philipp/facerec/data/at/s6/6.pgm;5
/home/philipp/facerec/data/at/s6/9.pgm;5
/home/philipp/facerec/data/at/s6/5.pgm;5
/home/philipp/facerec/data/at/s6/3.pgm;5
/home/philipp/facerec/data/at/s6/4.pgm;5
/home/philipp/facerec/data/at/s6/10.pgm;5
/home/philipp/facerec/data/at/s6/8.pgm;5
/home/philipp/facerec/data/at/s6/1.pgm;5
/home/philipp/facerec/data/at/s22/2.pgm;21
/home/philipp/facerec/data/at/s22/7.pgm;21
/home/philipp/facerec/data/at/s22/6.pgm;21
/home/philipp/facerec/data/at/s22/9.pgm;21
/home/philipp/facerec/data/at/s22/5.pgm;21
/home/philipp/facerec/data/at/s22/3.pgm;21
/home/philipp/facerec/data/at/s22/4.pgm;21
/home/philipp/facerec/data/at/s22/10.pgm;21
/home/philipp/facerec/data/at/s22/8.pgm;21
/home/philipp/facerec/data/at/s22/1.pgm;21
/home/philipp/facerec/data/at/s15/2.pgm;14
/home/philipp/facerec/data/at/s15/7.pgm;14
/home/philipp/facerec/data/at/s15/6.pgm;14
/home/philipp/facerec/data/at/s15/9.pgm;14
/home/philipp/facerec/data/at/s15/5.pgm;14
/home/philipp/facerec/data/at/s15/3.pgm;14
/home/philipp/facerec/data/at/s15/4.pgm;14
/home/philipp/facerec/data/at/s15/10.pgm;14
/home/philipp/facerec/data/at/s15/8.pgm;14
/home/philipp/facerec/data/at/s15/1.pgm;14
/home/philipp/facerec/data/at/s2/2.pgm;1
/home/philipp/facerec/data/at/s2/7.pgm;1
/home/philipp/facerec/data/at/s2/6.pgm;1
/home/philipp/facerec/data/at/s2/9.pgm;1
/home/philipp/facerec/data/at/s2/5.pgm;1
/home/philipp/facerec/data/at/s2/3.pgm;1
/home/philipp/facerec/data/at/s2/4.pgm;1
/home/philipp/facerec/data/at/s2/10.pgm;1
/home/philipp/facerec/data/at/s2/8.pgm;1
/home/philipp/facerec/data/at/s2/1.pgm;1
/home/philipp/facerec/data/at/s31/2.pgm;30
/home/philipp/facerec/data/at/s31/7.pgm;30
/home/philipp/facerec/data/at/s31/6.pgm;30
/home/philipp/facerec/data/at/s31/9.pgm;30
/home/philipp/facerec/data/at/s31/5.pgm;30
/home/philipp/facerec/data/at/s31/3.pgm;30
/home/philipp/facerec/data/at/s31/4.pgm;30
/home/philipp/facerec/data/at/s31/10.pgm;30
/home/philipp/facerec/data/at/s31/8.pgm;30
/home/philipp/facerec/data/at/s31/1.pgm;30
/home/philipp/facerec/data/at/s28/2.pgm;27
/home/philipp/facerec/data/at/s28/7.pgm;27
/home/philipp/facerec/data/at/s28/6.pgm;27
/home/philipp/facerec/data/at/s28/9.pgm;27
/home/philipp/facerec/data/at/s28/5.pgm;27
/home/philipp/facerec/data/at/s28/3.pgm;27
/home/philipp/facerec/data/at/s28/4.pgm;27
/home/philipp/facerec/data/at/s28/10.pgm;27
/home/philipp/facerec/data/at/s28/8.pgm;27
/home/philipp/facerec/data/at/s28/1.pgm;27
/home/philipp/facerec/data/at/s40/2.pgm;39
/home/philipp/facerec/data/at/s40/7.pgm;39
/home/philipp/facerec/data/at/s40/6.pgm;39
/home/philipp/facerec/data/at/s40/9.pgm;39
/home/philipp/facerec/data/at/s40/5.pgm;39
/home/philipp/facerec/data/at/s40/3.pgm;39
/home/philipp/facerec/data/at/s40/4.pgm;39
/home/philipp/facerec/data/at/s40/10.pgm;39
/home/philipp/facerec/data/at/s40/8.pgm;39
/home/philipp/facerec/data/at/s40/1.pgm;39
/home/philipp/facerec/data/at/s3/2.pgm;2
/home/philipp/facerec/data/at/s3/7.pgm;2
/home/philipp/facerec/data/at/s3/6.pgm;2
/home/philipp/facerec/data/at/s3/9.pgm;2
/home/philipp/facerec/data/at/s3/5.pgm;2
/home/philipp/facerec/data/at/s3/3.pgm;2
/home/philipp/facerec/data/at/s3/4.pgm;2
/home/philipp/facerec/data/at/s3/10.pgm;2
/home/philipp/facerec/data/at/s3/8.pgm;2
/home/philipp/facerec/data/at/s3/1.pgm;2
/home/philipp/facerec/data/at/s38/2.pgm;37
/home/philipp/facerec/data/at/s38/7.pgm;37
/home/philipp/facerec/data/at/s38/6.pgm;37
/home/philipp/facerec/data/at/s38/9.pgm;37
/home/philipp/facerec/data/at/s38/5.pgm;37
/home/philipp/facerec/data/at/s38/3.pgm;37
/home/philipp/facerec/data/at/s38/4.pgm;37
/home/philipp/facerec/data/at/s38/10.pgm;37
/home/philipp/facerec/data/at/s38/8.pgm;37
/home/philipp/facerec/data/at/s38/1.pgm;37
modules/face/samples/etc/create_csv.py
Executable
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env python
import sys
import os.path
# This is a tiny script to help you creating a CSV file from a face
# database with a similar hierarchie:
#
# philipp@mango:~/facerec/data/at$ tree
# .
# |-- README
# |-- s1
# | |-- 1.pgm
# | |-- ...
# | |-- 10.pgm
# |-- s2
# | |-- 1.pgm
# | |-- ...
# | |-- 10.pgm
# ...
# |-- s40
# | |-- 1.pgm
# | |-- ...
# | |-- 10.pgm
#
if __name__ == "__main__":
if len(sys.argv) != 2:
print "usage: create_csv <base_path>"
sys.exit(1)
BASE_PATH=sys.argv[1]
SEPARATOR=";"
label = 0
for dirname, dirnames, filenames in os.walk(BASE_PATH):
for subdirname in dirnames:
subject_path = os.path.join(dirname, subdirname)
for filename in os.listdir(subject_path):
abs_path = "%s/%s" % (subject_path, filename)
print "%s%s%d" % (abs_path, SEPARATOR, label)
label = label + 1
modules/face/samples/etc/crop_face.py
Executable
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python
# Software License Agreement (BSD License)
#
# Copyright (c) 2012, Philipp Wagner
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials provided
# with the distribution.
# * Neither the name of the author nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import sys, math, Image
def Distance(p1,p2):
dx = p2[0] - p1[0]
dy = p2[1] - p1[1]
return math.sqrt(dx*dx+dy*dy)
def ScaleRotateTranslate(image, angle, center = None, new_center = None, scale = None, resample=Image.BICUBIC):
if (scale is None) and (center is None):
return image.rotate(angle=angle, resample=resample)
nx,ny = x,y = center
sx=sy=1.0
if new_center:
(nx,ny) = new_center
if scale:
(sx,sy) = (scale, scale)
cosine = math.cos(angle)
sine = math.sin(angle)
a = cosine/sx
b = sine/sx
c = x-nx*a-ny*b
d = -sine/sy
e = cosine/sy
f = y-nx*d-ny*e
return image.transform(image.size, Image.AFFINE, (a,b,c,d,e,f), resample=resample)
def CropFace(image, eye_left=(0,0), eye_right=(0,0), offset_pct=(0.2,0.2), dest_sz = (70,70)):
# calculate offsets in original image
offset_h = math.floor(float(offset_pct[0])*dest_sz[0])
offset_v = math.floor(float(offset_pct[1])*dest_sz[1])
# get the direction
eye_direction = (eye_right[0] - eye_left[0], eye_right[1] - eye_left[1])
# calc rotation angle in radians
rotation = -math.atan2(float(eye_direction[1]),float(eye_direction[0]))
# distance between them
dist = Distance(eye_left, eye_right)
# calculate the reference eye-width
reference = dest_sz[0] - 2.0*offset_h
# scale factor
scale = float(dist)/float(reference)
# rotate original around the left eye
image = ScaleRotateTranslate(image, center=eye_left, angle=rotation)
# crop the rotated image
crop_xy = (eye_left[0] - scale*offset_h, eye_left[1] - scale*offset_v)
crop_size = (dest_sz[0]*scale, dest_sz[1]*scale)
image = image.crop((int(crop_xy[0]), int(crop_xy[1]), int(crop_xy[0]+crop_size[0]), int(crop_xy[1]+crop_size[1])))
# resize it
image = image.resize(dest_sz, Image.ANTIALIAS)
return image
def readFileNames():
try:
inFile = open('path_to_created_csv_file.csv')
except:
raise IOError('There is no file named path_to_created_csv_file.csv in current directory.')
return False
picPath = []
picIndex = []
for line in inFile.readlines():
if line != '':
fields = line.rstrip().split(';')
picPath.append(fields[0])
picIndex.append(int(fields[1]))
return (picPath, picIndex)
if __name__ == "__main__":
[images, indexes]=readFileNames()
if not os.path.exists("modified"):
os.makedirs("modified")
for img in images:
image = Image.open(img)
CropFace(image, eye_left=(252,364), eye_right=(420,366), offset_pct=(0.1,0.1), dest_sz=(200,200)).save("modified/"+img.rstrip().split('/')[1]+"_10_10_200_200.jpg")
CropFace(image, eye_left=(252,364), eye_right=(420,366), offset_pct=(0.2,0.2), dest_sz=(200,200)).save("modified/"+img.rstrip().split('/')[1]+"_20_20_200_200.jpg")
CropFace(image, eye_left=(252,364), eye_right=(420,366), offset_pct=(0.3,0.3), dest_sz=(200,200)).save("modified/"+img.rstrip().split('/')[1]+"_30_30_200_200.jpg")
CropFace(image, eye_left=(252,364), eye_right=(420,366), offset_pct=(0.2,0.2)).save("modified/"+img.rstrip().split('/')[1]+"_20_20_70_70.jpg")
modules/face/samples/facemark_demo_aam.cpp
+290
View File
@@ -0,0 +1,290 @@
/*
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
/*----------------------------------------------
* Usage:
* facemark_demo_aam <face_cascade_model> <eyes_cascade_model> <training_images> <annotation_files> [test_files]
*
* Example:
* facemark_demo_aam ../face_cascade.xml ../eyes_cascade.xml ../images_train.txt ../points_train.txt ../test.txt
*
* Notes:
* the user should provides the list of training images_train
* accompanied by their corresponding landmarks location in separated files.
* example of contents for images_train.txt:
* ../trainset/image_0001.png
* ../trainset/image_0002.png
* example of contents for points_train.txt:
* ../trainset/image_0001.pts
* ../trainset/image_0002.pts
* where the image_xxxx.pts contains the position of each face landmark.
* example of the contents:
* version: 1
* n_points: 68
* {
* 115.167660 220.807529
* 116.164839 245.721357
* 120.208690 270.389841
* ...
* }
* example of the dataset is available at https://ibug.doc.ic.ac.uk/download/annotations/lfpw.zip
*--------------------------------------------------*/
#include <stdio.h>
#include <fstream>
#include <sstream>
#include "opencv2/core.hpp"
#include "opencv2/geometry.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/face.hpp"
#include <iostream>
#include <string>
#include <ctime>
using namespace std;
using namespace cv;
using namespace cv::face;
bool myDetector( InputArray image, OutputArray ROIs, CascadeClassifier *face_cascade);
bool getInitialFitting(Mat image, Rect face, std::vector<Point2f> s0,
CascadeClassifier eyes_cascade, Mat & R, Point2f & Trans, float & scale);
bool parseArguments(int argc, char** argv, String & cascade,
String & model, String & images, String & annotations, String & testImages
);
int main(int argc, char** argv )
{
String cascade_path,eyes_cascade_path,images_path, annotations_path, test_images_path;
if(!parseArguments(argc, argv, cascade_path,eyes_cascade_path,images_path, annotations_path, test_images_path))
return -1;
//! [instance_creation]
/*create the facemark instance*/
FacemarkAAM::Params params;
params.scales.push_back(2.0);
params.scales.push_back(4.0);
params.model_filename = "AAM.yaml";
Ptr<FacemarkAAM> facemark = FacemarkAAM::create(params);
//! [instance_creation]
//! [load_dataset]
/*Loads the dataset*/
std::vector<String> images_train;
std::vector<String> landmarks_train;
loadDatasetList(images_path,annotations_path,images_train,landmarks_train);
//! [load_dataset]
//! [add_samples]
Mat image;
std::vector<Point2f> facial_points;
for(size_t i=0;i<images_train.size();i++){
image = imread(images_train[i].c_str());
loadFacePoints(landmarks_train[i],facial_points);
facemark->addTrainingSample(image, facial_points);
}
//! [add_samples]
//! [training]
/* trained model will be saved to AAM.yml */
facemark->training();
//! [training]
//! [load_test_images]
/*test using some images*/
String testFiles(images_path), testPts(annotations_path);
if(!test_images_path.empty()){
testFiles = test_images_path;
testPts = test_images_path; //unused
}
std::vector<String> images;
std::vector<String> facePoints;
loadDatasetList(testFiles, testPts, images, facePoints);
//! [load_test_images]
//! [trainsformation_variables]
float scale ;
Point2f T;
Mat R;
//! [trainsformation_variables]
//! [base_shape]
FacemarkAAM::Data data;
facemark->getData(&data);
std::vector<Point2f> s0 = data.s0;
//! [base_shape]
//! [fitting]
/*fitting process*/
std::vector<Rect> faces;
//! [load_cascade_models]
CascadeClassifier face_cascade(cascade_path);
CascadeClassifier eyes_cascade(eyes_cascade_path);
//! [load_cascade_models]
for(int i=0;i<(int)images.size();i++){
printf("image #%i ", i);
//! [detect_face]
image = imread(images[i]);
myDetector(image, faces, &face_cascade);
//! [detect_face]
if(faces.size()>0){
//! [get_initialization]
std::vector<FacemarkAAM::Config> conf;
std::vector<Rect> faces_eyes;
for(unsigned j=0;j<faces.size();j++){
if(getInitialFitting(image,faces[j],s0,eyes_cascade, R,T,scale)){
conf.push_back(FacemarkAAM::Config(R,T,scale,(int)params.scales.size()-1));
faces_eyes.push_back(faces[j]);
}
}
//! [get_initialization]
//! [fitting_process]
if(conf.size()>0){
printf(" - face with eyes found %i ", (int)conf.size());
std::vector<std::vector<Point2f> > landmarks;
double newtime = (double)getTickCount();
facemark->fitConfig(image, faces_eyes, landmarks, conf);
double fittime = ((getTickCount() - newtime)/getTickFrequency());
for(unsigned j=0;j<landmarks.size();j++){
drawFacemarks(image, landmarks[j],Scalar(0,255,0));
}
printf("%f ms\n",fittime*1000);
imshow("fitting", image);
waitKey(0);
}else{
printf("initialization cannot be computed - skipping\n");
}
//! [fitting_process]
}
} //for
//! [fitting]
}
bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
bool getInitialFitting(Mat image, Rect face, std::vector<Point2f> s0 ,CascadeClassifier eyes_cascade, Mat & R, Point2f & Trans, float & scale){
std::vector<Point2f> mybase;
std::vector<Point2f> T;
std::vector<Point2f> base = Mat(Mat(s0)+Scalar(image.cols/2,image.rows/2)).reshape(2);
std::vector<Point2f> base_shape,base_shape2 ;
Point2f e1 = Point2f((float)((base[39].x+base[36].x)/2.0),(float)((base[39].y+base[36].y)/2.0)); //eye1
Point2f e2 = Point2f((float)((base[45].x+base[42].x)/2.0),(float)((base[45].y+base[42].y)/2.0)); //eye2
if(face.width==0 || face.height==0) return false;
std::vector<Point2f> eye;
bool found=false;
Mat faceROI = image( face);
std::vector<Rect> eyes;
//-- In each face, detect eyes
eyes_cascade.detectMultiScale( faceROI, eyes, 1.1, 2, CASCADE_SCALE_IMAGE, Size(20, 20) );
if(eyes.size()==2){
found = true;
int j=0;
Point2f c1( (float)(face.x + eyes[j].x + eyes[j].width*0.5), (float)(face.y + eyes[j].y + eyes[j].height*0.5));
j=1;
Point2f c2( (float)(face.x + eyes[j].x + eyes[j].width*0.5), (float)(face.y + eyes[j].y + eyes[j].height*0.5));
Point2f pivot;
double a0,a1;
if(c1.x<c2.x){
pivot = c1;
a0 = atan2(c2.y-c1.y, c2.x-c1.x);
}else{
pivot = c2;
a0 = atan2(c1.y-c2.y, c1.x-c2.x);
}
scale = (float)(norm(Mat(c1)-Mat(c2))/norm(Mat(e1)-Mat(e2)));
mybase= Mat(Mat(s0)*scale).reshape(2);
Point2f ey1 = Point2f((float)((mybase[39].x+mybase[36].x)/2.0),(float)((mybase[39].y+mybase[36].y)/2.0));
Point2f ey2 = Point2f((float)((mybase[45].x+mybase[42].x)/2.0),(float)((mybase[45].y+mybase[42].y)/2.0));
#define TO_DEGREE 180.0/3.14159265
a1 = atan2(ey2.y-ey1.y, ey2.x-ey1.x);
Mat rot = getRotationMatrix2D(Point2f(0,0), (a1-a0)*TO_DEGREE, 1.0);
rot(Rect(0,0,2,2)).convertTo(R, CV_32F);
base_shape = Mat(Mat(R*scale*Mat(Mat(s0).reshape(1)).t()).t()).reshape(2);
ey1 = Point2f((float)((base_shape[39].x+base_shape[36].x)/2.0),(float)((base_shape[39].y+base_shape[36].y)/2.0));
ey2 = Point2f((float)((base_shape[45].x+base_shape[42].x)/2.0),(float)((base_shape[45].y+base_shape[42].y)/2.0));
T.push_back(Point2f(pivot.x-ey1.x,pivot.y-ey1.y));
Trans = Point2f(pivot.x-ey1.x,pivot.y-ey1.y);
return true;
}else{
Trans = Point2f( (float)(face.x + face.width*0.5),(float)(face.y + face.height*0.5));
}
return found;
}
bool parseArguments(int argc, char** argv,
String & cascade,
String & model,
String & images,
String & annotations,
String & test_images
){
const String keys =
"{ @f face-cascade | | (required) path to the cascade model file for the face detector }"
"{ @e eyes-cascade | | (required) path to the cascade model file for the eyes detector }"
"{ @i images | | (required) path of a text file contains the list of paths to all training images}"
"{ @a annotations | | (required) Path of a text file contains the list of paths to all annotations files}"
"{ @t test-images | | Path of a text file contains the list of paths to the test images}"
"{ help h usage ? | | facemark_demo_aam -face-cascade -eyes-cascade -images -annotations [-t]\n"
" example: facemark_demo_aam ../face_cascade.xml ../eyes_cascade.xml ../images_train.txt ../points_train.txt ../test.txt}"
;
CommandLineParser parser(argc, argv,keys);
parser.about("hello");
if (parser.has("help")){
parser.printMessage();
return false;
}
cascade = String(parser.get<String>("face-cascade"));
model = String(parser.get<string>("eyes-cascade"));
images = String(parser.get<string>("images"));
annotations = String(parser.get<string>("annotations"));
test_images = String(parser.get<string>("test-images"));
if(cascade.empty() || model.empty() || images.empty() || annotations.empty()){
std::cerr << "one or more required arguments are not found" << '\n';
cout<<"face-cascade : "<<cascade.c_str()<<endl;
cout<<"eyes-cascade : "<<model.c_str()<<endl;
cout<<"images : "<<images.c_str()<<endl;
cout<<"annotations : "<<annotations.c_str()<<endl;
parser.printMessage();
return false;
}
return true;
}
modules/face/samples/facemark_demo_lbf.cpp
+182
View File
@@ -0,0 +1,182 @@
/*
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
/*----------------------------------------------
* Usage:
* facemark_demo_lbf <face_cascade_model> <saved_model_filename> <training_images> <annotation_files> [test_files]
*
* Example:
* facemark_demo_lbf ../face_cascade.xml ../LBF.model ../images_train.txt ../points_train.txt ../test.txt
*
* Notes:
* the user should provides the list of training images_train
* accompanied by their corresponding landmarks location in separated files.
* example of contents for images_train.txt:
* ../trainset/image_0001.png
* ../trainset/image_0002.png
* example of contents for points_train.txt:
* ../trainset/image_0001.pts
* ../trainset/image_0002.pts
* where the image_xxxx.pts contains the position of each face landmark.
* example of the contents:
* version: 1
* n_points: 68
* {
* 115.167660 220.807529
* 116.164839 245.721357
* 120.208690 270.389841
* ...
* }
* example of the dataset is available at https://ibug.doc.ic.ac.uk/download/annotations/ibug.zip
*--------------------------------------------------*/
#include <stdio.h>
#include <fstream>
#include <sstream>
#include <iostream>
#include "opencv2/core.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/face.hpp"
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector( InputArray image, OutputArray roi, CascadeClassifier *face_detector);
static bool parseArguments(int argc, char** argv, String & cascade,
String & model, String & images, String & annotations, String & testImages
);
int main(int argc, char** argv)
{
String cascade_path,model_path,images_path, annotations_path, test_images_path;
if(!parseArguments(argc, argv, cascade_path,model_path,images_path, annotations_path, test_images_path))
return -1;
/*create the facemark instance*/
FacemarkLBF::Params params;
params.model_filename = model_path;
params.cascade_face = cascade_path;
Ptr<FacemarkLBF> facemark = FacemarkLBF::create(params);
CascadeClassifier face_cascade;
face_cascade.load(params.cascade_face.c_str());
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
/*Loads the dataset*/
std::vector<String> images_train;
std::vector<String> landmarks_train;
loadDatasetList(images_path,annotations_path,images_train,landmarks_train);
Mat image;
std::vector<Point2f> facial_points;
for(size_t i=0;i<images_train.size();i++){
printf("%i/%i :: %s\n", (int)(i+1), (int)images_train.size(),images_train[i].c_str());
image = imread(images_train[i].c_str());
loadFacePoints(landmarks_train[i],facial_points);
facemark->addTrainingSample(image, facial_points);
}
/*train the Algorithm*/
facemark->training();
/*test using some images*/
String testFiles(images_path), testPts(annotations_path);
if(!test_images_path.empty()){
testFiles = test_images_path;
testPts = test_images_path; //unused
}
std::vector<String> images;
std::vector<String> facePoints;
loadDatasetList(testFiles, testPts, images, facePoints);
std::vector<Rect> rects;
CascadeClassifier cc(params.cascade_face.c_str());
for(size_t i=0;i<images.size();i++){
std::vector<std::vector<Point2f> > landmarks;
cout<<images[i];
Mat img = imread(images[i]);
facemark->getFaces(img, rects);
facemark->fit(img, rects, landmarks);
for(size_t j=0;j<rects.size();j++){
drawFacemarks(img, landmarks[j], Scalar(0,0,255));
rectangle(img, rects[j], Scalar(255,0,255));
}
if(rects.size()>0){
cout<<endl;
imshow("result", img);
waitKey(0);
}else{
cout<<"face not found"<<endl;
}
}
}
bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
bool parseArguments(int argc, char** argv,
String & cascade,
String & model,
String & images,
String & annotations,
String & test_images
){
const String keys =
"{ @c cascade | | (required) path to the face cascade xml file fo the face detector }"
"{ @i images | | (required) path of a text file contains the list of paths to all training images}"
"{ @a annotations | | (required) Path of a text file contains the list of paths to all annotations files}"
"{ @m model | | (required) path to save the trained model }"
"{ t test-images | | Path of a text file contains the list of paths to the test images}"
"{ help h usage ? | | facemark_demo_lbf -cascade -images -annotations -model [-t] \n"
" example: facemark_demo_lbf ../face_cascade.xml ../images_train.txt ../points_train.txt ../lbf.model}"
;
CommandLineParser parser(argc, argv,keys);
parser.about("hello");
if (parser.has("help")){
parser.printMessage();
return false;
}
cascade = String(parser.get<String>("cascade"));
model = String(parser.get<string>("model"));
images = String(parser.get<string>("images"));
annotations = String(parser.get<string>("annotations"));
test_images = String(parser.get<string>("t"));
cout<<"cascade : "<<cascade.c_str()<<endl;
cout<<"model : "<<model.c_str()<<endl;
cout<<"images : "<<images.c_str()<<endl;
cout<<"annotations : "<<annotations.c_str()<<endl;
if(cascade.empty() || model.empty() || images.empty() || annotations.empty()){
std::cerr << "one or more required arguments are not found" << '\n';
parser.printMessage();
return false;
}
return true;
}
modules/face/samples/facemark_lbf_fitting.cpp
+198
View File
@@ -0,0 +1,198 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
/*----------------------------------------------
* Usage:
* facemark_lbf_fitting <face_cascade_model> <lbf_model> <video_name>
*
* example:
* facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4
*
* note: do not forget to provide the LBF_MODEL and DETECTOR_MODEL
* the model are available at opencv_contrib/modules/face/data/
*--------------------------------------------------*/
#include <stdio.h>
#include <ctime>
#include <iostream>
#include "opencv2/core.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/face.hpp"
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray ROIs, CascadeClassifier *face_cascade);
static bool parseArguments(int argc, char** argv,
String & cascade, String & model,String & video);
int main(int argc, char** argv ){
String cascade_path,model_path,images_path, video_path;
if(!parseArguments(argc, argv, cascade_path,model_path,video_path))
return -1;
CascadeClassifier face_cascade;
face_cascade.load(cascade_path);
FacemarkLBF::Params params;
params.model_filename = model_path;
params.cascade_face = cascade_path;
Ptr<FacemarkLBF> facemark = FacemarkLBF::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(params.model_filename.c_str());
VideoCapture capture(video_path);
Mat frame;
if( !capture.isOpened() ){
printf("Error when reading vide\n");
return 0;
}
Mat img;
String text;
char buff[255];
double fittime;
int nfaces;
std::vector<Rect> rects,rects_scaled;
std::vector<std::vector<Point2f> > landmarks;
CascadeClassifier cc(params.cascade_face.c_str());
namedWindow( "w", 1);
for( ; ; )
{
capture >> frame;
if(frame.empty())
break;
double __time__ = (double)getTickCount();
float scale = (float)(400.0/frame.cols);
resize(frame, img, Size((int)(frame.cols*scale), (int)(frame.rows*scale)), 0, 0, INTER_LINEAR_EXACT);
facemark->getFaces(img, rects);
rects_scaled.clear();
for(int j=0;j<(int)rects.size();j++){
rects_scaled.push_back(Rect(
(int)(rects[j].x/scale),
(int)(rects[j].y/scale),
(int)(rects[j].width/scale),
(int)(rects[j].height/scale)));
}
rects = rects_scaled;
fittime=0;
nfaces = (int)rects.size();
if(rects.size()>0){
double newtime = (double)getTickCount();
facemark->fit(frame, rects, landmarks);
fittime = ((getTickCount() - newtime)/getTickFrequency());
for(int j=0;j<(int)rects.size();j++){
landmarks[j] = Mat(Mat(landmarks[j]));
drawFacemarks(frame, landmarks[j], Scalar(0,0,255));
}
}
double fps = (getTickFrequency()/(getTickCount() - __time__));
sprintf(buff, "faces: %i %03.2f fps, fit:%03.0f ms",nfaces,fps,fittime*1000);
text = buff;
putText(frame, text, Point(20,40), FONT_HERSHEY_PLAIN , 2.0,Scalar::all(255), 2, 8);
imshow("w", frame);
waitKey(1); // waits to display frame
}
waitKey(0); // key press to close window
}
bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
bool parseArguments(int argc, char** argv,
String & cascade,
String & model,
String & video
){
const String keys =
"{ @c cascade | | (required) path to the cascade model file for the face detector }"
"{ @m model | | (required) path to the trained model }"
"{ @v video | | (required) path input video}"
"{ help h usage ? | | facemark_lbf_fitting -cascade -model -video [-t]\n"
" example: facemark_lbf_fitting ../face_cascade.xml ../LBF.model ../video.mp4}"
;
CommandLineParser parser(argc, argv,keys);
parser.about("hello");
if (parser.has("help")){
parser.printMessage();
return false;
}
cascade = String(parser.get<String>("cascade"));
model = String(parser.get<string>("model"));
video = String(parser.get<string>("video"));
if(cascade.empty() || model.empty() || video.empty() ){
std::cerr << "one or more required arguments are not found" << '\n';
cout<<"cascade : "<<cascade.c_str()<<endl;
cout<<"model : "<<model.c_str()<<endl;
cout<<"video : "<<video.c_str()<<endl;
parser.printMessage();
return false;
}
return true;
}
modules/face/samples/facerec_demo.cpp
+192
View File
@@ -0,0 +1,192 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/face.hpp"
#include "opencv2/core/utility.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
#include <map>
using namespace cv;
using namespace cv::face;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, std::map<int, string>& labelsInfo, char separator = ';') {
ifstream csv(filename.c_str());
if (!csv) CV_Error(Error::StsBadArg, "No valid input file was given, please check the given filename.");
string line, path, classlabel, info;
while (getline(csv, line)) {
stringstream liness(line);
path.clear(); classlabel.clear(); info.clear();
getline(liness, path, separator);
getline(liness, classlabel, separator);
getline(liness, info, separator);
if(!path.empty() && !classlabel.empty()) {
cout << "Processing " << path << endl;
int label = atoi(classlabel.c_str());
if(!info.empty())
labelsInfo.insert(std::make_pair(label, info));
// 'path' can be file, dir or wildcard path
String root(path.c_str());
vector<String> files;
glob(root, files, true);
for(vector<String>::const_iterator f = files.begin(); f != files.end(); ++f) {
cout << "\t" << *f << endl;
Mat img = imread(*f, IMREAD_GRAYSCALE);
static int w=-1, h=-1;
static bool showSmallSizeWarning = true;
if(w>0 && h>0 && (w!=img.cols || h!=img.rows)) cout << "\t* Warning: images should be of the same size!" << endl;
if(showSmallSizeWarning && (img.cols<50 || img.rows<50)) {
cout << "* Warning: for better results images should be not smaller than 50x50!" << endl;
showSmallSizeWarning = false;
}
images.push_back(img);
labels.push_back(label);
}
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 2 && argc != 3) {
cout << "Usage: " << argv[0] << " <csv> [arg2]\n"
<< "\t<csv> - path to config file in CSV format\n"
<< "\targ2 - if the 2nd argument is provided (with any value) "
<< "the advanced stuff is run and shown to console.\n"
<< "The CSV config file consists of the following lines:\n"
<< "<path>;<label>[;<comment>]\n"
<< "\t<path> - file, dir or wildcard path\n"
<< "\t<label> - non-negative integer person label\n"
<< "\t<comment> - optional comment string (e.g. person name)"
<< endl;
exit(1);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
std::map<int, string> labelsInfo;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels, labelsInfo);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(Error::StsError, error_message);
}
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::FaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int nlabels = (int)labels.size();
int testLabel = labels[nlabels-1];
images.pop_back();
labels.pop_back();
// The following lines create an Eigenfaces model for
// face recognition and train it with the images and
// labels read from the given CSV file.
// This here is a full PCA, if you just want to keep
// 10 principal components (read Eigenfaces), then call
// the factory method like this:
//
// EigenFaceRecognizer::create(10);
//
// If you want to create a FaceRecognizer with a
// confidennce threshold, call it with:
//
// EigenFaceRecognizer::create(10, 123.0);
//
Ptr<EigenFaceRecognizer> model = EigenFaceRecognizer::create();
for( int i = 0; i < nlabels; i++ )
model->setLabelInfo(i, labelsInfo[i]);
model->train(images, labels);
string saveModelPath = "face-rec-model.txt";
cout << "Saving the trained model to " << saveModelPath << endl;
model->save(saveModelPath);
// The following line predicts the label of a given
// test image:
int predictedLabel = model->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
if( (predictedLabel == testLabel) && !model->getLabelInfo(predictedLabel).empty() )
cout << format("%d-th label's info: %s", predictedLabel, model->getLabelInfo(predictedLabel).c_str()) << endl;
// advanced stuff
if(argc>2) {
// Sometimes you'll need to get/set internal model data,
// which isn't exposed by the public cv::FaceRecognizer.
// Since each cv::FaceRecognizer is derived from a
// cv::Algorithm, you can query the data.
//
// First we'll use it to set the threshold of the FaceRecognizer
// to 0.0 without retraining the model. This can be useful if
// you are evaluating the model:
//
model->setThreshold(0.0);
// Now the threshold of this model is set to 0.0. A prediction
// now returns -1, as it's impossible to have a distance below
// it
predictedLabel = model->predict(testSample);
cout << "Predicted class = " << predictedLabel << endl;
// Here is how to get the eigenvalues of this Eigenfaces model:
Mat eigenvalues = model->getEigenValues();
// And we can do the same to display the Eigenvectors (read Eigenfaces):
Mat W = model->getEigenVectors();
// From this we will display the (at most) first 10 Eigenfaces:
for (int i = 0; i < min(10, W.cols); i++) {
string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));
cout << msg << endl;
// get eigenvector #i
Mat ev = W.col(i).clone();
// Reshape to original size & normalize to [0...255] for imshow.
Mat grayscale;
normalize(ev.reshape(1), grayscale, 0, 255, NORM_MINMAX, CV_8UC1);
// Show the image & apply a Jet colormap for better sensing.
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_JET);
imshow(format("%d", i), cgrayscale);
}
waitKey(0);
}
return 0;
}
modules/face/samples/facerec_eigenfaces.cpp
+195
View File
@@ -0,0 +1,195 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static Mat norm_0_255(InputArray _src) {
Mat src = _src.getMat();
// Create and return normalized image:
Mat dst;
switch(src.channels()) {
case 1:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);
break;
case 3:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);
break;
default:
src.copyTo(dst);
break;
}
return dst;
}
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(Error::StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc < 2) {
cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;
exit(1);
}
string output_folder = ".";
if (argc == 3) {
output_folder = string(argv[2]);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(Error::StsError, error_message);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size:
int height = images[0].rows;
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::BasicFaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
images.pop_back();
labels.pop_back();
// The following lines create an Eigenfaces model for
// face recognition and train it with the images and
// labels read from the given CSV file.
// This here is a full PCA, if you just want to keep
// 10 principal components (read Eigenfaces), then call
// the factory method like this:
//
// EigenFaceRecognizer::create(10);
//
// If you want to create a FaceRecognizer with a
// confidence threshold (e.g. 123.0), call it with:
//
// EigenFaceRecognizer::create(10, 123.0);
//
// If you want to use _all_ Eigenfaces and have a threshold,
// then call the method like this:
//
// EigenFaceRecognizer::create(0, 123.0);
//
Ptr<EigenFaceRecognizer> model = EigenFaceRecognizer::create();
model->train(images, labels);
// The following line predicts the label of a given
// test image:
int predictedLabel = model->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
// Here is how to get the eigenvalues of this Eigenfaces model:
Mat eigenvalues = model->getEigenValues();
// And we can do the same to display the Eigenvectors (read Eigenfaces):
Mat W = model->getEigenVectors();
// Get the sample mean from the training data
Mat mean = model->getMean();
// Display or save:
if(argc == 2) {
imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));
} else {
imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));
}
// Display or save the Eigenfaces:
for (int i = 0; i < min(10, W.cols); i++) {
string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));
cout << msg << endl;
// get eigenvector #i
Mat ev = W.col(i).clone();
// Reshape to original size & normalize to [0...255] for imshow.
Mat grayscale = norm_0_255(ev.reshape(1, height));
// Show the image & apply a Jet colormap for better sensing.
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_JET);
// Display or save:
if(argc == 2) {
imshow(format("eigenface_%d", i), cgrayscale);
} else {
imwrite(format("%s/eigenface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));
}
}
// Display or save the image reconstruction at some predefined steps:
for(int num_components = min(W.cols, 10); num_components < min(W.cols, 300); num_components+=15) {
// slice the eigenvectors from the model
Mat evs = Mat(W, Range::all(), Range(0, num_components));
Mat projection = LDA::subspaceProject(evs, mean, images[0].reshape(1,1));
Mat reconstruction = LDA::subspaceReconstruct(evs, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);
} else {
imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);
}
}
// Display if we are not writing to an output folder:
if(argc == 2) {
waitKey(0);
}
return 0;
}
modules/face/samples/facerec_fisherfaces.cpp
+193
View File
@@ -0,0 +1,193 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static Mat norm_0_255(InputArray _src) {
Mat src = _src.getMat();
// Create and return normalized image:
Mat dst;
switch(src.channels()) {
case 1:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);
break;
case 3:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);
break;
default:
src.copyTo(dst);
break;
}
return dst;
}
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(Error::StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc < 2) {
cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;
exit(1);
}
string output_folder = ".";
if (argc == 3) {
output_folder = string(argv[2]);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(Error::StsError, error_message);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size:
int height = images[0].rows;
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::BasicFaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
images.pop_back();
labels.pop_back();
// The following lines create an Fisherfaces model for
// face recognition and train it with the images and
// labels read from the given CSV file.
// If you just want to keep 10 Fisherfaces, then call
// the factory method like this:
//
// FisherFaceRecognizer::create(10);
//
// However it is not useful to discard Fisherfaces! Please
// always try to use _all_ available Fisherfaces for
// classification.
//
// If you want to create a FaceRecognizer with a
// confidence threshold (e.g. 123.0) and use _all_
// Fisherfaces, then call it with:
//
// FisherFaceRecognizer::create(0, 123.0);
//
Ptr<FisherFaceRecognizer> model = FisherFaceRecognizer::create();
model->train(images, labels);
// The following line predicts the label of a given
// test image:
int predictedLabel = model->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
// Here is how to get the eigenvalues of this Eigenfaces model:
Mat eigenvalues = model->getEigenValues();
// And we can do the same to display the Eigenvectors (read Eigenfaces):
Mat W = model->getEigenVectors();
// Get the sample mean from the training data
Mat mean = model->getMean();
// Display or save:
if(argc == 2) {
imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));
} else {
imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));
}
// Display or save the first, at most 16 Fisherfaces:
for (int i = 0; i < min(16, W.cols); i++) {
string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));
cout << msg << endl;
// get eigenvector #i
Mat ev = W.col(i).clone();
// Reshape to original size & normalize to [0...255] for imshow.
Mat grayscale = norm_0_255(ev.reshape(1, height));
// Show the image & apply a Bone colormap for better sensing.
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_BONE);
// Display or save:
if(argc == 2) {
imshow(format("fisherface_%d", i), cgrayscale);
} else {
imwrite(format("%s/fisherface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));
}
}
// Display or save the image reconstruction at some predefined steps:
for(int num_component = 0; num_component < min(16, W.cols); num_component++) {
// Slice the Fisherface from the model:
Mat ev = W.col(num_component);
Mat projection = LDA::subspaceProject(ev, mean, images[0].reshape(1,1));
Mat reconstruction = LDA::subspaceReconstruct(ev, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("fisherface_reconstruction_%d", num_component), reconstruction);
} else {
imwrite(format("%s/fisherface_reconstruction_%d.png", output_folder.c_str(), num_component), reconstruction);
}
}
// Display if we are not writing to an output folder:
if(argc == 2) {
waitKey(0);
}
return 0;
}
modules/face/samples/facerec_lbph.cpp
+147
View File
@@ -0,0 +1,147 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(Error::StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 2) {
cout << "usage: " << argv[0] << " <csv.ext>" << endl;
exit(1);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(Error::StsError, error_message);
}
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::LBPHFaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
images.pop_back();
labels.pop_back();
// The following lines create an LBPH model for
// face recognition and train it with the images and
// labels read from the given CSV file.
//
// The LBPHFaceRecognizer uses Extended Local Binary Patterns
// (it's probably configurable with other operators at a later
// point), and has the following default values
//
// radius = 1
// neighbors = 8
// grid_x = 8
// grid_y = 8
//
// So if you want a LBPH FaceRecognizer using a radius of
// 2 and 16 neighbors, call the factory method with:
//
// cv::face::LBPHFaceRecognizer::create(2, 16);
//
// And if you want a threshold (e.g. 123.0) call it with its default values:
//
// cv::face::LBPHFaceRecognizer::create(1,8,8,8,123.0)
//
Ptr<LBPHFaceRecognizer> model = LBPHFaceRecognizer::create();
model->train(images, labels);
// The following line predicts the label of a given
// test image:
int predictedLabel = model->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
// First we'll use it to set the threshold of the LBPHFaceRecognizer
// to 0.0 without retraining the model. This can be useful if
// you are evaluating the model:
//
model->setThreshold(0.0);
// Now the threshold of this model is set to 0.0. A prediction
// now returns -1, as it's impossible to have a distance below
// it
predictedLabel = model->predict(testSample);
cout << "Predicted class = " << predictedLabel << endl;
// Show some informations about the model, as there's no cool
// Model data to display as in Eigenfaces/Fisherfaces.
// Due to efficiency reasons the LBP images are not stored
// within the model:
cout << "Model Information:" << endl;
string model_info = format("\tLBPH(radius=%i, neighbors=%i, grid_x=%i, grid_y=%i, threshold=%.2f)",
model->getRadius(),
model->getNeighbors(),
model->getGridX(),
model->getGridY(),
model->getThreshold());
cout << model_info << endl;
// We could get the histograms for example:
vector<Mat> histograms = model->getHistograms();
// But should I really visualize it? Probably the length is interesting:
cout << "Size of the histograms: " << histograms[0].total() << endl;
return 0;
}
modules/face/samples/facerec_save_load.cpp
+201
View File
@@ -0,0 +1,201 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static Mat norm_0_255(InputArray _src) {
Mat src = _src.getMat();
// Create and return normalized image:
Mat dst;
switch(src.channels()) {
case 1:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);
break;
case 3:
cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);
break;
default:
src.copyTo(dst);
break;
}
return dst;
}
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(Error::StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc < 2) {
cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;
exit(1);
}
string output_folder = ".";
if (argc == 3) {
output_folder = string(argv[2]);
}
// Get the path to your CSV.
string fn_csv = string(argv[1]);
// These vectors hold the images and corresponding labels.
vector<Mat> images;
vector<int> labels;
// Read in the data. This can fail if no valid
// input filename is given.
try {
read_csv(fn_csv, images, labels);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Quit if there are not enough images for this demo.
if(images.size() <= 1) {
string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";
CV_Error(Error::StsError, error_message);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size:
int height = images[0].rows;
// The following lines simply get the last images from
// your dataset and remove it from the vector. This is
// done, so that the training data (which we learn the
// cv::FaceRecognizer on) and the test data we test
// the model with, do not overlap.
Mat testSample = images[images.size() - 1];
int testLabel = labels[labels.size() - 1];
images.pop_back();
labels.pop_back();
// The following lines create an Eigenfaces model for
// face recognition and train it with the images and
// labels read from the given CSV file.
// This here is a full PCA, if you just want to keep
// 10 principal components (read Eigenfaces), then call
// the factory method like this:
//
// cv::face::EigenFaceRecognizer::create(10);
//
// If you want to create a FaceRecognizer with a
// confidence threshold (e.g. 123.0), call it with:
//
// cv::face::EigenFaceRecognizer::create(10, 123.0);
//
// If you want to use _all_ Eigenfaces and have a threshold,
// then call the method like this:
//
// cv::face::EigenFaceRecognizer::create(0, 123.0);
//
Ptr<EigenFaceRecognizer> model0 = EigenFaceRecognizer::create();
model0->train(images, labels);
// save the model to eigenfaces_at.yaml
model0->save("eigenfaces_at.yml");
//
//
// Now create a new Eigenfaces Recognizer
//
Ptr<EigenFaceRecognizer> model1 = Algorithm::load<EigenFaceRecognizer>("eigenfaces_at.yml");
// The following line predicts the label of a given
// test image:
int predictedLabel = model1->predict(testSample);
//
// To get the confidence of a prediction call the model with:
//
// int predictedLabel = -1;
// double confidence = 0.0;
// model->predict(testSample, predictedLabel, confidence);
//
string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);
cout << result_message << endl;
// Here is how to get the eigenvalues of this Eigenfaces model:
Mat eigenvalues = model1->getEigenValues();
// And we can do the same to display the Eigenvectors (read Eigenfaces):
Mat W = model1->getEigenVectors();
// Get the sample mean from the training data
Mat mean = model1->getMean();
// Display or save:
if(argc == 2) {
imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));
} else {
imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));
}
// Display or save the Eigenfaces:
for (int i = 0; i < min(10, W.cols); i++) {
string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));
cout << msg << endl;
// get eigenvector #i
Mat ev = W.col(i).clone();
// Reshape to original size & normalize to [0...255] for imshow.
Mat grayscale = norm_0_255(ev.reshape(1, height));
// Show the image & apply a Jet colormap for better sensing.
Mat cgrayscale;
applyColorMap(grayscale, cgrayscale, COLORMAP_JET);
// Display or save:
if(argc == 2) {
imshow(format("eigenface_%d", i), cgrayscale);
} else {
imwrite(format("%s/eigenface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));
}
}
// Display or save the image reconstruction at some predefined steps:
for(int num_components = 10; num_components < 300; num_components+=15) {
// slice the eigenvectors from the model
Mat evs = Mat(W, Range::all(), Range(0, num_components));
Mat projection = LDA::subspaceProject(evs, mean, images[0].reshape(1,1));
Mat reconstruction = LDA::subspaceReconstruct(evs, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);
} else {
imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);
}
}
// Display if we are not writing to an output folder:
if(argc == 2) {
waitKey(0);
}
return 0;
}
modules/face/samples/facerec_video.cpp
+153
View File
@@ -0,0 +1,153 @@
/*
* Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/xobjdetect.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
using namespace cv;
using namespace cv::face;
using namespace std;
static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(Error::StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if(!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
}
int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
if (argc != 4) {
cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>" << endl;
cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
exit(1);
}
// Get the path to your CSV:
string fn_haar = string(argv[1]);
string fn_csv = string(argv[2]);
int deviceId = atoi(argv[3]);
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
read_csv(fn_csv, images, labels);
} catch (const cv::Exception& e) {
cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
// nothing more we can do
exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FisherFaceRecognizer> model = FisherFaceRecognizer::create();
model->train(images, labels);
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:
//
CascadeClassifier haar_cascade;
haar_cascade.load(fn_haar);
// Get a handle to the Video device:
VideoCapture cap(deviceId);
// Check if we can use this device at all:
if(!cap.isOpened()) {
cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
return -1;
}
// Holds the current frame from the Video device:
Mat frame;
for(;;) {
cap >> frame;
// Clone the current frame:
Mat original = frame.clone();
// Convert the current frame to grayscale:
Mat gray;
cvtColor(original, gray, COLOR_BGR2GRAY);
// Find the faces in the frame:
vector< Rect_<int> > faces;
haar_cascade.detectMultiScale(gray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
for(size_t i = 0; i < faces.size(); i++) {
// Process face by face:
Rect face_i = faces[i];
// Crop the face from the image. So simple with OpenCV C++:
Mat face = gray(face_i);
// Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
// verify this, by reading through the face recognition tutorial coming with OpenCV.
// Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
// input data really depends on the algorithm used.
//
// I strongly encourage you to play around with the algorithms. See which work best
// in your scenario, LBPH should always be a contender for robust face recognition.
//
// Since I am showing the Fisherfaces algorithm here, I also show how to resize the
// face you have just found:
Mat face_resized;
cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
int prediction = model->predict(face_resized);
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
rectangle(original, face_i, Scalar(0, 255,0), 1);
// Create the text we will annotate the box with:
string box_text = format("Prediction = %d", prediction);
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = std::max(face_i.tl().x - 10, 0);
int pos_y = std::max(face_i.tl().y - 10, 0);
// And now put it into the image:
putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, Scalar(0,255,0), 2);
}
// Show the result:
imshow("face_recognizer", original);
// And display it:
char key = (char) waitKey(20);
// Exit this loop on escape:
if(key == 27)
break;
}
return 0;
}
modules/face/samples/landmarks_demo.py
+33
View File
@@ -0,0 +1,33 @@
import random
import numpy as np
import cv2 as cv
frame1 = cv.imread(cv.samples.findFile('lena.jpg'))
if frame1 is None:
print("image not found")
exit()
frame = np.vstack((frame1,frame1))
facemark = cv.face.createFacemarkLBF()
try:
facemark.loadModel(cv.samples.findFile('lbfmodel.yaml'))
except cv.error:
print("Model not found\nlbfmodel.yaml can be download at")
print("https://raw.githubusercontent.com/kurnianggoro/GSOC2017/master/data/lbfmodel.yaml")
cascade = cv.CascadeClassifier(cv.samples.findFile('lbpcascade_frontalface_improved.xml'))
if cascade.empty() :
print("cascade not found")
exit()
faces = cascade.detectMultiScale(frame, 1.05, 3, cv.CASCADE_SCALE_IMAGE, (30, 30))
if len(faces) == 0:
print('no faces detected')
landmarks = []
else:
ok, landmarks = facemark.fit(frame, faces=faces)
cv.imshow("Image", frame)
for marks in landmarks:
couleur = (random.randint(0,255),
random.randint(0,255),
random.randint(0,255))
cv.face.drawFacemarks(frame, marks, couleur)
cv.imshow("Image Landmarks", frame)
cv.waitKey()
modules/face/samples/mace_webcam.cpp
+137
View File
@@ -0,0 +1,137 @@
// This file is part of the OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "opencv2/videoio.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/xobjdetect.hpp"
#include "opencv2/face/mace.hpp"
#include <iostream>
using namespace cv;
using namespace cv::face;
using namespace std;
enum STATE {
NEUTRAL,
RECORD,
PREDICT
};
const char *help =
"press 'r' to record images. once N trainimages were recorded, train the mace filter\n"
"press 'p' to predict (twofactor mode will switch back to neutral after each prediction attempt)\n"
"press 's' to save a trained model\n"
"press 'esc' to return\n"
"any other key will reset to neutral state\n";
int main(int argc, char **argv) {
CommandLineParser parser(argc, argv,
"{ help h usage ? || show this help message }"
"{ cascade c || (required) path to a cascade file for face detection }"
"{ pre p || load a pretrained mace filter file, saved from previous session (e.g. my.xml.gz) }"
"{ num n |50| num train images }"
"{ size s |64| image size }"
"{ twofactor t || pass phrase(text) for 2 factor authentification.\n"
" (random convolute images seeded with the crc of this)\n"
" users will get prompted to guess the secrect, additional to the image. }"
);
String cascade = parser.get<String>("cascade");
if (parser.has("help") || cascade.empty()) {
parser.printMessage();
return 1;
} else {
cout << help << endl;
}
String defname = "mace.xml.gz";
String pre = parser.get<String>("pre");
String two = parser.get<String>("twofactor");
int N = parser.get<int>("num");
int Z = parser.get<int>("size");
int state = NEUTRAL;
Ptr<MACE> mace;
if (! pre.empty()) { // load pretrained model, if available
mace = MACE::load(pre);
if (mace->empty()) {
cerr << "loading the MACE failed !" << endl;
return -1;
}
state = PREDICT;
} else {
mace = MACE::create(Z);
if (! two.empty()) {
cout << "'" << two << "' initial passphrase" << endl;
mace->salt(two);
}
}
CascadeClassifier head(cascade);
if (head.empty()) {
cerr << "loading the cascade failed !" << endl;
return -2;
}
VideoCapture cap(0);
if (! cap.isOpened()) {
cerr << "VideoCapture could not be opened !" << endl;
return -3;
}
vector<Mat> train_img;
while(1) {
Mat frame;
cap >> frame;
vector<Rect> rects;
head.detectMultiScale(frame,rects);
if (rects.size()>0) {
Scalar col = Scalar(0,120,0);
if (state == RECORD) {
if (train_img.size() >= size_t(N)) {
mace->train(train_img);
train_img.clear();
state = PREDICT;
} else {
train_img.push_back(frame(rects[0]).clone());
}
col = Scalar(200,0,0);
}
if (state == PREDICT) {
if (! two.empty()) { // prompt for secret on console
cout << "enter passphrase: ";
string pass;
getline(cin, pass);
mace->salt(pass);
state = NEUTRAL;
cout << "'" << pass << "' : ";
}
bool same = mace->same(frame(rects[0]));
if (same) col = Scalar(0,220,220);
else col = Scalar(60,60,60);
if (! two.empty()) {
cout << (same ? "accepted." : "denied.") << endl;
}
}
rectangle(frame, rects[0], col, 2);
}
imshow("MACE",frame);
int k = waitKey(10);
switch (k) {
case -1 : break;
case 27 : return 0;
default : state = NEUTRAL; break;
case 'r': state = RECORD; break;
case 'p': state = PREDICT; break;
case 's': mace->save(defname); break;
}
}
return 0;
}
modules/face/samples/sampleDetectLandmarks.cpp
+100
View File
@@ -0,0 +1,100 @@
#include "opencv2/face.hpp"
#include "opencv2/videoio.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/xobjdetect.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ model_filename f | | (required) path to binary file storing the trained model which is to be loaded [example - /data/file.dat]}"
"{ image i | | (required) path to image in which face landmarks have to be detected.[example - /data/image.jpg] }"
"{ face_cascade c | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string filename(parser.get<string>("model_filename"));
if (filename.empty()){
parser.printMessage();
cerr << "The name of the model file to be loaded for detecting landmarks is not found" << endl;
return -1;
}
string image(parser.get<string>("image"));
if (image.empty()){
parser.printMessage();
cerr << "The name of the image file in which landmarks have to be detected is not found" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
Mat img = imread(image);
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
vector<Rect> faces;
resize(img,img,Size(460,460), 0, 0, INTER_LINEAR_EXACT);
facemark->getFaces(img,faces);
vector< vector<Point2f> > shapes;
// Check if faces detected or not
// Helps in proper exception handling when writing images to the directories.
if(faces.size() != 0) {
if(facemark->fit(img,faces,shapes))
{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],5,cv::Scalar(0,0,255),FILLED);
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
waitKey(0);
}
} else {
cout << "Faces not detected." << endl;
}
return 0;
}
modules/face/samples/sampleDetectLandmarksvideo.cpp
+110
View File
@@ -0,0 +1,110 @@
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/videoio.hpp"
#include "opencv2/xobjdetect.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ model_filename f | | (required) path to binary file storing the trained model which is to be loaded [example - /data/file.dat]}"
"{ video v | | (required) path to video in which face landmarks have to be detected.[example - /data/video.avi] }"
"{ face_cascade c | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string filename(parser.get<string>("model_filename"));
if (filename.empty()){
parser.printMessage();
cerr << "The name of the model file to be loaded for detecting landmarks is not found" << endl;
return -1;
}
string video(parser.get<string>("video"));
if (video.empty()){
parser.printMessage();
cerr << "The name of the video file in which landmarks have to be detected is not found" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
VideoCapture cap(video);
if(!cap.isOpened()){
cerr<<"Video cannot be loaded. Give correct path"<<endl;
return -1;
}
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
//vector to store the faces detected in the image
vector<Rect> faces;
vector< vector<Point2f> > shapes;
Mat img;
while(1){
faces.clear();
shapes.clear();
cap>>img;
//Detect faces in the current image
resize(img,img,Size(600,600), 0, 0, INTER_LINEAR_EXACT);
facemark->getFaces(img,faces);
if(faces.size()==0){
cout<<"No faces found in this frame"<<endl;
}
else{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
//vector to store the landmarks of all the faces in the image
if(facemark->fit(img,faces,shapes))
{
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],3,cv::Scalar(0,0,255),FILLED);
}
}
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
if(waitKey(1) >= 0) break;
}
return 0;
}
modules/face/samples/sample_config_file.xml
+20
View File
@@ -0,0 +1,20 @@
<?xml version="1.0"?>
<!-- cascade_depth stores the depth of cascade of regressors used for training.
tree_depth stores the depth of trees created as weak learners during gradient boosting.
num_trees_per_cascade_level stores number of trees required per cascade level.
learning_rate stores the learning rate for gradient boosting.This is required to prevent overfitting using shrinkage.
oversampling_amount stores the oversampling amount for the samples.
num_test_coordinates stores number of test coordinates to be generated as samples to decide for making the split.
lambda stores the value used for calculating the probabilty which helps to select closer pixels for making the split.
num_test_splits stores the number of test splits to be generated before making the best split.
-->
<opencv_storage>
<cascade_depth>15</cascade_depth>
<tree_depth>4</tree_depth>
<num_trees_per_cascade_level>500</num_trees_per_cascade_level>
<learning_rate>1.0000000149011612e-01</learning_rate>
<oversampling_amount>20</oversampling_amount>
<num_test_coordinates>400</num_test_coordinates>
<lambda>1.0000000149011612e-01</lambda>
<num_test_splits>20</num_test_splits>
</opencv_storage>
modules/face/samples/sample_face_swapping.cpp
+205
View File
@@ -0,0 +1,205 @@
#include "opencv2/face.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/xobjdetect.hpp"
#include "opencv2/photo.hpp" // seamlessClone()
#include "opencv2/geometry.hpp" // Subdiv2D()
#include <iostream>
using namespace cv;
using namespace cv::face;
using namespace std;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
void divideIntoTriangles(Rect rect, vector<Point2f> &points, vector< vector<int> > &delaunayTri);
void warpTriangle(Mat &img1, Mat &img2, vector<Point2f> &triangle1, vector<Point2f> &triangle2);
//Divide the face into triangles for warping
void divideIntoTriangles(Rect rect, vector<Point2f> &points, vector< vector<int> > &Tri){
// Create an instance of Subdiv2D
Subdiv2D subdiv(rect);
// Insert points into subdiv
for( vector<Point2f>::iterator it = points.begin(); it != points.end(); it++)
subdiv.insert(*it);
vector<Vec6f> triangleList;
subdiv.getTriangleList(triangleList);
vector<Point2f> pt(3);
vector<int> ind(3);
for( size_t i = 0; i < triangleList.size(); i++ )
{
Vec6f triangle = triangleList[i];
pt[0] = Point2f(triangle[0], triangle[1]);
pt[1] = Point2f(triangle[2], triangle[3]);
pt[2] = Point2f(triangle[4], triangle[5]);
// Workaround for https://github.com/opencv/opencv/issues/26016
// To keep its behaviour, pt casts to Point_<int>.
if ( rect.contains(Point_<int>(pt[0])) && rect.contains(Point_<int>(pt[1])) && rect.contains(Point_<int>(pt[2]))){
for(int j = 0; j < 3; j++)
for(size_t k = 0; k < points.size(); k++)
if(abs(pt[j].x - points[k].x) < 1.0 && abs(pt[j].y - points[k].y) < 1)
ind[j] =(int) k;
Tri.push_back(ind);
}
}
}
void warpTriangle(Mat &img1, Mat &img2, vector<Point2f> &triangle1, vector<Point2f> &triangle2)
{
Rect rectangle1 = boundingRect(triangle1);
Rect rectangle2 = boundingRect(triangle2);
// Offset points by left top corner of the respective rectangles
vector<Point2f> triangle1Rect, triangle2Rect;
vector<Point> triangle2RectInt;
for(int i = 0; i < 3; i++)
{
triangle1Rect.push_back( Point2f( triangle1[i].x - rectangle1.x, triangle1[i].y - rectangle1.y) );
triangle2Rect.push_back( Point2f( triangle2[i].x - rectangle2.x, triangle2[i].y - rectangle2.y) );
triangle2RectInt.push_back( Point((int)(triangle2[i].x - rectangle2.x),(int) (triangle2[i].y - rectangle2.y))); // for fillConvexPoly
}
// Get mask by filling triangle
Mat mask = Mat::zeros(rectangle2.height, rectangle2.width, CV_32FC3);
fillConvexPoly(mask, triangle2RectInt, Scalar(1.0, 1.0, 1.0), 16, 0);
// Apply warpImage to small rectangular patches
Mat img1Rect;
img1(rectangle1).copyTo(img1Rect);
Mat img2Rect = Mat::zeros(rectangle2.height, rectangle2.width, img1Rect.type());
Mat warp_mat = getAffineTransform(triangle1Rect, triangle2Rect);
warpAffine( img1Rect, img2Rect, warp_mat, img2Rect.size(), INTER_LINEAR, BORDER_REFLECT_101);
multiply(img2Rect,mask, img2Rect);
multiply(img2(rectangle2), Scalar(1.0,1.0,1.0) - mask, img2(rectangle2));
img2(rectangle2) = img2(rectangle2) + img2Rect;
}
int main( int argc, char** argv)
{
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ image1 i1 | | (required) path to the first image file in which you want to apply swapping }"
"{ image2 i2 | | (required) path to the second image file in which you want to apply face swapping }"
"{ model m | | (required) path to the file containing model to be loaded for face landmark detection}"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
Mat img1=imread(parser.get<string>("image1"));
Mat img2=imread(parser.get<string>("image2"));
if (img1.empty()||img2.empty()){
if(img1.empty()){
parser.printMessage();
cerr << parser.get<string>("image1")<<" not found" << endl;
return -1;
}
if (img2.empty()){
parser.printMessage();
cerr << parser.get<string>("image2")<<" not found" << endl;
return -1;
}
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "Model file name not found." << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
facemark->loadModel(modelfile_name);
cout<<"Loaded model"<<endl;
//vector to store the faces detected in the image
vector<Rect> faces1,faces2;
vector< vector<Point2f> > shape1,shape2;
//Detect faces in the current image
float ratio1 = (float)img1.cols/(float)img1.rows;
float ratio2 = (float)img2.cols/(float)img2.rows;
resize(img1,img1,Size((int)(640*ratio1),(int)(640*ratio1)), 0, 0, INTER_LINEAR_EXACT);
resize(img2,img2,Size((int)(640*ratio2),(int)(640*ratio2)), 0, 0, INTER_LINEAR_EXACT);
Mat img1Warped = img2.clone();
facemark->getFaces(img1,faces1);
facemark->getFaces(img2,faces2);
//Initialise the shape of the faces
facemark->fit(img1,faces1,shape1);
facemark->fit(img2,faces2,shape2);
unsigned long numswaps = (unsigned long)min((unsigned long)shape1.size(),(unsigned long)shape2.size());
for(unsigned long z=0;z<numswaps;z++){
vector<Point2f> points1 = shape1[z];
vector<Point2f> points2 = shape2[z];
img1.convertTo(img1, CV_32F);
img1Warped.convertTo(img1Warped, CV_32F);
// Find convex hull
vector<Point2f> boundary_image1;
vector<Point2f> boundary_image2;
vector<int> index;
convexHull(Mat(points2),index, false, false);
for(size_t i = 0; i < index.size(); i++)
{
boundary_image1.push_back(points1[index[i]]);
boundary_image2.push_back(points2[index[i]]);
}
// Triangulation for points on the convex hull
vector< vector<int> > triangles;
Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
divideIntoTriangles(rect, boundary_image2, triangles);
// Apply affine transformation to Delaunay triangles
for(size_t i = 0; i < triangles.size(); i++)
{
vector<Point2f> triangle1, triangle2;
// Get points for img1, img2 corresponding to the triangles
for(int j = 0; j < 3; j++)
{
triangle1.push_back(boundary_image1[triangles[i][j]]);
triangle2.push_back(boundary_image2[triangles[i][j]]);
}
warpTriangle(img1, img1Warped, triangle1, triangle2);
}
// Calculate mask
vector<Point> hull;
for(size_t i = 0; i < boundary_image2.size(); i++)
{
Point pt((int)boundary_image2[i].x,(int)boundary_image2[i].y);
hull.push_back(pt);
}
Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
fillConvexPoly(mask,&hull[0],(int)hull.size(), Scalar(255,255,255));
// Clone seamlessly.
Rect r = boundingRect(boundary_image2);
Point center = (r.tl() + r.br()) / 2;
Mat output;
img1Warped.convertTo(img1Warped, CV_8UC3);
seamlessClone(img1Warped,img2, mask, center, output, NORMAL_CLONE);
imshow("Face_Swapped", output);
waitKey(0);
destroyAllWindows();
}
return 0;
}
modules/face/samples/sample_train_landmark_detector.cpp
+117
View File
@@ -0,0 +1,117 @@
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/xobjdetect.hpp"
#include "opencv2/imgproc.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ annotations a |. | (required) path to annotations txt file [example - /data/annotations.txt] }"
"{ config c | | (required) path to configuration xml file containing parameters for training.[ example - /data/config.xml] }"
"{ model m | | (required) path to configuration xml file containing parameters for training.[ example - /data/model.dat] }"
"{ width w | 460 | The width which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ height h | 460 | The height which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
//Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string directory(parser.get<string>("annotations"));
//default initialisation
Size scale(460,460);
scale = Size(parser.get<int>("width"),parser.get<int>("height"));
if (directory.empty()){
parser.printMessage();
cerr << "The name of the directory from which annotations have to be found is empty" << endl;
return -1;
}
string configfile_name(parser.get<string>("config"));
if (configfile_name.empty()){
parser.printMessage();
cerr << "No configuration file name found which contains the parameters for training" << endl;
return -1;
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "No name for the model_file found in which the trained model has to be saved" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a vector to store names of files in which annotations
//and image names are found
/*The format of the file containing annotations should be of following format
/data/abc/abc.jpg
123.45,345.65
321.67,543.89
The above format is similar to HELEN dataset which is used for training model
*/
vector<String> filenames;
//reading the files from the given directory
glob(directory + "*.txt",filenames);
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
//create a vector to store image names
vector<String> imagenames;
//create object to get landmarks
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
//gets landmarks and corresponding image names in both the vectors
//vector to store images
vector<Mat> trainimages;
loadTrainingData(filenames,trainlandmarks,imagenames);
for(unsigned long i=0;i<300;i++){
string imgname = imagenames[i].substr(0, imagenames[i].size()-1);
string img = directory + string(imgname) + ".jpg";
Mat src = imread(img);
if(src.empty()){
cerr<<string("Image "+img+" not found\n.")<<endl;
continue;
}
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
cout<<"Got data"<<endl;
facemark->training(trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
cout<<"Training complete"<<endl;
return 0;
}
modules/face/samples/sample_train_landmark_detector2.cpp
+134
View File
@@ -0,0 +1,134 @@
/*----------------------------------------------
* the user should provide the list of training images_train,
* accompanied by their corresponding landmarks location in separated files.
* example of contents for images.txt:
* ../trainset/image_0001.png
* ../trainset/image_0002.png
* example of contents for annotation.txt:
* ../trainset/image_0001.pts
* ../trainset/image_0002.pts
* where the image_xxxx.pts contains the position of each face landmark.
* example of the contents:
* version: 1
* n_points: 68
* {
* 115.167660 220.807529
* 116.164839 245.721357
* 120.208690 270.389841
* ...
* }
* example of the dataset is available at https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
*--------------------------------------------------*/
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/xobjdetect.hpp"
#include <iostream>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
using namespace cv::face;
static bool myDetector(InputArray image, OutputArray faces, CascadeClassifier *face_cascade)
{
Mat gray;
if (image.channels() > 1)
cvtColor(image, gray, COLOR_BGR2GRAY);
else
gray = image.getMat().clone();
equalizeHist(gray, gray);
std::vector<Rect> faces_;
face_cascade->detectMultiScale(gray, faces_, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(faces_).copyTo(faces);
return true;
}
int main(int argc,char** argv){
//Give the path to the directory containing all the files containing data
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ images i | | (required) path to images txt file [example - /data/images.txt] }"
"{ annotations a |. | (required) path to annotations txt file [example - /data/annotations.txt] }"
"{ config c | | (required) path to configuration xml file containing parameters for training.[example - /data/config.xml] }"
"{ model m | | (required) path to file containing trained model for face landmark detection[example - /data/model.dat] }"
"{ width w | 460 | The width which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ height h | 460 | The height which you want all images to get to scale the annotations. large images are slow to process [default = 460] }"
"{ face_cascade f | | Path to the face cascade xml file which you want to use as a detector}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
string annotations(parser.get<string>("annotations"));
string imagesList(parser.get<string>("images"));
//default initialisation
Size scale(460,460);
scale = Size(parser.get<int>("width"),parser.get<int>("height"));
if (annotations.empty()){
parser.printMessage();
cerr << "Name for annotations file not found. Aborting...." << endl;
return -1;
}
if (imagesList.empty()){
parser.printMessage();
cerr << "Name for file containing image list not found. Aborting....." << endl;
return -1;
}
string configfile_name(parser.get<string>("config"));
if (configfile_name.empty()){
parser.printMessage();
cerr << "No configuration file name found which contains the parameters for training" << endl;
return -1;
}
string modelfile_name(parser.get<string>("model"));
if (modelfile_name.empty()){
parser.printMessage();
cerr << "No name for the model_file found in which the trained model has to be saved" << endl;
return -1;
}
string cascade_name(parser.get<string>("face_cascade"));
if (cascade_name.empty()){
parser.printMessage();
cerr << "The name of the cascade classifier to be loaded to detect faces is not found" << endl;
return -1;
}
//create a pointer to call the base class
//pass the face cascade xml file which you want to pass as a detector
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imagesList, annotations, images, facePoints, 0.0);
//gets landmarks and corresponding image names in both the vectors
vector<Mat> Trainimages;
std::vector<std::vector<Point2f> > Trainlandmarks;
//vector to store images
Mat src;
for(unsigned long i=0;i<images.size();i++){
src = imread(images[i]);
if(src.empty()){
cout<<images[i]<<endl;
cerr<<string("Image not found\n.Aborting...")<<endl;
continue;
}
Trainimages.push_back(src);
Trainlandmarks.push_back(facePoints[i]);
}
cout<<"Got data"<<endl;
facemark->training(Trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
cout<<"Training complete"<<endl;
return 0;
}
modules/face/samples/samplewriteconfigfile.cpp
+63
View File
@@ -0,0 +1,63 @@
#include "opencv2/core.hpp"
#include <iostream>
#include <string>
using namespace cv;
using namespace std;
int main(int argc,const char ** argv){
CommandLineParser parser(argc, argv,
"{ help h usage ? | | give the following arguments in following format }"
"{ filename f |. | (required) path to file which you want to create as config file [example - /data/config.xml] }"
"{ cascade_depth cd | 10 | (required) This stores the depth of cascade of regressors used for training.}"
"{ tree_depth td | 4 | (required) This stores the depth of trees created as weak learners during gradient boosting.}"
"{ num_trees_per_cascade_level| 500 | (required) This stores number of trees required per cascade level.}"
"{ learning_rate | 0.1 | (required) This stores the learning rate for gradient boosting.}"
"{ oversampling_amount | 20 | (required) This stores the oversampling amount for the samples.}"
"{ num_test_coordinates | 400 | (required) This stores number of test coordinates required for making the split.}"
"{ lambda | 0.1 | (required) This stores the value used for calculating the probabilty.}"
"{ num_test_splits | 20 | (required) This stores the number of test splits to be generated before making the best split.}"
);
// Read in the input arguments
if (parser.has("help")){
parser.printMessage();
cerr << "TIP: Use absolute paths to avoid any problems with the software!" << endl;
return 0;
}
//These variables have been initialised as defined in the research paper "One millisecond face alignment" CVPR 2014
int cascade_depth = 15;
int tree_depth = 4;
int num_trees_per_cascade_level = 500;
float learning_rate = float(0.1);
int oversampling_amount = 20;
int num_test_coordinates = 400;
float lambda = float(0.1);
int num_test_splits = 20;
cascade_depth = parser.get<int>("cascade_depth");
tree_depth = parser.get<int>("tree_depth");
num_trees_per_cascade_level = parser.get<int>("num_trees_per_cascade_level");
learning_rate = parser.get<float>("learning_rate");
oversampling_amount = parser.get<int>("oversampling_amount");
num_test_coordinates = parser.get<int>("num_test_coordinates");
lambda = parser.get<float>("lambda");
num_test_splits = parser.get<int>("num_test_splits");
string filename(parser.get<string>("filename"));
FileStorage fs(filename, FileStorage::WRITE);
if (!fs.isOpened())
{
cerr << "Failed to open " << filename << endl;
parser.printMessage();
return -1;
}
fs << "cascade_depth" << cascade_depth;
fs << "tree_depth"<< tree_depth;
fs << "num_trees_per_cascade_level" << num_trees_per_cascade_level;
fs << "learning_rate" << learning_rate;
fs << "oversampling_amount" << oversampling_amount;
fs << "num_test_coordinates" << num_test_coordinates;
fs << "lambda" << lambda ;
fs << "num_test_splits"<< num_test_splits;
fs.release();
cout << "Write Done." << endl;
return 0;
}
modules/face/src/bif.cpp
+221
View File
@@ -0,0 +1,221 @@
/*
By downloading, copying, installing or using the software you agree to this license.
If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2000-2015, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2015, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015, OpenCV Foundation, all rights reserved.
Copyright (C) 2015, Itseez Inc., all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are disclaimed.
In no event shall copyright holders or contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
/*
This file contains implementation of the bio-inspired features (BIF) approach
for computing image descriptors, applicable for human age estimation. For more
details we refer to [1,2].
REFERENCES
[1] Guo, Guodong, et al. "Human age estimation using bio-inspired features."
Computer Vision and Pattern Recognition, 2009. CVPR 2009.
[2] Spizhevoi, A. S., and A. V. Bovyrin. "Estimating human age using
bio-inspired features and the ranking method." Pattern Recognition and
Image Analysis 25.3 (2015): 547-552.
*/
#include "precomp.hpp"
#include "opencv2/face/bif.hpp"
#include <iostream>
#include <vector>
namespace {
// The constants below are taken from paper [1].
const int kNumBandsMax = 8;
const cv::Size kCellSizes[kNumBandsMax] = {
cv::Size(6,6), cv::Size(8,8), cv::Size(10,10), cv::Size(12,12),
cv::Size(14,14), cv::Size(16,16), cv::Size(18,18), cv::Size(20,20)
};
const cv::Size kGaborSize[kNumBandsMax][2] = {
{cv::Size(5,5), cv::Size(7,7)}, {cv::Size(9,9), cv::Size(11,11)},
{cv::Size(13,13), cv::Size(15,15)}, {cv::Size(17,17), cv::Size(19,19)},
{cv::Size(21,21), cv::Size(23,23)}, {cv::Size(25,25), cv::Size(27,27)},
{cv::Size(29,29), cv::Size(31,31)}, {cv::Size(33,33), cv::Size(35,35)}
};
const double kGaborGamma = 0.3;
const double kGaborSigmas[kNumBandsMax][2] = {
{2.0, 2.8}, {3.6, 4.5}, {5.4, 6.3}, {7.3, 8.2},
{9.2, 10.2}, {11.3, 12.3}, {13.4, 14.6}, {15.8, 17.0}
};
const double kGaborWavelens[kNumBandsMax][2] = {
{2.5, 3.5}, {4.6, 5.6}, {6.8, 7.9}, {9.1, 10.3},
{11.5, 12.7}, {14.1, 15.4}, {16.8, 18.2}, {19.7, 21.2}
};
class BIFImpl CV_FINAL : public cv::face::BIF {
public:
BIFImpl(int num_bands, int num_rotations) {
initUnits(num_bands, num_rotations);
}
virtual int getNumBands() const CV_OVERRIDE { return num_bands_; }
virtual int getNumRotations() const CV_OVERRIDE { return num_rotations_; }
virtual void compute(cv::InputArray image,
cv::OutputArray features) const CV_OVERRIDE;
private:
struct UnitParams {
cv::Size cell_size;
cv::Mat filter1, filter2;
};
void initUnits(int num_bands, int num_rotations);
void computeUnit(int unit_idx, const cv::Mat &img, cv::Mat &dst) const;
int num_bands_;
int num_rotations_;
std::vector<UnitParams> units_;
};
void BIFImpl::compute(cv::InputArray _image,
cv::OutputArray _features) const {
cv::Mat image = _image.getMat();
CV_Assert(image.type() == CV_32F);
std::vector<cv::Mat> fea_units(units_.size());
int fea_dim = 0;
for (size_t i = 0; i < units_.size(); ++i) {
computeUnit(static_cast<int>(i), image, fea_units[i]);
fea_dim += fea_units[i].rows;
}
_features.create(fea_dim, 1, CV_32F);
cv::Mat fea = _features.getMat();
int offset = 0;
for (size_t i = 0; i < fea_units.size(); ++i) {
cv::Mat roi = fea.rowRange(offset, offset + fea_units[i].rows);
fea_units[i].copyTo(roi);
offset += fea_units[i].rows;
}
CV_Assert(offset == fea_dim);
}
void BIFImpl::initUnits(int num_bands, int num_rotations) {
CV_Assert(num_bands > 0 && num_bands <= kNumBandsMax);
CV_Assert(num_rotations > 0);
num_bands_ = num_bands;
num_rotations_ = num_rotations;
for (int ri = 0; ri < num_rotations; ++ri) {
double angle = CV_PI / num_rotations * ri;
for (int bi = 0; bi < num_bands; ++bi) {
cv::Mat kernel[2];
for (int i = 0; i < 2; ++i) {
kernel[i] = cv::getGaborKernel(
kGaborSize[bi][i], kGaborSigmas[bi][i], angle,
kGaborWavelens[bi][i], kGaborGamma, 0, CV_32F);
// Make variance for the Gaussian part of the Gabor filter
// the same across all filters.
kernel[i] /= 2 * kGaborSigmas[bi][i] * kGaborSigmas[bi][i]
/ kGaborGamma;
}
UnitParams unit;
unit.cell_size = kCellSizes[bi];
unit.filter1 = kernel[0];
unit.filter2 = kernel[1];
units_.push_back(unit);
}
}
}
void BIFImpl::computeUnit(int unit_idx, const cv::Mat &img,
cv::Mat &dst) const {
cv::Mat resp1, resp2;
cv::filter2D(img, resp1, CV_32F, units_[unit_idx].filter1);
cv::filter2D(img, resp2, CV_32F, units_[unit_idx].filter2);
cv::Mat resp, sum, sumsq;
cv::max(resp1, resp2, resp);
cv::integral(resp, sum, sumsq);
int Hhalf = units_[unit_idx].cell_size.height / 2;
int Whalf = units_[unit_idx].cell_size.width / 2;
int nrows = (resp.rows + Hhalf - 1) / Hhalf;
int ncols = (resp.cols + Whalf - 1) / Whalf;
dst.create(nrows*ncols, 1, CV_32F);
for (int pos = 0, yc = 0; yc < resp.rows; yc += Hhalf) {
int y0 = std::max(0, yc - Hhalf);
int y1 = std::min(resp.rows, yc + Hhalf);
for (int xc = 0; xc < resp.cols; xc += Whalf, ++pos) {
int x0 = std::max(0, xc - Whalf);
int x1 = std::min(resp.cols, xc + Whalf);
int area = (y1-y0) * (x1-x0);
double mean = sum.at<double>(y1,x1) - sum.at<double>(y1,x0)
- sum.at<double>(y0,x1) + sum.at<double>(y0,x0);
mean /= area;
double sd = sumsq.at<double>(y1,x1) - sumsq.at<double>(y1,x0)
- sumsq.at<double>(y0,x1) + sumsq.at<double>(y0,x0);
sd = sqrt(std::max(0.0, sd / area - mean * mean));
dst.at<float>(pos) = static_cast<float>(sd);
}
}
}
} // namespace
cv::Ptr<cv::face::BIF> cv::face::BIF::create(int num_bands, int num_rotations) {
return cv::Ptr<cv::face::BIF>(new BIFImpl(num_bands, num_rotations));
}
modules/face/src/eigen_faces.cpp
+139
View File
@@ -0,0 +1,139 @@
/*
* Copyright (c) 2011,2012. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "precomp.hpp"
#include <opencv2/face.hpp>
#include "face_utils.hpp"
#include <set>
#include <limits>
#include <iostream>
namespace cv
{
namespace face
{
// Turk, M., and Pentland, A. "Eigenfaces for recognition.". Journal of
// Cognitive Neuroscience 3 (1991), 71–86.
class Eigenfaces : public EigenFaceRecognizer
{
public:
// Initializes an empty Eigenfaces model.
Eigenfaces(int num_components = 0, double threshold = DBL_MAX)
//: BasicFaceRecognizerImpl(num_components, threshold)
{
_num_components = num_components;
_threshold = threshold;
}
// Computes an Eigenfaces model with images in src and corresponding labels
// in labels.
void train(InputArrayOfArrays src, InputArray labels) CV_OVERRIDE;
// Send all predict results to caller side for custom result handling
void predict(InputArray src, Ptr<PredictCollector> collector) const CV_OVERRIDE;
String getDefaultName() const CV_OVERRIDE
{
return "opencv_eigenfaces";
}
};
//------------------------------------------------------------------------------
// Eigenfaces
//------------------------------------------------------------------------------
void Eigenfaces::train(InputArrayOfArrays _src, InputArray _local_labels) {
if(_src.total() == 0) {
String error_message = format("Empty training data was given. You'll need more than one sample to learn a model.");
CV_Error(Error::StsBadArg, error_message);
} else if(_local_labels.getMat().type() != CV_32SC1) {
String error_message = format("Labels must be given as integer (CV_32SC1). Expected %d, but was %d.", CV_32SC1, _local_labels.type());
CV_Error(Error::StsBadArg, error_message);
}
// make sure data has correct size
if(_src.total() > 1) {
for(int i = 1; i < static_cast<int>(_src.total()); i++) {
if(_src.getMat(i-1).total() != _src.getMat(i).total()) {
String error_message = format("In the Eigenfaces method all input samples (training images) must be of equal size! Expected %zu pixels, but was %zu pixels.", _src.getMat(i-1).total(), _src.getMat(i).total());
CV_Error(Error::StsUnsupportedFormat, error_message);
}
}
}
// get labels
Mat labels = _local_labels.getMat();
// observations in row
Mat data = asRowMatrix(_src, CV_64FC1);
// number of samples
int n = data.rows;
// assert there are as much samples as labels
if(static_cast<int>(labels.total()) != n) {
String error_message = format("The number of samples (src) must equal the number of labels (labels)! len(src)=%d, len(labels)=%zu.", n, labels.total());
CV_Error(Error::StsBadArg, error_message);
}
// clear existing model data
_labels.release();
_projections.clear();
// clip number of components to be valid
if((_num_components <= 0) || (_num_components > n))
_num_components = n;
// perform the PCA
PCA pca(data, Mat(), PCA::DATA_AS_ROW, _num_components);
// copy the PCA results
_mean = pca.mean.reshape(1,1); // store the mean vector
_eigenvalues = pca.eigenvalues.clone(); // eigenvalues by row
transpose(pca.eigenvectors, _eigenvectors); // eigenvectors by column
// store labels for prediction
_labels = labels.clone();
// save projections
for(int sampleIdx = 0; sampleIdx < data.rows; sampleIdx++) {
Mat p = LDA::subspaceProject(_eigenvectors, _mean, data.row(sampleIdx));
_projections.push_back(p);
}
}
void Eigenfaces::predict(InputArray _src, Ptr<PredictCollector> collector) const {
// get data
Mat src = _src.getMat();
// make sure the user is passing correct data
if(_projections.empty()) {
// throw error if no data (or simply return -1?)
String error_message = "This Eigenfaces model is not computed yet. Did you call Eigenfaces::train?";
CV_Error(Error::StsError, error_message);
} else if(_eigenvectors.rows != static_cast<int>(src.total())) {
// check data alignment just for clearer exception messages
String error_message = format("Wrong input image size. Reason: Training and Test images must be of equal size! Expected an image with %d elements, but got %zu.", _eigenvectors.rows, src.total());
CV_Error(Error::StsBadArg, error_message);
}
// project into PCA subspace
Mat q = LDA::subspaceProject(_eigenvectors, _mean, src.reshape(1, 1));
collector->init(_projections.size());
for (size_t sampleIdx = 0; sampleIdx < _projections.size(); sampleIdx++) {
double dist = norm(_projections[sampleIdx], q, NORM_L2);
int label = _labels.at<int>((int)sampleIdx);
if (!collector->collect(label, dist))return;
}
}
Ptr<EigenFaceRecognizer> EigenFaceRecognizer::create(int num_components, double threshold)
{
return makePtr<Eigenfaces>(num_components, threshold);
}
}
}
modules/face/src/face_alignment.cpp
+194
View File
@@ -0,0 +1,194 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
#include <vector>
using namespace std;
namespace cv{
namespace face{
FacemarkKazemi::~FacemarkKazemi(){}
FacemarkKazemiImpl:: ~FacemarkKazemiImpl(){}
unsigned long FacemarkKazemiImpl::left(unsigned long index){
return 2*index+1;
}
unsigned long FacemarkKazemiImpl::right(unsigned long index){
return 2*index+2;
}
bool FacemarkKazemiImpl::setFaceDetector(FN_FaceDetector f, void* userData){
faceDetector = f;
faceDetectorData = userData;
//printf("face detector is configured\n");
return true;
}
bool FacemarkKazemiImpl::getFaces(InputArray image, OutputArray faces)
{
CV_Assert(faceDetector);
return faceDetector(image, faces, faceDetectorData);
}
FacemarkKazemiImpl::FacemarkKazemiImpl(const FacemarkKazemi::Params& parameters) :
faceDetector(NULL),
faceDetectorData(NULL)
{
minmeanx=8000.0;
maxmeanx=0.0;
minmeany=8000.0;
maxmeany=0.0;
isModelLoaded =false;
params = parameters;
}
FacemarkKazemi::Params::Params(){
//These variables are used for training data
//These are initialised as described in the research paper
//referenced above
cascade_depth = 15;
tree_depth = 5;
num_trees_per_cascade_level = 500;
learning_rate = float(0.1);
oversampling_amount = 20;
num_test_coordinates = 500;
lambda = float(0.1);
num_test_splits = 20;
}
bool FacemarkKazemiImpl::convertToActual(Rect r,Mat &warp){
Point2f srcTri[3],dstTri[3];
srcTri[0]=Point2f(0,0);
srcTri[1]=Point2f(1,0);
srcTri[2]=Point2f(0,1);
dstTri[0]=Point2f((float)r.x,(float)r.y);
dstTri[1]=Point2f((float)r.x+r.width,(float)r.y);
dstTri[2]=Point2f((float)r.x,(float)r.y+(float)1.3*r.height);
warp=getAffineTransform(srcTri,dstTri);
return true;
}
bool FacemarkKazemiImpl::convertToUnit(Rect r,Mat &warp){
Point2f srcTri[3],dstTri[3];
dstTri[0]=Point2f(0,0);
dstTri[1]=Point2f(1,0);
dstTri[2]=Point2f(0,1);
srcTri[0]=Point2f((float)r.x,(float)r.y);
srcTri[1]=Point2f((float)r.x+r.width,(float)r.y);
srcTri[2]=Point2f((float)r.x,(float)r.y+(float)1.3*r.height);
warp=getAffineTransform(srcTri,dstTri);
return true;
}
bool FacemarkKazemiImpl::setMeanExtreme(){
if(meanshape.empty()){
String error_message = "Model not loaded properly.No mean shape found.Aborting...";
CV_Error(Error::StsBadArg, error_message);
}
for(size_t i=0;i<meanshape.size();i++){
if(meanshape[i].x>maxmeanx)
maxmeanx = meanshape[i].x;
if(meanshape[i].x<minmeanx)
minmeanx = meanshape[i].x;
if(meanshape[i].y>maxmeany)
maxmeany = meanshape[i].y;
if(meanshape[i].y<minmeany)
minmeany = meanshape[i].y;
}
return true;
}
bool FacemarkKazemiImpl::calcMeanShape (vector< vector<Point2f> >& trainlandmarks,vector<Mat>& trainimages,std::vector<Rect>& faces){
//clear the loaded meanshape
if(trainimages.empty()||trainlandmarks.size()!=trainimages.size()) {
// throw error if no data (or simply return -1?)
CV_Error(Error::StsBadArg, "Number of images is not equal to corresponding landmarks. Aborting...");
}
meanshape.clear();
vector<Mat> finalimages;
vector< vector<Point2f> > finallandmarks;
float xmean[200] = {0.0};
//array to store mean of y coordinates
float ymean[200] = {0.0};
size_t k=0;
//loop to calculate mean
Mat warp_mat,src,C,D;
vector<Rect> facesp;
Rect face;
for(size_t i = 0;i < trainimages.size();i++){
src = trainimages[i].clone();
//get bounding rectangle of image for reference
//function from facemark class
facesp.clear();
if(!getFaces(src,facesp)){
continue;
}
if(facesp.size()>1||facesp.empty())
continue;
face = facesp[0];
convertToUnit(face,warp_mat);
//loop to bring points to a common reference and adding
for(k=0;k<trainlandmarks[i].size();k++){
Point2f pt=trainlandmarks[i][k];
C = (Mat_<double>(3,1) << pt.x, pt.y, 1);
D = warp_mat*C;
pt.x = float(D.at<double>(0,0));
pt.y = float(D.at<double>(1,0));
trainlandmarks[i][k] = pt;
xmean[k] = xmean[k]+pt.x;
ymean[k] = ymean[k]+pt.y;
}
finalimages.push_back(trainimages[i]);
finallandmarks.push_back(trainlandmarks[i]);
faces.push_back(face);
}
//dividing by size to get mean and initialize meanshape
for(size_t i=0;i<k;i++){
xmean[i]=xmean[i]/finalimages.size();
ymean[i]=ymean[i]/finalimages.size();
if(xmean[i]>maxmeanx)
maxmeanx = xmean[i];
if(xmean[i]<minmeanx)
minmeanx = xmean[i];
if(ymean[i]>maxmeany)
maxmeany = ymean[i];
if(ymean[i]<minmeany)
minmeany = ymean[i];
meanshape.push_back(Point2f(xmean[i],ymean[i]));
}
trainimages.clear();
trainlandmarks.clear();
trainimages = finalimages;
trainlandmarks = finallandmarks;
finalimages.clear();
finallandmarks.clear();
return true;
}
bool FacemarkKazemiImpl::scaleData( vector< vector<Point2f> > & trainlandmarks,
vector<Mat> & trainimages ,Size s)
{
if(trainimages.empty()||trainimages.size()!=trainlandmarks.size()){
// throw error if no data (or simply return -1?)
CV_Error(Error::StsBadArg, "The data is not loaded properly by train function. Aborting...");
}
float scalex,scaley;
//scale all images and their landmarks according to input size
for(size_t i=0;i< trainimages.size();i++){
//calculating scale for x and y axis
scalex=float(s.width)/float(trainimages[i].cols);
scaley=float(s.height)/float(trainimages[i].rows);
resize(trainimages[i],trainimages[i],s,0,0,INTER_LINEAR_EXACT);
for (vector<Point2f>::iterator it = trainlandmarks[i].begin(); it != trainlandmarks[i].end(); it++) {
Point2f pt = (*it);
pt.x = pt.x*scalex;
pt.y = pt.y*scaley;
(*it) = pt;
}
}
return true;
}
Ptr<FacemarkKazemi> FacemarkKazemi::create(const FacemarkKazemi::Params &parameters){
return Ptr<FacemarkKazemiImpl>(new FacemarkKazemiImpl(parameters));
}
Ptr<Facemark> createFacemarkKazemi() {
FacemarkKazemi::Params parameters;
return Ptr<FacemarkKazemiImpl>(new FacemarkKazemiImpl(parameters));
}
}//cv
}//face
modules/face/src/face_alignmentimpl.hpp
+177
View File
@@ -0,0 +1,177 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_FACE_ALIGNMENTIMPL_HPP__
#define __OPENCV_FACE_ALIGNMENTIMPL_HPP__
#include "opencv2/face.hpp"
#include <string>
#include <sstream>
#include <vector>
#include <fstream>
#include <queue>
#include <algorithm>
#include <ctime>
using namespace std;
namespace cv{
namespace face{
/**@brief structure determining split in regression tree
*/
struct splitr{
//!index1 Index of the first coordinates among the test coordinates for deciding split.
uint64_t index1=0;
//! index2 index of the second coordinate among the test coordinates for deciding split.
uint64_t index2=0;
//! thresh threshold for deciding the split.
float thresh=0;
};
/** @brief represents a node of the regression tree*/
struct node_info{
//First pixel coordinate of split
long index1;
//Second pixel coordinate .split
long index2;
long depth;
long node_no;
};
/** @brief regression tree structure. Each leaf node is a vector storing residual shape.
* The tree is represented as vector of leaves.
*/
struct tree_node{
splitr split;
std::vector<Point2f> leaf;
};
struct regtree{
std::vector<tree_node> nodes;
};
/** @brief Represents a training sample
*It contains current shape, difference between actual shape
*and current shape. It also stores the image whose shape is being
*detected.
*/
struct training_sample{
//! shapeResiduals vector which stores the residual shape remaining to be corrected.
std::vector<Point2f> shapeResiduals;
//! current_shape vector containing current estimate of the shape
std::vector<Point2f> current_shape;
//! actual_shape vector containing the actual shape of the face or the ground truth.
std::vector<Point2f> actual_shape;
//! image A mat object which stores the image.
Mat image ;
//! pixel_intensities vector containing pixel intensities of the coordinates chosen for testing
std::vector<int> pixel_intensities;
//! pixel_coordinates vector containing pixel coordinates used for testing
std::vector<Point2f> pixel_coordinates;
//! bound Rectangle enclosing the face found in the image for training
Rect bound;
};
class FacemarkKazemiImpl : public FacemarkKazemi{
public:
FacemarkKazemiImpl(const FacemarkKazemi::Params& parameters);
void loadModel(String fs) CV_OVERRIDE;
bool setFaceDetector(FN_FaceDetector f, void* userdata) CV_OVERRIDE;
bool getFaces(InputArray image, OutputArray faces) CV_OVERRIDE;
bool fit(InputArray image, InputArray faces, OutputArrayOfArrays landmarks ) CV_OVERRIDE;
void training(String imageList, String groundTruth);
bool training(vector<Mat>& images, vector< vector<Point2f> >& landmarks,string filename,Size scale,string modelFilename) CV_OVERRIDE;
// Destructor for the class.
virtual ~FacemarkKazemiImpl() CV_OVERRIDE;
virtual void read( const FileNode& ) CV_OVERRIDE {}
virtual void write( FileStorage& ) const CV_OVERRIDE {}
protected:
FacemarkKazemi::Params params;
float minmeanx;
float maxmeanx;
float minmeany;
float maxmeany;
bool isModelLoaded;
/* meanshape This is a vector which stores the mean shape of all the images used in training*/
std::vector<Point2f> meanshape;
std::vector< std::vector<regtree> > loaded_forests;
std::vector< std::vector<Point2f> > loaded_pixel_coordinates;
FN_FaceDetector faceDetector;
void* faceDetectorData;
bool findNearestLandmarks(std::vector< std::vector<int> >& nearest);
/*Extract left node of the current node in the regression tree*/
unsigned long left(unsigned long index);
// Extract the right node of the current node in the regression tree
unsigned long right(unsigned long index);
// This function randomly generates test splits to get the best split.
splitr getTestSplits(std::vector<Point2f> pixel_coordinates,int seed);
// This function writes a split node to the XML file storing the trained model
void writeSplit(std::ofstream& os, const splitr& split);
// This function writes a leaf node to the binary file storing the trained model
void writeLeaf(std::ofstream& os, const std::vector<Point2f> &leaf);
// This function writes a tree to the binary file containing the model
void writeTree(std::ofstream &f,regtree tree);
// This function saves the pixel coordinates to a binary file
void writePixels(std::ofstream& f,int index);
// This function saves model to the binary file
bool saveModel(String filename);
// This funcrion reads pixel coordinates from the model file
void readPixels(std::ifstream& is,uint64_t index);
//This function reads the split node of the tree from binary file
void readSplit(std::ifstream& is, splitr &vec);
//This function reads a leaf node of the tree.
void readLeaf(std::ifstream& is, std::vector<Point2f> &leaf);
/* This function generates pixel intensities of the randomly generated test coordinates used to decide the split.
*/
bool getPixelIntensities(Mat img,std::vector<Point2f> pixel_coordinates_,std::vector<int>& pixel_intensities_,Rect face);
//This function initialises the training parameters.
bool setTrainingParameters(String filename);
//This function finds a warp matrix that warp the pixels from the normalised space to the actual space
bool convertToActual(Rect r,Mat &warp);
//This function finds a warp matrix that warps the pixels from the actual space to normaluised space
bool convertToUnit(Rect r,Mat &warp);
/** @brief This function calculates mean shape while training.
* This function is only called when new training data is supplied by the train function.
*@param trainlandmarks A vector of type cv::Point2f which stores the landmarks of corresponding images.
*@param trainimages A vector of type cv::Mat which stores the images which serve as training data.
*@param faces A vector of type cv::Rect which stores the bounding recatngle of each training image
*@returns A boolean value. It returns true if mean shape is found successfully else returns false.
*/
bool calcMeanShape(std::vector< std::vector<Point2f> > & trainlandmarks,std::vector<Mat>& trainimages,std::vector<Rect>& faces);
/** @brief This functions scales the annotations to a common size which is considered same for all images.
* @param trainlandmarks A vector of type cv::Point2f stores the landmarks of the corresponding training images.
* @param trainimages A vector of type cv::Mat which stores the images which are to be scaled.
* @param s A variable of type cv::Size stores the common size to which all the images are scaled.
* @returns A boolean value. It returns true when data is scaled properly else returns false.
*/
bool scaleData(std::vector< std::vector<Point2f> >& trainlandmarks,
std::vector<Mat>& trainimages , Size s=Size(460,460) );
// This function gets the landmarks in the meanshape nearest to the pixel coordinates.
unsigned long getNearestLandmark (Point2f pixels );
// This function gets the relative position of the test pixel coordinates relative to the current shape.
bool getRelativePixels(std::vector<Point2f> sample,std::vector<Point2f>& pixel_coordinates , std::vector<int> nearest_landmark = std::vector<int>());
// This function partitions samples according to the split
unsigned long divideSamples (splitr split,std::vector<training_sample>& samples,unsigned long start,unsigned long end);
// This function fits a regression tree according to the shape residuals calculated to give weak learners for GBT algorithm.
bool buildRegtree(regtree &tree,std::vector<training_sample>& samples,std::vector<Point2f> pixel_coordinates);
// This function greedily decides the best split among the test splits generated.
bool getBestSplit(std::vector<Point2f> pixel_coordinates, std::vector<training_sample>& samples,unsigned long start ,
unsigned long end,splitr& split,std::vector< std::vector<Point2f> >& sum,long node_no);
// This function randomly generates test coordinates for each level of cascade.
void getTestCoordinates ();
// This function implements gradient boosting by fitting regression trees
std::vector<regtree> gradientBoosting(std::vector<training_sample>& samples,std::vector<Point2f> pixel_coordinates);
// This function creates training sample by randomly assigning a current shape from set of shapes available.
void createLeafNode(regtree& tree,long node_no,std::vector<Point2f> assign);
// This function creates a split node in the regression tree.
void createSplitNode(regtree& tree, splitr split,long node_no);
// This function prepares the training samples
bool createTrainingSamples(std::vector<training_sample> &samples,std::vector<Mat> images,std::vector< std::vector<Point2f> > landmarks,
std::vector<Rect> rectangle);
//This function generates a split
bool generateSplit(std::queue<node_info>& curr,std::vector<Point2f> pixel_coordinates, std::vector<training_sample>& samples,
splitr &split , std::vector< std::vector<Point2f> >& sum);
bool setMeanExtreme();
//friend class getRelShape;
friend class getRelPixels;
};
}//face
}//cv
#endif
modules/face/src/face_basic.cpp
+100
View File
@@ -0,0 +1,100 @@
#include "opencv2/face.hpp"
#include "face_utils.hpp"
#include "precomp.hpp"
using namespace cv;
using namespace face;
int BasicFaceRecognizer::getNumComponents() const
{
return _num_components;
}
void BasicFaceRecognizer::setNumComponents(int val)
{
_num_components = val;
}
double BasicFaceRecognizer::getThreshold() const
{
return _threshold;
}
void BasicFaceRecognizer::setThreshold(double val)
{
_threshold = val;
}
std::vector<cv::Mat> BasicFaceRecognizer::getProjections() const
{
return _projections;
}
cv::Mat BasicFaceRecognizer::getLabels() const
{
return _labels;
}
cv::Mat BasicFaceRecognizer::getEigenValues() const
{
return _eigenvalues;
}
cv::Mat BasicFaceRecognizer::getEigenVectors() const
{
return _eigenvectors;
}
cv::Mat BasicFaceRecognizer::getMean() const
{
return _mean;
}
void BasicFaceRecognizer::read(const FileNode& fs)
{
//read matrices
double _t = 0;
fs["threshold"] >> _t; // older versions might not have "threshold"
if (_t !=0)
_threshold = _t; // be careful, not to overwrite DBL_MAX with 0 !
fs["num_components"] >> _num_components;
fs["mean"] >> _mean;
fs["eigenvalues"] >> _eigenvalues;
fs["eigenvectors"] >> _eigenvectors;
// read sequences
readFileNodeList(fs["projections"], _projections);
fs["labels"] >> _labels;
const FileNode& fn = fs["labelsInfo"];
if (fn.type() == FileNode::SEQ)
{
_labelsInfo.clear();
for (FileNodeIterator it = fn.begin(); it != fn.end();)
{
LabelInfo item;
it >> item;
_labelsInfo.insert(std::make_pair(item.label, item.value));
}
}
}
void BasicFaceRecognizer::write(FileStorage& fs) const
{
// write matrices
fs << "threshold" << _threshold;
fs << "num_components" << _num_components;
fs << "mean" << _mean;
fs << "eigenvalues" << _eigenvalues;
fs << "eigenvectors" << _eigenvectors;
// write sequences
writeFileNodeList(fs, "projections", _projections);
fs << "labels" << _labels;
fs << "labelsInfo" << "[";
for (std::map<int, String>::const_iterator it = _labelsInfo.begin(); it != _labelsInfo.end(); it++)
fs << LabelInfo(it->first, it->second);
fs << "]";
}
bool BasicFaceRecognizer::empty() const
{
return (_labels.empty());
}
modules/face/src/face_utils.hpp
+108
View File
@@ -0,0 +1,108 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_FACE_UTILS_HPP
#define __OPENCV_FACE_UTILS_HPP
#include "precomp.hpp"
using namespace cv;
inline Mat asRowMatrix(InputArrayOfArrays src, int rtype, double alpha=1, double beta=0) {
// make sure the input data is a vector of matrices or vector of vector
if(src.kind() != _InputArray::STD_VECTOR_MAT && src.kind() != _InputArray::STD_VECTOR_VECTOR) {
String error_message = "The data is expected as InputArray::STD_VECTOR_MAT (a std::vector<Mat>) or _InputArray::STD_VECTOR_VECTOR (a std::vector< std::vector<...> >).";
CV_Error(Error::StsBadArg, error_message);
}
// number of samples
size_t n = src.total();
// return empty matrix if no matrices given
if(n == 0)
return Mat();
// dimensionality of (reshaped) samples
size_t d = src.getMat(0).total();
// create data matrix
Mat data((int)n, (int)d, rtype);
// now copy data
for(unsigned int i = 0; i < n; i++) {
// make sure data can be reshaped, throw exception if not!
if(src.getMat(i).total() != d) {
String error_message = format("Wrong number of elements in matrix #%u! Expected %zu was %zu.", i, d, src.getMat(i).total());
CV_Error(Error::StsBadArg, error_message);
}
// get a hold of the current row
Mat xi = data.row(i);
// make reshape happy by cloning for non-continuous matrices
if(src.getMat(i).isContinuous()) {
src.getMat(i).reshape(1, 1).convertTo(xi, rtype, alpha, beta);
} else {
src.getMat(i).clone().reshape(1, 1).convertTo(xi, rtype, alpha, beta);
}
}
return data;
}
// Reads a sequence from a FileNode::SEQ with type _Tp into a result vector.
template<typename _Tp>
inline void readFileNodeList(const FileNode& fn, std::vector<_Tp>& result) {
if (fn.type() == FileNode::SEQ) {
for (FileNodeIterator it = fn.begin(); it != fn.end();) {
_Tp item;
it >> item;
result.push_back(item);
}
}
}
// Writes the a list of given items to a cv::FileStorage.
template<typename _Tp>
inline void writeFileNodeList(FileStorage& fs, const String& name,
const std::vector<_Tp>& items) {
// typedefs
typedef typename std::vector<_Tp>::const_iterator constVecIterator;
// write the elements in item to fs
fs << name << "[";
for (constVecIterator it = items.begin(); it != items.end(); ++it) {
fs << *it;
}
fs << "]";
}
// Utility structure to load/save face label info (a pair of int and string) via FileStorage
struct LabelInfo
{
LabelInfo():label(-1), value("") {}
LabelInfo(int _label, const String &_value): label(_label), value(_value) {}
int label;
String value;
void write(cv::FileStorage& fs) const
{
fs << "{" << "label" << label << "value" << value << "}";
}
void read(const cv::FileNode& node)
{
label = (int)node["label"];
value = (String)node["value"];
}
std::ostream& operator<<(std::ostream& out)
{
out << "{ label = " << label << ", " << "value = " << value.c_str() << "}";
return out;
}
};
inline void write(cv::FileStorage& fs, const String&, const LabelInfo& x)
{
x.write(fs);
}
inline void read(const cv::FileNode& node, LabelInfo& x, const LabelInfo& default_value = LabelInfo())
{
if(node.empty())
x = default_value;
else
x.read(node);
}
#endif
modules/face/src/facemark.cpp
+265
View File
@@ -0,0 +1,265 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
/*
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#include "precomp.hpp"
#include "opencv2/face/facemark_train.hpp"
/*dataset parser*/
#include <fstream>
#include <sstream>
#include <string>
#include <stdlib.h> /* atoi */
namespace cv {
namespace face {
using namespace std;
CParams::CParams(String s, double sf, int minN, Size minSz, Size maxSz){
cascade = s;
scaleFactor = sf;
minNeighbors = minN;
minSize = minSz;
maxSize = maxSz;
if (!face_cascade.load(cascade))
{
CV_Error_(Error::StsBadArg, ("Error loading face_cascade: %s", cascade.c_str()));
}
}
bool getFaces(InputArray image, OutputArray faces, CParams* params)
{
CV_Assert(params);
Mat gray;
std::vector<Rect> roi;
cvtColor(image.getMat(), gray, COLOR_BGR2GRAY);
equalizeHist(gray, gray);
params->face_cascade.detectMultiScale( gray, roi, params->scaleFactor, params->minNeighbors, CASCADE_SCALE_IMAGE, params->minSize, params->maxSize);
Mat(roi).copyTo(faces);
return true;
}
bool loadDatasetList(String imageList, String groundTruth, std::vector<String> & images, std::vector<String> & landmarks){
std::string line;
/*clear the output containers*/
images.clear();
landmarks.clear();
/*open the files*/
std::ifstream infile;
infile.open(imageList.c_str(), std::ios::in);
std::ifstream ss_gt;
ss_gt.open(groundTruth.c_str(), std::ios::in);
if ((!infile) || !(ss_gt)) {
printf("No valid input file was given, please check the given filename.\n");
return false;
}
/*load the images path*/
while (getline (infile, line)){
images.push_back(line);
}
/*load the points*/
while (getline (ss_gt, line)){
landmarks.push_back(line);
}
return true;
}
bool loadTrainingData(String filename, std::vector<String> & images, OutputArray _facePoints, char delim, float offset){
std::string line;
std::string item;
std::vector<Point2f> pts;
std::vector<float> raw;
// FIXIT
std::vector<std::vector<Point2f> > & facePoints =
*(std::vector<std::vector<Point2f> >*) _facePoints.getObj();
std::ifstream infile;
infile.open(filename.c_str(), std::ios::in);
if (!infile) {
CV_Error_(Error::StsBadArg, ("No valid input file was given, please check the given filename: %s", filename.c_str()));
}
/*clear the output containers*/
images.clear();
facePoints.clear();
/*the main loading process*/
while (getline (infile, line)){
std::istringstream ss(line); // string stream for the current line
/*pop the image path*/
getline (ss, item, delim);
images.push_back(item);
/*load all numbers*/
raw.clear();
while (getline (ss, item, delim)){
raw.push_back((float)atof(item.c_str()));
}
/*convert to opencv points*/
pts.clear();
for(unsigned i = 0;i< raw.size();i+=2){
pts.push_back(Point2f(raw[i]+offset,raw[i+1]+offset));
}
facePoints.push_back(pts);
} // main loading process
return true;
}
bool loadTrainingData(String imageList, String groundTruth, std::vector<String> & images, OutputArray _facePoints, float offset){
std::string line;
std::vector<Point2f> facePts;
// FIXIT
std::vector<std::vector<Point2f> > & facePoints =
*(std::vector<std::vector<Point2f> >*) _facePoints.getObj();
/*clear the output containers*/
images.clear();
facePoints.clear();
/*load the images path*/
std::ifstream infile;
infile.open(imageList.c_str(), std::ios::in);
if (!infile) {
CV_Error_(Error::StsBadArg, ("No valid input file was given, please check the given filename: %s", imageList.c_str()));
}
while (getline (infile, line)){
images.push_back(line);
}
/*load the points*/
std::ifstream ss_gt(groundTruth.c_str());
while (getline (ss_gt, line)){
facePts.clear();
loadFacePoints(line, facePts, offset);
facePoints.push_back(facePts);
}
return true;
}
bool loadFacePoints(String filename, OutputArray points, float offset){
vector<Point2f> pts;
std::string line, item;
std::ifstream infile(filename.c_str());
/*pop the version*/
std::getline(infile, line);
CV_Assert(line.compare(0,7,"version")==0);
/*pop the number of points*/
std::getline(infile, line);
CV_Assert(line.compare(0,8,"n_points")==0);
/*get the number of points*/
std::string item_npts;
int npts;
std::istringstream linestream(line);
linestream>>item_npts>>npts;
/*pop out '{' character*/
std::getline(infile, line);
/*main process*/
int cnt = 0;
std::string x, y;
pts.clear();
while (std::getline(infile, line) && cnt<npts )
{
cnt+=1;
std::istringstream ss(line);
ss>>x>>y;
pts.push_back(Point2f((float)atof(x.c_str())+offset,(float)atof(y.c_str())+offset));
}
Mat(pts).copyTo(points);
return true;
}
bool getFacesHAAR(InputArray image, OutputArray faces, const String& face_cascade_name)
{
Mat gray;
vector<Rect> roi;
CascadeClassifier face_cascade;
CV_Assert(face_cascade.load(face_cascade_name) && "Can't loading face_cascade");
cvtColor(image.getMat(), gray, COLOR_BGR2GRAY);
equalizeHist(gray, gray);
face_cascade.detectMultiScale(gray, roi, 1.1, 2, 0|CASCADE_SCALE_IMAGE, Size(30, 30));
Mat(roi).copyTo(faces);
return true;
}
bool loadTrainingData(vector<String> filename,vector< vector<Point2f> >
& trainlandmarks,vector<String> & trainimages)
{
string img;
vector<Point2f> temp;
string s;
string tok;
vector<string> coordinates;
ifstream f1;
for(unsigned long j=0;j<filename.size();j++){
f1.open(filename[j].c_str(),ios::in);
if(!f1.is_open()){
cout<<filename[j]<<endl;
CV_Error(Error::StsError, "File can't be opened for reading!");
}
//get the path of the image whose landmarks have to be detected
getline(f1,img);
//push the image paths in the vector
trainimages.push_back(img);
img.clear();
while(getline(f1,s)){
Point2f pt;
stringstream ss(s); // Turn the string into a stream.
while(getline(ss, tok,',')) {
coordinates.push_back(tok);
tok.clear();
}
pt.x = (float)atof(coordinates[0].c_str());
pt.y = (float)atof(coordinates[1].c_str());
coordinates.clear();
temp.push_back(pt);
}
trainlandmarks.push_back(temp);
temp.clear();
f1.close();
}
return true;
}
void drawFacemarks(InputOutputArray image, InputArray points, Scalar color){
Mat img = image.getMat();
vector<Point2f> pts = points.getMat();
for(size_t i=0;i<pts.size();i++){
circle(img, pts[i],3, color,-1);
}
}
} /* namespace face */
} /* namespace cv */
modules/face/src/facemarkAAM.cpp
+1265
View File
File diff suppressed because it is too large Load Diff
modules/face/src/facemarkLBF.cpp
+1458
View File
File diff suppressed because it is too large Load Diff
modules/face/src/facerec.cpp
+93
View File
@@ -0,0 +1,93 @@
/*
* Copyright (c) 2011,2012. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "precomp.hpp"
#include "opencv2/face.hpp"
namespace cv
{
namespace face
{
std::vector<int> FaceRecognizer::getLabelsByString(const String &str) const
{
std::vector<int> labels;
for (std::map<int, String>::const_iterator it = _labelsInfo.begin(); it != _labelsInfo.end(); it++)
{
size_t found = (it->second).find(str);
if (found != String::npos)
labels.push_back(it->first);
}
return labels;
}
String FaceRecognizer::getLabelInfo(int label) const
{
std::map<int, String>::const_iterator iter(_labelsInfo.find(label));
return iter != _labelsInfo.end() ? iter->second : "";
}
void FaceRecognizer::setLabelInfo(int label, const String &strInfo)
{
_labelsInfo[label] = strInfo;
}
void FaceRecognizer::update(InputArrayOfArrays src, InputArray labels)
{
CV_UNUSED(src);
CV_UNUSED(labels);
String error_msg = format("This FaceRecognizer does not support updating, you have to use FaceRecognizer::train to update it.");
CV_Error(Error::StsNotImplemented, error_msg);
}
void FaceRecognizer::read(const String &filename)
{
FileStorage fs(filename, FileStorage::READ);
if (!fs.isOpened())
CV_Error(Error::StsError, "File can't be opened for reading!");
this->read(fs.getFirstTopLevelNode());
fs.release();
}
void FaceRecognizer::write(const String &filename) const
{
FileStorage fs(filename, FileStorage::WRITE);
if (!fs.isOpened())
CV_Error(Error::StsError, "File can't be opened for writing!");
fs << getDefaultName() << "{";
this->write(fs);
fs << "}";
fs.release();
}
int FaceRecognizer::predict(InputArray src) const {
int _label;
double _dist;
predict(src, _label, _dist);
return _label;
}
void FaceRecognizer::predict(InputArray src, CV_OUT int &label, CV_OUT double &confidence) const {
Ptr<StandardCollector> collector = StandardCollector::create(getThreshold());
predict(src, collector);
label = collector->getMinLabel();
confidence = collector->getMinDist();
}
}
}
modules/face/src/fisher_faces.cpp
+158
View File
@@ -0,0 +1,158 @@
/*
* Copyright (c) 2011,2012. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "precomp.hpp"
#include <opencv2/face.hpp>
#include "face_utils.hpp"
namespace cv { namespace face {
// Belhumeur, P. N., Hespanha, J., and Kriegman, D. "Eigenfaces vs. Fisher-
// faces: Recognition using class specific linear projection.". IEEE
// Transactions on Pattern Analysis and Machine Intelligence 19, 7 (1997),
// 711–720.
class Fisherfaces: public FisherFaceRecognizer
{
public:
// Initializes an empty Fisherfaces model.
Fisherfaces(int num_components = 0, double threshold = DBL_MAX)
//: BasicFaceRecognizer(num_components, threshold)
{
_num_components = num_components;
_threshold = threshold;
}
// Computes a Fisherfaces model with images in src and corresponding labels
// in labels.
void train(InputArrayOfArrays src, InputArray labels) CV_OVERRIDE;
// Send all predict results to caller side for custom result handling
void predict(InputArray src, Ptr<PredictCollector> collector) const CV_OVERRIDE;
String getDefaultName() const CV_OVERRIDE
{
return "opencv_fisherfaces";
}
};
// Removes duplicate elements in a given vector.
template<typename _Tp>
inline std::vector<_Tp> remove_dups(const std::vector<_Tp>& src) {
typedef typename std::set<_Tp>::const_iterator constSetIterator;
typedef typename std::vector<_Tp>::const_iterator constVecIterator;
std::set<_Tp> set_elems;
for (constVecIterator it = src.begin(); it != src.end(); ++it)
set_elems.insert(*it);
std::vector<_Tp> elems;
for (constSetIterator it = set_elems.begin(); it != set_elems.end(); ++it)
elems.push_back(*it);
return elems;
}
//------------------------------------------------------------------------------
// Fisherfaces
//------------------------------------------------------------------------------
void Fisherfaces::train(InputArrayOfArrays src, InputArray _lbls) {
if(src.total() == 0) {
String error_message = format("Empty training data was given. You'll need more than one sample to learn a model.");
CV_Error(Error::StsBadArg, error_message);
} else if(_lbls.getMat().type() != CV_32SC1) {
String error_message = format("Labels must be given as integer (CV_32SC1). Expected %d, but was %d.", CV_32SC1, _lbls.type());
CV_Error(Error::StsBadArg, error_message);
}
// make sure data has correct size
if(src.total() > 1) {
for(int i = 1; i < static_cast<int>(src.total()); i++) {
if(src.getMat(i-1).total() != src.getMat(i).total()) {
String error_message = format("In the Fisherfaces method all input samples (training images) must be of equal size! Expected %zu pixels, but was %zu pixels.", src.getMat(i-1).total(), src.getMat(i).total());
CV_Error(Error::StsUnsupportedFormat, error_message);
}
}
}
// get data
Mat labels = _lbls.getMat();
Mat data = asRowMatrix(src, CV_64FC1);
// number of samples
int N = data.rows;
// make sure labels are passed in correct shape
if(labels.total() != (size_t) N) {
String error_message = format("The number of samples (src) must equal the number of labels (labels)! len(src)=%d, len(labels)=%zu.", N, labels.total());
CV_Error(Error::StsBadArg, error_message);
} else if(labels.rows != 1 && labels.cols != 1) {
String error_message = format("Expected the labels in a matrix with one row or column! Given dimensions are rows=%d, cols=%d.", labels.rows, labels.cols);
CV_Error(Error::StsBadArg, error_message);
}
// clear existing model data
_labels.release();
_projections.clear();
// safely copy from cv::Mat to std::vector
std::vector<int> ll;
for(unsigned int i = 0; i < labels.total(); i++) {
ll.push_back(labels.at<int>(i));
}
// get the number of unique classes
int C = (int) remove_dups(ll).size();
// clip number of components to be a valid number
if((_num_components <= 0) || (_num_components > (C-1)))
_num_components = (C-1);
// perform a PCA and keep (N-C) components
PCA pca(data, Mat(), PCA::DATA_AS_ROW, (N-C));
// project the data and perform a LDA on it
LDA lda(pca.project(data),labels, _num_components);
// store the total mean vector
_mean = pca.mean.reshape(1,1);
// store labels
_labels = labels.clone();
// store the eigenvalues of the discriminants
lda.eigenvalues().convertTo(_eigenvalues, CV_64FC1);
// Now calculate the projection matrix as pca.eigenvectors * lda.eigenvectors.
// Note: OpenCV stores the eigenvectors by row, so we need to transpose it!
gemm(pca.eigenvectors, lda.eigenvectors(), 1.0, Mat(), 0.0, _eigenvectors, GEMM_1_T);
// store the projections of the original data
for(int sampleIdx = 0; sampleIdx < data.rows; sampleIdx++) {
Mat p = LDA::subspaceProject(_eigenvectors, _mean, data.row(sampleIdx));
_projections.push_back(p);
}
}
void Fisherfaces::predict(InputArray _src, Ptr<PredictCollector> collector) const {
Mat src = _src.getMat();
// check data alignment just for clearer exception messages
if(_projections.empty()) {
// throw error if no data (or simply return -1?)
String error_message = "This Fisherfaces model is not computed yet. Did you call Fisherfaces::train?";
CV_Error(Error::StsBadArg, error_message);
} else if(src.total() != (size_t) _eigenvectors.rows) {
String error_message = format("Wrong input image size. Reason: Training and Test images must be of equal size! Expected an image with %d elements, but got %zu.", _eigenvectors.rows, src.total());
CV_Error(Error::StsBadArg, error_message);
}
// project into LDA subspace
Mat q = LDA::subspaceProject(_eigenvectors, _mean, src.reshape(1,1));
// find 1-nearest neighbor
collector->init((int)_projections.size());
for (size_t sampleIdx = 0; sampleIdx < _projections.size(); sampleIdx++) {
double dist = norm(_projections[sampleIdx], q, NORM_L2);
int label = _labels.at<int>((int)sampleIdx);
if (!collector->collect(label, dist))return;
}
}
Ptr<FisherFaceRecognizer> FisherFaceRecognizer::create(int num_components, double threshold)
{
return makePtr<Fisherfaces>(num_components, threshold);
}
} }
modules/face/src/getlandmarks.cpp
+288
View File
@@ -0,0 +1,288 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
#include <fstream>
#include <ctime>
using namespace std;
namespace cv{
namespace face{
bool FacemarkKazemiImpl :: findNearestLandmarks( vector< vector<int> >& nearest){
if(meanshape.empty()||loaded_pixel_coordinates.empty()){
String error_message = "Model not loaded properly.Aborting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
nearest.resize(loaded_pixel_coordinates.size());
for(unsigned long i=0 ; i< loaded_pixel_coordinates.size(); i++){
for(unsigned long j = 0;j<loaded_pixel_coordinates[i].size();j++){
nearest[i].push_back(getNearestLandmark(loaded_pixel_coordinates[i][j]));
}
}
return true;
}
void FacemarkKazemiImpl :: readSplit(ifstream& is, splitr &vec)
{
is.read((char*)&vec.index1, sizeof(vec.index1));
is.read((char*)&vec.index2, sizeof(vec.index2));
is.read((char*)&vec.thresh, sizeof(vec.thresh));
uint32_t dummy_ = 0;
is.read((char*)&dummy_, sizeof(dummy_)); // buggy writer structure alignment
CV_CheckEQ((int)(sizeof(vec.index1) + sizeof(vec.index2) + sizeof(vec.thresh) + sizeof(dummy_)), 24, "Invalid build configuration");
}
void FacemarkKazemiImpl :: readLeaf(ifstream& is, vector<Point2f> &leaf)
{
uint64_t size;
is.read((char*)&size, sizeof(size));
leaf.resize((size_t)size);
is.read((char*)&leaf[0], leaf.size() * sizeof(Point2f));
}
void FacemarkKazemiImpl :: readPixels(ifstream& is,uint64_t index)
{
is.read((char*)&loaded_pixel_coordinates[(unsigned long)index][0], loaded_pixel_coordinates[(unsigned long)index].size() * sizeof(Point2f));
}
void FacemarkKazemiImpl :: loadModel(String filename){
if(filename.empty()){
String error_message = "No filename found.Aborting....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
ifstream f(filename.c_str(),ios::binary);
if(!f.is_open()){
String error_message = "No file with given name found.Aborting....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t len;
f.read((char*)&len, sizeof(len));
char* temp = new char[(size_t)len+1];
f.read(temp, len);
temp[len] = '\0';
string s(temp);
delete [] temp;
if(s.compare("cascade_depth")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t cascade_size;
f.read((char*)&cascade_size,sizeof(cascade_size));
loaded_forests.resize((unsigned long)cascade_size);
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("pixel_coordinates")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
loaded_pixel_coordinates.resize((unsigned long)cascade_size);
uint64_t num_pixels;
f.read((char*)&num_pixels,sizeof(num_pixels));
for(unsigned long i=0 ; i < cascade_size ; i++){
loaded_pixel_coordinates[i].resize((unsigned long)num_pixels);
readPixels(f,i);
}
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("mean_shape")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t mean_shape_size;
f.read((char*)&mean_shape_size,sizeof(mean_shape_size));
meanshape.resize((unsigned long)mean_shape_size);
f.read((char*)&meanshape[0], meanshape.size() * sizeof(Point2f));
if(!setMeanExtreme())
exit(0);
f.read((char*)&len, sizeof(len));
temp = new char[(unsigned long)len+1];
f.read(temp, len);
temp[len] = '\0';
s = string(temp);
delete [] temp;
if(s.compare("num_trees")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t num_trees;
f.read((char*)&num_trees,sizeof(num_trees));
for(unsigned long i=0;i<cascade_size;i++){
for(unsigned long j=0;j<num_trees;j++){
regtree tree;
f.read((char*)&len, sizeof(len));
char* temp2 = new char[(unsigned long)len+1];
f.read(temp2, len);
temp2[len] = '\0';
s =string(temp2);
delete [] temp2;
if(s.compare("num_nodes")!=0){
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
uint64_t num_nodes;
f.read((char*)&num_nodes,sizeof(num_nodes));
tree.nodes.resize((unsigned long)num_nodes+1);
for(unsigned long k=0; k < num_nodes ; k++){
f.read((char*)&len, sizeof(len));
char* temp3 = new char[(unsigned long)len+1];
f.read(temp3, len);
temp3[len] = '\0';
s =string(temp3);
delete [] temp3;
tree_node node;
if(s.compare("split")==0){
splitr split;
readSplit(f,split);
node.split = split;
node.leaf.clear();
}
else if(s.compare("leaf")==0){
vector<Point2f> leaf;
readLeaf(f,leaf);
node.leaf = leaf;
}
else{
String error_message = "Data not saved properly.Aborting.....";
CV_Error(Error::StsBadArg, error_message);
return ;
}
tree.nodes[k]=node;
}
loaded_forests[i].push_back(tree);
}
}
f.close();
isModelLoaded = true;
}
/**
* @brief Copy the contents of a corners vector to an OutputArray, settings its size.
*/
static void _copyVector2Output(std::vector< std::vector< Point2f > > &vec, OutputArrayOfArrays out)
{
out.create((int)vec.size(), 1, CV_32FC2);
if (out.isMatVector()) {
for (unsigned int i = 0; i < vec.size(); i++) {
out.create(68, 1, CV_32FC2, i);
Mat &m = out.getMatRef(i);
Mat(Mat(vec[i]).t()).copyTo(m);
}
}
else if (out.isUMatVector()) {
for (unsigned int i = 0; i < vec.size(); i++) {
out.create(68, 1, CV_32FC2, i);
UMat &m = out.getUMatRef(i);
Mat(Mat(vec[i]).t()).copyTo(m);
}
}
else if (out.kind() == _OutputArray::STD_VECTOR_VECTOR) {
for (unsigned int i = 0; i < vec.size(); i++) {
out.create(68, 1, CV_32FC2, i);
Mat m = out.getMat(i);
Mat(Mat(vec[i]).t()).copyTo(m);
}
}
else {
CV_Error(cv::Error::StsNotImplemented,
"Only Mat vector, UMat vector, and vector<vector> OutputArrays are currently supported.");
}
}
bool FacemarkKazemiImpl::fit(InputArray img, InputArray roi, OutputArrayOfArrays _landmarks)
{
if(!isModelLoaded){
String error_message = "No model loaded. Aborting....";
CV_Error(Error::StsBadArg, error_message);
return false;
}
Mat image = img.getMat();
Mat roimat = roi.getMat();
std::vector<Rect> faces = roimat.reshape(4, roimat.rows);
std::vector<std::vector<Point2f> > shapes;
shapes.resize(faces.size());
if(image.empty()){
String error_message = "No image found.Aborting..";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(faces.empty()){
String error_message = "No faces found.Aborting..";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(meanshape.empty()||loaded_forests.empty()||loaded_pixel_coordinates.empty()){
String error_message = "Model not loaded properly.Aborting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(loaded_forests.size()==0){
String error_message = "Model not loaded properly.Aboerting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
if(loaded_pixel_coordinates.size()==0){
String error_message = "Model not loaded properly.Aboerting...";
CV_Error(Error::StsBadArg, error_message);
return false;
}
vector< vector<int> > nearest_landmarks;
findNearestLandmarks(nearest_landmarks);
tree_node curr_node;
vector<Point2f> pixel_relative;
vector<int> pixel_intensity;
Mat warp_mat;
for(size_t e=0;e<faces.size();e++){
shapes[e]=meanshape;
convertToActual(faces[e],warp_mat);
for(size_t i=0;i<loaded_forests.size();i++){
pixel_intensity.clear();
pixel_relative = loaded_pixel_coordinates[i];
getRelativePixels(shapes[e],pixel_relative,nearest_landmarks[i]);
getPixelIntensities(image,pixel_relative,pixel_intensity,faces[e]);
for(size_t j=0;j<loaded_forests[i].size();j++){
regtree tree = loaded_forests[i][j];
curr_node = tree.nodes[0];
unsigned long curr_node_index = 0;
while(curr_node.leaf.size()==0)
{
if ((float)pixel_intensity[(unsigned long)curr_node.split.index1] - (float)pixel_intensity[(unsigned long)curr_node.split.index2] > curr_node.split.thresh)
{
curr_node_index=left(curr_node_index);
} else
curr_node_index=right(curr_node_index);
curr_node = tree.nodes[curr_node_index];
}
for(size_t p=0;p<curr_node.leaf.size();p++){
shapes[e][p]=shapes[e][p] + curr_node.leaf[p];
}
}
}
for(unsigned long j=0;j<shapes[e].size();j++){
Mat C = (Mat_<double>(3,1) << shapes[e][j].x, shapes[e][j].y, 1);
Mat D = warp_mat*C;
shapes[e][j].x=float(D.at<double>(0,0));
shapes[e][j].y=float(D.at<double>(1,0));
}
}
_copyVector2Output(shapes, _landmarks);
return true;
}
}//cv
}//face
modules/face/src/lbph_faces.cpp
+432
View File
@@ -0,0 +1,432 @@
/*
* Copyright (c) 2011,2012. Philipp Wagner <bytefish[at]gmx[dot]de>.
* Released to public domain under terms of the BSD Simplified license.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the organization nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* See <http://www.opensource.org/licenses/bsd-license>
*/
#include "precomp.hpp"
#include "opencv2/face.hpp"
#include "face_utils.hpp"
namespace cv { namespace face {
// Face Recognition based on Local Binary Patterns.
//
// Ahonen T, Hadid A. and Pietikäinen M. "Face description with local binary
// patterns: Application to face recognition." IEEE Transactions on Pattern
// Analysis and Machine Intelligence, 28(12):2037-2041.
//
class LBPH : public LBPHFaceRecognizer
{
private:
int _grid_x;
int _grid_y;
int _radius;
int _neighbors;
double _threshold;
std::vector<Mat> _histograms;
Mat _labels;
// Computes a LBPH model with images in src and
// corresponding labels in labels, possibly preserving
// old model data.
void train(InputArrayOfArrays src, InputArray labels, bool preserveData);
public:
using FaceRecognizer::read;
using FaceRecognizer::write;
// Initializes this LBPH Model. The current implementation is rather fixed
// as it uses the Extended Local Binary Patterns per default.
//
// radius, neighbors are used in the local binary patterns creation.
// grid_x, grid_y control the grid size of the spatial histograms.
LBPH(int radius_=1, int neighbors_=8,
int gridx=8, int gridy=8,
double threshold = DBL_MAX) :
_grid_x(gridx),
_grid_y(gridy),
_radius(radius_),
_neighbors(neighbors_),
_threshold(threshold) {}
// Initializes and computes this LBPH Model. The current implementation is
// rather fixed as it uses the Extended Local Binary Patterns per default.
//
// (radius=1), (neighbors=8) are used in the local binary patterns creation.
// (grid_x=8), (grid_y=8) controls the grid size of the spatial histograms.
LBPH(InputArrayOfArrays src,
InputArray labels,
int radius_=1, int neighbors_=8,
int gridx=8, int gridy=8,
double threshold = DBL_MAX) :
_grid_x(gridx),
_grid_y(gridy),
_radius(radius_),
_neighbors(neighbors_),
_threshold(threshold) {
train(src, labels);
}
~LBPH() CV_OVERRIDE { }
// Computes a LBPH model with images in src and
// corresponding labels in labels.
void train(InputArrayOfArrays src, InputArray labels) CV_OVERRIDE;
// Updates this LBPH model with images in src and
// corresponding labels in labels.
void update(InputArrayOfArrays src, InputArray labels) CV_OVERRIDE;
// Send all predict results to caller side for custom result handling
void predict(InputArray src, Ptr<PredictCollector> collector) const CV_OVERRIDE;
// See FaceRecognizer::write.
void read(const FileNode& fn) CV_OVERRIDE;
// See FaceRecognizer::save.
void write(FileStorage& fs) const CV_OVERRIDE;
bool empty() const CV_OVERRIDE {
return (_labels.empty());
}
String getDefaultName() const CV_OVERRIDE
{
return "opencv_lbphfaces";
}
inline int getGridX() const CV_OVERRIDE { return _grid_x; }
inline void setGridX(int val) CV_OVERRIDE { _grid_x = val; }
inline int getGridY() const CV_OVERRIDE { return _grid_y; }
inline void setGridY(int val) CV_OVERRIDE { _grid_y = val; }
inline int getRadius() const CV_OVERRIDE { return _radius; }
inline void setRadius(int val) CV_OVERRIDE { _radius = val; }
inline int getNeighbors() const CV_OVERRIDE { return _neighbors; }
inline void setNeighbors(int val) CV_OVERRIDE { _neighbors = val; }
inline double getThreshold() const CV_OVERRIDE { return _threshold; }
inline void setThreshold(double val) CV_OVERRIDE { _threshold = val; }
inline std::vector<cv::Mat> getHistograms() const CV_OVERRIDE { return _histograms; }
inline cv::Mat getLabels() const CV_OVERRIDE { return _labels; }
};
void LBPH::read(const FileNode& fs) {
double _t = 0;
fs["threshold"] >> _t; // older versions might not have "threshold"
if (_t !=0)
_threshold = _t; // be careful, not to overwrite DBL_MAX with 0 !
fs["radius"] >> _radius;
fs["neighbors"] >> _neighbors;
fs["grid_x"] >> _grid_x;
fs["grid_y"] >> _grid_y;
//read matrices
readFileNodeList(fs["histograms"], _histograms);
fs["labels"] >> _labels;
const FileNode& fn = fs["labelsInfo"];
if (fn.type() == FileNode::SEQ)
{
_labelsInfo.clear();
for (FileNodeIterator it = fn.begin(); it != fn.end();)
{
LabelInfo item;
it >> item;
_labelsInfo.insert(std::make_pair(item.label, item.value));
}
}
}
// See FaceRecognizer::save.
void LBPH::write(FileStorage& fs) const {
fs << "threshold" << _threshold;
fs << "radius" << _radius;
fs << "neighbors" << _neighbors;
fs << "grid_x" << _grid_x;
fs << "grid_y" << _grid_y;
// write matrices
writeFileNodeList(fs, "histograms", _histograms);
fs << "labels" << _labels;
fs << "labelsInfo" << "[";
for (std::map<int, String>::const_iterator it = _labelsInfo.begin(); it != _labelsInfo.end(); it++)
fs << LabelInfo(it->first, it->second);
fs << "]";
}
void LBPH::train(InputArrayOfArrays _in_src, InputArray _in_labels) {
this->train(_in_src, _in_labels, false);
}
void LBPH::update(InputArrayOfArrays _in_src, InputArray _in_labels) {
// got no data, just return
if(_in_src.total() == 0)
return;
this->train(_in_src, _in_labels, true);
}
//------------------------------------------------------------------------------
// LBPH
//------------------------------------------------------------------------------
template <typename _Tp> static
void olbp_(InputArray _src, OutputArray _dst) {
// get matrices
Mat src = _src.getMat();
// allocate memory for result
_dst.create(src.rows-2, src.cols-2, CV_8UC1);
Mat dst = _dst.getMat();
// zero the result matrix
dst.setTo(0);
// calculate patterns
for(int i=1;i<src.rows-1;i++) {
for(int j=1;j<src.cols-1;j++) {
_Tp center = src.at<_Tp>(i,j);
unsigned char code = 0;
code |= (src.at<_Tp>(i-1,j-1) >= center) << 7;
code |= (src.at<_Tp>(i-1,j) >= center) << 6;
code |= (src.at<_Tp>(i-1,j+1) >= center) << 5;
code |= (src.at<_Tp>(i,j+1) >= center) << 4;
code |= (src.at<_Tp>(i+1,j+1) >= center) << 3;
code |= (src.at<_Tp>(i+1,j) >= center) << 2;
code |= (src.at<_Tp>(i+1,j-1) >= center) << 1;
code |= (src.at<_Tp>(i,j-1) >= center) << 0;
dst.at<unsigned char>(i-1,j-1) = code;
}
}
}
//------------------------------------------------------------------------------
// cv::elbp
//------------------------------------------------------------------------------
template <typename _Tp> static
inline void elbp_(InputArray _src, OutputArray _dst, int radius, int neighbors) {
//get matrices
Mat src = _src.getMat();
// allocate memory for result
_dst.create(src.rows-2*radius, src.cols-2*radius, CV_32SC1);
Mat dst = _dst.getMat();
// zero
dst.setTo(0);
for(int n=0; n<neighbors; n++) {
// sample points
float x = static_cast<float>(radius * cos(2.0*CV_PI*n/static_cast<float>(neighbors)));
float y = static_cast<float>(-radius * sin(2.0*CV_PI*n/static_cast<float>(neighbors)));
// relative indices
int fx = static_cast<int>(floor(x));
int fy = static_cast<int>(floor(y));
int cx = static_cast<int>(ceil(x));
int cy = static_cast<int>(ceil(y));
// fractional part
float ty = y - fy;
float tx = x - fx;
// set interpolation weights
float w1 = (1 - tx) * (1 - ty);
float w2 = tx * (1 - ty);
float w3 = (1 - tx) * ty;
float w4 = tx * ty;
// iterate through your data
for(int i=radius; i < src.rows-radius;i++) {
for(int j=radius;j < src.cols-radius;j++) {
// calculate interpolated value
float t = static_cast<float>(w1*src.at<_Tp>(i+fy,j+fx) + w2*src.at<_Tp>(i+fy,j+cx) + w3*src.at<_Tp>(i+cy,j+fx) + w4*src.at<_Tp>(i+cy,j+cx));
// floating point precision, so check some machine-dependent epsilon
dst.at<int>(i-radius,j-radius) += ((t > src.at<_Tp>(i,j)) || (std::abs(t-src.at<_Tp>(i,j)) < std::numeric_limits<float>::epsilon())) << n;
}
}
}
}
static void elbp(InputArray src, OutputArray dst, int radius, int neighbors)
{
int type = src.type();
switch (type) {
case CV_8SC1: elbp_<char>(src,dst, radius, neighbors); break;
case CV_8UC1: elbp_<unsigned char>(src, dst, radius, neighbors); break;
case CV_16SC1: elbp_<short>(src,dst, radius, neighbors); break;
case CV_16UC1: elbp_<unsigned short>(src,dst, radius, neighbors); break;
case CV_32SC1: elbp_<int>(src,dst, radius, neighbors); break;
case CV_32FC1: elbp_<float>(src,dst, radius, neighbors); break;
case CV_64FC1: elbp_<double>(src,dst, radius, neighbors); break;
default:
String error_msg = format("Using Original Local Binary Patterns for feature extraction only works on single-channel images (given %d). Please pass the image data as a grayscale image!", type);
CV_Error(Error::StsNotImplemented, error_msg);
break;
}
}
static Mat
histc_(const Mat& src, int minVal=0, int maxVal=255, bool normed=false)
{
Mat result;
// Establish the number of bins.
int histSize = maxVal-minVal+1;
// Set the ranges.
float range[] = { static_cast<float>(minVal), static_cast<float>(maxVal+1) };
const float* histRange = { range };
// calc histogram
calcHist(&src, 1, 0, Mat(), result, 1, &histSize, &histRange, true, false);
// normalize
if(normed) {
result /= (int)src.total();
}
return result.reshape(1,1);
}
static Mat histc(InputArray _src, int minVal, int maxVal, bool normed)
{
Mat src = _src.getMat();
switch (src.type()) {
case CV_8SC1:
return histc_(Mat_<float>(src), minVal, maxVal, normed);
break;
case CV_8UC1:
return histc_(src, minVal, maxVal, normed);
break;
case CV_16SC1:
return histc_(Mat_<float>(src), minVal, maxVal, normed);
break;
case CV_16UC1:
return histc_(src, minVal, maxVal, normed);
break;
case CV_32SC1:
return histc_(Mat_<float>(src), minVal, maxVal, normed);
break;
case CV_32FC1:
return histc_(src, minVal, maxVal, normed);
break;
}
CV_Error(Error::StsUnmatchedFormats, "This type is not implemented yet.");
}
static Mat spatial_histogram(InputArray _src, int numPatterns,
int grid_x, int grid_y, bool /*normed*/)
{
Mat src = _src.getMat();
// calculate LBP patch size
int width = src.cols/grid_x;
int height = src.rows/grid_y;
// allocate memory for the spatial histogram
Mat result = Mat::zeros(grid_x * grid_y, numPatterns, CV_32FC1);
// return matrix with zeros if no data was given
if(src.empty())
return result.reshape(1,1);
// initial result_row
int resultRowIdx = 0;
// iterate through grid
for(int i = 0; i < grid_y; i++) {
for(int j = 0; j < grid_x; j++) {
Mat src_cell = Mat(src, Range(i*height,(i+1)*height), Range(j*width,(j+1)*width));
Mat cell_hist = histc(src_cell, 0, (numPatterns-1), true);
// copy to the result matrix
Mat result_row = result.row(resultRowIdx);
cell_hist.reshape(1,1).convertTo(result_row, CV_32FC1);
// increase row count in result matrix
resultRowIdx++;
}
}
// return result as reshaped feature vector
return result.reshape(1,1);
}
//------------------------------------------------------------------------------
// wrapper to cv::elbp (extended local binary patterns)
//------------------------------------------------------------------------------
static Mat elbp(InputArray src, int radius, int neighbors) {
Mat dst;
elbp(src, dst, radius, neighbors);
return dst;
}
void LBPH::train(InputArrayOfArrays _in_src, InputArray _in_labels, bool preserveData) {
if(_in_src.kind() != _InputArray::STD_VECTOR_MAT && _in_src.kind() != _InputArray::STD_VECTOR_VECTOR) {
String error_message = "The images are expected as InputArray::STD_VECTOR_MAT (a std::vector<Mat>) or _InputArray::STD_VECTOR_VECTOR (a std::vector< std::vector<...> >).";
CV_Error(Error::StsBadArg, error_message);
}
if(_in_src.total() == 0) {
String error_message = format("Empty training data was given. You'll need more than one sample to learn a model.");
CV_Error(Error::StsUnsupportedFormat, error_message);
} else if(_in_labels.getMat().type() != CV_32SC1) {
String error_message = format("Labels must be given as integer (CV_32SC1). Expected %d, but was %d.", CV_32SC1, _in_labels.type());
CV_Error(Error::StsUnsupportedFormat, error_message);
}
// get the vector of matrices
std::vector<Mat> src;
_in_src.getMatVector(src);
// get the label matrix
Mat labels = _in_labels.getMat();
// check if data is well- aligned
if(labels.total() != src.size()) {
String error_message = format("The number of samples (src) must equal the number of labels (labels). Was len(samples)=%zu, len(labels)=%zu.", src.size(), _labels.total());
CV_Error(Error::StsBadArg, error_message);
}
// if this model should be trained without preserving old data, delete old model data
if(!preserveData) {
_labels.release();
_histograms.clear();
}
// append labels to _labels matrix
for(size_t labelIdx = 0; labelIdx < labels.total(); labelIdx++) {
_labels.push_back(labels.at<int>((int)labelIdx));
}
// store the spatial histograms of the original data
for(size_t sampleIdx = 0; sampleIdx < src.size(); sampleIdx++) {
// calculate lbp image
Mat lbp_image = elbp(src[sampleIdx], _radius, _neighbors);
// get spatial histogram from this lbp image
Mat p = spatial_histogram(
lbp_image, /* lbp_image */
static_cast<int>(std::pow(2.0, static_cast<double>(_neighbors))), /* number of possible patterns */
_grid_x, /* grid size x */
_grid_y, /* grid size y */
true);
// add to templates
_histograms.push_back(p);
}
}
void LBPH::predict(InputArray _src, Ptr<PredictCollector> collector) const {
if(_histograms.empty()) {
// throw error if no data (or simply return -1?)
String error_message = "This LBPH model is not computed yet. Did you call the train method?";
CV_Error(Error::StsBadArg, error_message);
}
Mat src = _src.getMat();
// get the spatial histogram from input image
Mat lbp_image = elbp(src, _radius, _neighbors);
Mat query = spatial_histogram(
lbp_image, /* lbp_image */
static_cast<int>(std::pow(2.0, static_cast<double>(_neighbors))), /* number of possible patterns */
_grid_x, /* grid size x */
_grid_y, /* grid size y */
true /* normed histograms */);
// find 1-nearest neighbor
collector->init((int)_histograms.size());
for (size_t sampleIdx = 0; sampleIdx < _histograms.size(); sampleIdx++) {
double dist = compareHist(_histograms[sampleIdx], query, HISTCMP_CHISQR_ALT);
int label = _labels.at<int>((int)sampleIdx);
if (!collector->collect(label, dist))return;
}
}
Ptr<LBPHFaceRecognizer> LBPHFaceRecognizer::create(int radius, int neighbors,
int grid_x, int grid_y, double threshold)
{
return makePtr<LBPH>(radius, neighbors, grid_x, grid_y, threshold);
}
}}
modules/face/src/mace.cpp
+270
View File
@@ -0,0 +1,270 @@
#include "precomp.hpp"
#include "opencv2/face/mace.hpp"
namespace cv {
namespace face {
//
//! Rearrange the quadrants of Fourier image
//! so that the origin is at the image center
//
static void shiftDFT(const Mat &src, Mat &dst)
{
Size size = src.size();
if (dst.empty() || (dst.size().width != size.width || dst.size().height != size.height))
{
dst.create(src.size(), src.type());
}
int cx = size.width/2;
int cy = size.height/2; // image center
Mat q1 = src(Rect(0, 0, cx,cy));
Mat q2 = src(Rect(cx,0, cx,cy));
Mat q3 = src(Rect(cx,cy,cx,cy));
Mat q4 = src(Rect(0, cy,cx,cy));
Mat d1 = dst(Rect(0, 0, cx,cy));
Mat d2 = dst(Rect(cx,0, cx,cy));
Mat d3 = dst(Rect(cx,cy,cx,cy));
Mat d4 = dst(Rect(0, cy,cx,cy));
if (src.data != dst.data){
q3.copyTo(d1);
q4.copyTo(d2);
q1.copyTo(d3);
q2.copyTo(d4);
} else {
Mat tmp;
q3.copyTo(tmp);
q1.copyTo(d3);
tmp.copyTo(d1);
q4.copyTo(tmp);
q2.copyTo(d4);
tmp.copyTo(d2);
}
}
// Computes 64-bit "cyclic redundancy check" sum, as specified in ECMA-182
static uint64 crc64( const uchar* data, size_t size, uint64 crc0=0 )
{
static uint64 table[256];
static bool initialized = false;
if( !initialized )
{
for( int i = 0; i < 256; i++ )
{
uint64 c = i;
for( int j = 0; j < 8; j++ )
c = ((c & 1) ? CV_BIG_UINT(0xc96c5795d7870f42) : 0) ^ (c >> 1);
table[i] = c;
}
initialized = true;
}
uint64 crc = ~crc0;
for( size_t idx = 0; idx < size; idx++ )
crc = table[(uchar)crc ^ data[idx]] ^ (crc >> 8);
return ~crc;
}
struct MACEImpl CV_FINAL : MACE {
Mat_<Vec2d> maceFilter; // filled from compute()
Mat convFilter; // optional random convolution (cancellable)
int IMGSIZE; // images will get resized to this
double threshold; // minimal "sameness" threshold from the train images
MACEImpl(int siz) : IMGSIZE(siz), threshold(DBL_MAX) {}
void salt(const String &passphrase) CV_OVERRIDE {
theRNG().state = ((int64)crc64((uchar*)passphrase.c_str(), passphrase.size()));
convFilter.create(IMGSIZE, IMGSIZE, CV_64F);
randn(convFilter, 0, 1.0/(IMGSIZE*IMGSIZE));
}
Mat dftImage(Mat img) const {
Mat gray;
resize(img, gray, Size(IMGSIZE,IMGSIZE)) ;
if (gray.channels() > 1)
cvtColor(gray, gray, COLOR_BGR2GRAY);
equalizeHist(gray, gray);
gray.convertTo(gray, CV_64F);
if (! convFilter.empty()) { // optional, but unfortunately, it has to happen after resize/equalize ops.
filter2D(gray, gray, CV_64F, convFilter);
}
Mat input[2] = {gray, Mat(gray.size(), gray.type(), 0.0)};
Mat complexInput;
merge(input, 2, complexInput);
Mat dftImg(IMGSIZE*2, IMGSIZE*2, CV_64FC2, 0.0);
complexInput.copyTo(dftImg(Rect(0,0,IMGSIZE,IMGSIZE)));
dft(dftImg, dftImg);
return dftImg;
}
// compute the mace filter: `h = D(-1) * X * (X(+) * D(-1) * X)(-1) * C`
void compute(std::vector<Mat> images) {
return compute(images, false);
}
void compute(std::vector<Mat> images, bool isdft) {
int size = (int)images.size();
int IMGSIZE_2X = IMGSIZE * 2;
int TOTALPIXEL = IMGSIZE_2X * IMGSIZE_2X;
Mat_<double> D(TOTALPIXEL, 1, 0.0);
Mat_<Vec2d> S(TOTALPIXEL, size, Vec2d(0,0));
Mat_<Vec2d> SPLUS(size, TOTALPIXEL, Vec2d(0,0));
for (int i=0; i<size; i++) {
Mat_<Vec2d> dftImg = isdft ? images[i] : dftImage(images[i]);
for (int l=0; l<IMGSIZE_2X; l++) {
for (int m=0; m<IMGSIZE_2X; m++) {
int j = l * IMGSIZE_2X + m;
Vec2d s = dftImg(l, m);
S(j, i) = s;
SPLUS(i, j) = Vec2d(s[0], -s[1]);
D(j, 0) += (s[0]*s[0]) + (s[1]*s[1]);
}
}
}
Mat_<double> DSQ; cv::sqrt(D, DSQ);
Mat_<double> DINV = TOTALPIXEL * size / DSQ;
Mat_<Vec2d> DINV_S(TOTALPIXEL, size);
Mat_<Vec2d> SPLUS_DINV(size, TOTALPIXEL);
for (int l=0; l<size; l++) {
for (int m=0; m<TOTALPIXEL; m++) {
DINV_S(m, l) = S(m,l) * DINV(m,0);
SPLUS_DINV(l, m) = SPLUS(l,m) * DINV(m,0);
}
}
Mat_<Vec2d> SPLUS_DINV_S = SPLUS_DINV * S;
Mat_<double> SPLUS_DINV_S_INV_1(2*size, 2*size, 0.0);
for (int l=0; l<size; l++) {
for (int m=0; m<size; m++) {
Vec2d s = SPLUS_DINV_S(l, m);
SPLUS_DINV_S_INV_1(l, m) = s[0];
SPLUS_DINV_S_INV_1(l+size, m+size) = s[0];
SPLUS_DINV_S_INV_1(l, m+size) = s[1];
SPLUS_DINV_S_INV_1(l+size, m) = -s[1];
}
}
invert(SPLUS_DINV_S_INV_1, SPLUS_DINV_S_INV_1);
Mat_<Vec2d> SPLUS_DINV_S_INV(size, size);
for (int l=0; l<size; l++) {
for (int m=0; m<size; m++) {
SPLUS_DINV_S_INV(l, m) = Vec2d(SPLUS_DINV_S_INV_1(l,m), SPLUS_DINV_S_INV_1(l,m+size));
}
}
Mat_<Vec2d> Hmace = DINV_S * SPLUS_DINV_S_INV;
Mat_<Vec2d> C(size, 1, Vec2d(1,0));
maceFilter = Mat(Hmace * C).reshape(2,IMGSIZE_2X);
}
// get the lowest (worst) positive train correlation,
// our lower bound threshold for the "same()" test later
double computeThreshold(const std::vector<Mat> &images, bool isdft) const {
double best=DBL_MAX;
for (size_t i=0; i<images.size(); i++) {
double d = correlate(images[i], isdft);
if (d < best) {
best = d;
}
}
return best;
}
// convolute macefilter and dft image,
// calculate the peak to sidelobe ratio
// on the real part of the inverse dft
double correlate(const Mat &img) const {
return correlate(img, false);
}
double correlate(const Mat &img, bool isdft) const {
if (maceFilter.empty()) return -1; // not trained.
int IMGSIZE_2X = IMGSIZE * 2;
Mat dftImg = isdft ? img : dftImage(img);
mulSpectrums(dftImg, maceFilter, dftImg, DFT_ROWS, true);
dft(dftImg, dftImg, DFT_INVERSE|DFT_SCALE, 0);
Mat chn[2];
split(dftImg, chn);
Mat_<double> re;
shiftDFT(chn[0], re);
double m1,M1;
minMaxLoc(re, &m1, &M1, 0, 0);
double peakCorrPlaneEnergy = M1 / sqrt(sum(re)[0]);
re -= m1;
// circle mask for the sidelobe area
Mat mask(IMGSIZE_2X, IMGSIZE_2X, CV_8U, Scalar(0));
int rad_1 = int(floor((double)(45.0/64.0)*(double)IMGSIZE));
int rad_2 = int(floor((double)(27.0/64.0)*(double)IMGSIZE));
circle(mask, Point(IMGSIZE,IMGSIZE), rad_1, Scalar(255), -1);
circle(mask, Point(IMGSIZE,IMGSIZE), rad_2, Scalar(0), -1);
Scalar mean, dev;
meanStdDev(re, mean, dev, mask);
double peak = re(IMGSIZE, IMGSIZE);
double peakToSideLobeRatio = (peak - mean[0]) / dev[0];
return 100.0 * peakToSideLobeRatio * peakCorrPlaneEnergy;
}
// MACE interface
void train(InputArrayOfArrays input) CV_OVERRIDE {
std::vector<Mat> images, dftImg;
input.getMatVector(images);
for (size_t i=0; i<images.size(); i++) { // cache dft images
dftImg.push_back(dftImage(images[i]));
}
compute(dftImg, true);
threshold = computeThreshold(dftImg, true);
}
bool same(InputArray img) const CV_OVERRIDE {
return correlate(img.getMat()) >= threshold;
}
// cv::Algorithm:
bool empty() const CV_OVERRIDE {
return maceFilter.empty() || IMGSIZE == 0;
}
String getDefaultName () const CV_OVERRIDE {
return String("MACE");
}
void clear() CV_OVERRIDE {
maceFilter.release();
convFilter.release();
}
void write(cv::FileStorage &fs) const CV_OVERRIDE {
fs << "mace" << maceFilter;
fs << "conv" << convFilter;
fs << "threshold" << threshold;
}
void read(const cv::FileNode &fn) CV_OVERRIDE {
fn["mace"] >> maceFilter;
fn["conv"] >> convFilter;
fn["threshold"] >> threshold;
IMGSIZE = maceFilter.cols/2;
}
};
cv::Ptr<MACE> MACE::create(int siz) {
return makePtr<MACEImpl>(siz);
}
cv::Ptr<MACE> MACE::load(const String &filename, const String &objname) {
return Algorithm::load<MACE>(filename, objname);
}
} /* namespace face */
} /* namespace cv */
modules/face/src/precomp.hpp
+59
View File
@@ -0,0 +1,59 @@
/*M///////////////////////////////////////////////////////////////////////////////////////
//
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
//
// By downloading, copying, installing or using the software you agree to this license.
// If you do not agree to this license, do not download, install,
// copy or use the software.
//
//
// License Agreement
// For Open Source Computer Vision Library
//
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Third party copyrights are property of their respective owners.
//
// Redistribution and use in source and binary forms, with or without modification,
// are permitted provided that the following conditions are met:
//
// * Redistribution's of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// * Redistribution's in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// * The name of the copyright holders may not be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// This software is provided by the copyright holders and contributors "as is" and
// any express or implied warranties, including, but not limited to, the implied
// warranties of merchantability and fitness for a particular purpose are disclaimed.
// In no event shall the Intel Corporation or contributors be liable for any direct,
// indirect, incidental, special, exemplary, or consequential damages
// (including, but not limited to, procurement of substitute goods or services;
// loss of use, data, or profits; or business interruption) however caused
// and on any theory of liability, whether in contract, strict liability,
// or tort (including negligence or otherwise) arising in any way out of
// the use of this software, even if advised of the possibility of such damage.
//
//M*/
#ifndef __OPENCV_PRECOMP_H__
#define __OPENCV_PRECOMP_H__
#include "opencv2/core.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/geometry.hpp"
#include "opencv2/core/utility.hpp"
#include "opencv2/core/private.hpp"
#include "opencv2/core/persistence.hpp"
#include <map>
#include <set>
#include <limits>
#include <iostream>
#endif
modules/face/src/predict_collector.cpp
+114
View File
@@ -0,0 +1,114 @@
/*
By downloading, copying, installing or using the software you agree to this license.
If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2000-2015, Intel Corporation, all rights reserved.
Copyright (C) 2009-2011, Willow Garage Inc., all rights reserved.
Copyright (C) 2009-2015, NVIDIA Corporation, all rights reserved.
Copyright (C) 2010-2013, Advanced Micro Devices, Inc., all rights reserved.
Copyright (C) 2015, OpenCV Foundation, all rights reserved.
Copyright (C) 2015, Itseez Inc., all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are disclaimed.
In no event shall copyright holders or contributors be liable for any direct,
indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#include "opencv2/face/predict_collector.hpp"
namespace cv {namespace face {
static std::pair<int, double> toPair(const StandardCollector::PredictResult & val) {
return std::make_pair(val.label, val.distance);
}
static bool pairLess(const std::pair<int, double> & lhs, const std::pair<int, double> & rhs) {
return lhs.second < rhs.second;
}
//===================================
StandardCollector::StandardCollector(double threshold_) : threshold(threshold_) {
init(0);
}
void StandardCollector::init(size_t size) {
minRes = PredictResult();
data.clear();
data.reserve(size);
}
bool StandardCollector::collect(int label, double dist) {
if (dist < threshold)
{
PredictResult res(label, dist);
if (res.distance < minRes.distance)
minRes = res;
data.push_back(res);
}
return true;
}
int StandardCollector::getMinLabel() const {
return minRes.label;
}
double StandardCollector::getMinDist() const {
return minRes.distance;
}
std::vector< std::pair<int, double> > StandardCollector::getResults(bool sorted) const {
std::vector< std::pair<int, double> > res(data.size());
std::transform(data.begin(), data.end(), res.begin(), &toPair);
if (sorted)
{
std::sort(res.begin(), res.end(), &pairLess);
}
return res;
}
std::map<int, double> StandardCollector::getResultsMap() const {
std::map<int, double> res;
for (std::vector<PredictResult>::const_iterator i = data.begin(); i != data.end(); ++i) {
std::map<int, double>::iterator j = res.find(i->label);
if (j == res.end()) {
res.insert(toPair(*i));
} else if (i->distance < j->second) {
j->second = i->distance;
}
}
return res;
}
Ptr<StandardCollector> StandardCollector::create(double threshold) {
return makePtr<StandardCollector>(threshold);
}
}} // cv::face::
modules/face/src/regtree.cpp
+309
View File
@@ -0,0 +1,309 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
using namespace std;
namespace cv{
namespace face{
//Threading helper classes
class doSum : public ParallelLoopBody
{
public:
doSum(vector<training_sample>* samples_,vector<Point2f>* sum_) :
samples(samples_),
sum(sum_)
{
}
virtual void operator()( const Range& range) const CV_OVERRIDE
{
for (int j = range.start; j < range.end; ++j){
for(unsigned long k=0;k<(*samples)[j].shapeResiduals.size();k++){
(*sum)[k]=(*sum)[k]+(*samples)[j].shapeResiduals[k];
}
}
}
private:
vector<training_sample>* samples;
vector<Point2f>* sum;
};
class modifySamples : public ParallelLoopBody
{
public:
modifySamples(vector<training_sample>* samples_,vector<Point2f>* temp_) :
samples(samples_),
temp(temp_)
{
}
virtual void operator()( const Range& range) const CV_OVERRIDE
{
for (int j = range.start; j < range.end; ++j){
for(unsigned long k=0;k<(*samples)[j].shapeResiduals.size();k++){
(*samples)[j].shapeResiduals[k]=(*samples)[j].shapeResiduals[k]-(*temp)[k];
(*samples)[j].current_shape[k]=(*samples)[j].actual_shape[k]-(*samples)[j].shapeResiduals[k];
}
}
}
private:
vector<training_sample>* samples;
vector<Point2f>* temp;
};
class splitSamples : public ParallelLoopBody
{
public:
splitSamples(vector<training_sample>* samples_,vector< vector<Point2f> >* leftsumresiduals_,vector<unsigned long>* left_count_,unsigned long* num_test_splits_,vector<splitr>* feats_) :
samples(samples_),
leftsumresiduals(leftsumresiduals_),
left_count(left_count_),
num_test_splits(num_test_splits_),
feats(feats_)
{
}
virtual void operator()( const Range& range) const CV_OVERRIDE
{
for (int i = range.start; i < range.end; ++i){
for(unsigned long j=0;j<*(num_test_splits);j++){
(*left_count)[j]++;
if ((float)(*samples)[i].pixel_intensities[(unsigned long)(*feats)[j].index1] - (float)(*samples)[i].pixel_intensities[(unsigned long)(*feats)[j].index2] > (*feats)[j].thresh){
for(unsigned long k=0;k<(*samples)[i].shapeResiduals.size();k++){
(*leftsumresiduals)[j][k]=(*leftsumresiduals)[j][k]+(*samples)[i].shapeResiduals[k];
}
}
}
}
}
private:
vector<training_sample>* samples;
vector< vector<Point2f> >* leftsumresiduals;
vector<unsigned long>* left_count;
unsigned long* num_test_splits;
vector<splitr>* feats;
};
splitr FacemarkKazemiImpl::getTestSplits(vector<Point2f> pixel_coordinates,int seed)
{
splitr feat;
//generates splits whose probability is above a particular threshold.
//P(u,v)=e^(-distance/lambda) as described in the research paper
//cited above. This helps to select closer pixels hence make efficient
//splits.
double probability;
double check;
RNG rng(seed);
do
{
//select random pixel coordinate
feat.index1 = rng.uniform(0,params.num_test_coordinates);
//select another random coordinate
feat.index2 = rng.uniform(0,params.num_test_coordinates);
Point2f pt = pixel_coordinates[(unsigned long)feat.index1]-pixel_coordinates[(unsigned long)feat.index2];
double distance = sqrt((pt.x*pt.x)+(pt.y*pt.y));
//calculate the probability
probability = exp(-distance/params.lambda);
check = rng.uniform(double(0),double(1));
}
while(check>probability||feat.index1==feat.index2);
feat.thresh =(float)(((rng.uniform(double(0),double(1)))*256 - 128)/2.0);
return feat;
}
bool FacemarkKazemiImpl:: getBestSplit(vector<Point2f> pixel_coordinates, vector<training_sample>& samples,unsigned long start ,
unsigned long end,splitr& split,vector< vector<Point2f> >& sum,long node_no)
{
if(samples[0].shapeResiduals.size()!=samples[0].current_shape.size()){
String error_message = "Error while generating split.Residuals are not complete.Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
//This vector stores the matrices where each matrix represents
//sum of the residuals of shapes of samples which go to the left
//child after split
vector< vector<Point2f> > leftsumresiduals;
leftsumresiduals.resize(params.num_test_splits);
vector<splitr> feats;
//generate random splits and selects the best split amongst them.
for (unsigned long i = 0; i < params.num_test_splits; ++i){
feats.push_back(getTestSplits(pixel_coordinates,i+(int)time(0)));
leftsumresiduals[i].resize(samples[0].shapeResiduals.size());
}
vector<unsigned long> left_count;
left_count.resize(params.num_test_splits);
parallel_for_(Range(start,end),splitSamples(&samples,&leftsumresiduals,&left_count,&params.num_test_splits,&feats));
//Selecting the best split
double best_score =-1;
unsigned long best_feat = 0;
double score = -1;
vector<Point2f> right_sum;
right_sum.resize(sum[node_no].size());
vector<Point2f> left_sum;
left_sum.resize(sum[node_no].size());
unsigned long right_cnt;
for(unsigned long i=0;i<leftsumresiduals.size();i++){
right_cnt = (end-start+1)-left_count[i];
for(unsigned long k=0;k<leftsumresiduals[i].size();k++){
if (right_cnt!=0){
right_sum[k].x=(sum[node_no][k].x-leftsumresiduals[i][k].x)/right_cnt;
right_sum[k].y=(sum[node_no][k].y-leftsumresiduals[i][k].y)/right_cnt;
}
else
right_sum[k]=Point2f(0,0);
if(left_count[i]!=0){
left_sum[k].x=leftsumresiduals[i][k].x/left_count[i];
left_sum[k].y=leftsumresiduals[i][k].y/left_count[i];
}
else
left_sum[k]=Point2f(0,0);
}
Point2f pt1(0,0);
Point2f pt2(0,0);
for(unsigned long k=0;k<left_sum.size();k++){
pt1.x = pt1.x + (float)(left_sum[k].x*left_sum[k].x);
pt2.x = pt2.x + (float)(right_sum[k].x*right_sum[k].x);
pt1.y = pt1.y + (float)(left_sum[k].y*left_sum[k].y);
pt2.y = pt2.y + (float)(right_sum[k].y*right_sum[k].y);
}
score = (double)sqrt(pt1.x+pt1.y)*(double)left_count[i] + (double)sqrt(pt2.x+pt2.y)*(double)right_cnt;
if(score > best_score){
best_score = score;
best_feat = i;
}
}
sum[2*node_no+1] = leftsumresiduals[best_feat];
sum[2*node_no+2].resize(sum[node_no].size());
for(unsigned long k=0;k<sum[node_no].size();k++){
sum[2*node_no+2][k].x = sum[node_no][k].x-sum[2*node_no+1][k].x;
sum[2*node_no+2][k].y = sum[node_no][k].y-sum[2*node_no+1][k].y;
}
split = feats[best_feat];
return true;
}
void FacemarkKazemiImpl::createSplitNode(regtree& tree, splitr split,long node_no){
tree_node node;
node.split = split;
node.leaf.clear();
tree.nodes[node_no]=node;
}
void FacemarkKazemiImpl::createLeafNode(regtree& tree,long node_no,vector<Point2f> assign){
tree_node node;
node.split.index1 = (uint64_t)(-1);
node.split.index2 = (uint64_t)(-1);
node.leaf = assign;
tree.nodes[node_no] = node;
}
bool FacemarkKazemiImpl :: generateSplit(queue<node_info>& curr,vector<Point2f> pixel_coordinates, vector<training_sample>& samples,
splitr &split , vector< vector<Point2f> >& sum){
long start = curr.front().index1;
long end = curr.front().index2;
long _depth = curr.front().depth;
long node_no =curr.front().node_no;
curr.pop();
if(start == end)
return false;
getBestSplit(pixel_coordinates,samples,start,end,split,sum,node_no);
long mid = divideSamples(split, samples, start, end);
//cout<<mid<<endl;
if(mid==start||mid==end+1)
return false;
node_info _left,_right;
_left.index1 = start;
_left.index2 = mid-1;
_left.depth = _depth +1;
_left.node_no = 2*node_no+1;
_right.index1 = mid;
_right.index2 = end;
_right.depth = _depth +1;
_right.node_no = 2*node_no+2;
curr.push(_left);
curr.push(_right);
return true;
}
bool FacemarkKazemiImpl :: buildRegtree(regtree& tree,vector<training_sample>& samples,vector<Point2f> pixel_coordinates){
if(samples.size()==0){
String error_message = "Error while building regression tree.Empty samples. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
if(pixel_coordinates.size()==0){
String error_message = "Error while building regression tree.No pixel coordinates. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
queue<node_info> curr;
node_info parent;
vector< vector<Point2f> > sum;
const long numNodes =(long)pow(2,params.tree_depth);
const long numSplitNodes = numNodes/2 - 1;
sum.resize(numNodes+1);
sum[0].resize(samples[0].shapeResiduals.size());
parallel_for_(cv::Range(0,(int)samples.size()), doSum(&(samples),&(sum[0])));
parent.index1=0;
parent.index2=(long)samples.size()-1;
parent.node_no=0;
parent.depth=0;
curr.push(parent);
tree.nodes.resize(numNodes+1);
//Total number of split nodes
while(!curr.empty()){
pair<long,long> range= make_pair(curr.front().index1,curr.front().index2);
long node_no = curr.front().node_no;
splitr split = {0, 0, 0};
//generate a split
if(node_no<=numSplitNodes){
if(generateSplit(curr,pixel_coordinates,samples,split,sum)){
createSplitNode(tree,split,node_no);
}
//create leaf
else{
long count = range.second-range.first +1;
vector<Point2f> temp;
temp.resize(samples[range.first].shapeResiduals.size());
parallel_for_(Range(range.first, range.second), doSum(&(samples),&(temp)));
for(unsigned long k=0;k<temp.size();k++){
temp[k].x=(temp[k].x/count)*params.learning_rate;
temp[k].y=(temp[k].y/count)*params.learning_rate;
}
// Modify current shape according to the weak learners.
parallel_for_(Range(range.first,range.second), modifySamples(&(samples),&(temp)));
createLeafNode(tree,node_no,temp);
}
}
else
{
unsigned long count = range.second-range.first +1;
vector<Point2f> temp;
temp.resize(samples[range.first].shapeResiduals.size());
parallel_for_(Range(range.first, range.second), doSum(&(samples),&(temp)));
for(unsigned long k=0;k<temp.size();k++){
temp[k].x=(temp[k].x/count)*params.learning_rate;
temp[k].y=(temp[k].y/count)*params.learning_rate;
}
// Modify current shape according to the weak learners.
parallel_for_(Range(range.first,range.second), modifySamples(&(samples),&(temp)));
createLeafNode(tree,node_no,temp);
curr.pop();
}
}
return true;
}
unsigned long FacemarkKazemiImpl::divideSamples (splitr split,vector<training_sample>& samples,unsigned long start,unsigned long end)
{
if(samples.size()==0){
String error_message = "Error while dividing samples. Sample array empty. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
unsigned long i = start;
training_sample temp;
//partition samples according to the split
for (unsigned long j = start; j < end; ++j)
{
if ((float)samples[j].pixel_intensities[(unsigned long)split.index1] - (float)samples[j].pixel_intensities[(unsigned long)split.index2] > split.thresh)
{
temp=samples[i];
samples[i]=samples[j];
samples[j]=temp;
++i;
}
}
return i;
}
}//cv
}//face
modules/face/src/trainFacemark.cpp
+345
View File
@@ -0,0 +1,345 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "precomp.hpp"
#include "face_alignmentimpl.hpp"
#include "opencv2/geometry.hpp"
#include <climits>
using namespace std;
namespace cv{
namespace face{
// Threading helper classes
class getDiffShape : public ParallelLoopBody
{
public:
getDiffShape(vector<training_sample>* samples_) :
samples(samples_)
{
}
virtual void operator()( const cv::Range& range) const CV_OVERRIDE
{
for(size_t j = (size_t)range.start; j < (size_t)range.end; ++j){
(*samples)[j].shapeResiduals.resize((*samples)[j].current_shape.size());
for(unsigned long k=0;k<(*samples)[j].current_shape.size();k++)
(*samples)[j].shapeResiduals[k]=(*samples)[j].actual_shape[k]-(*samples)[j].current_shape[k];
}
}
private:
vector<training_sample>* samples;
};
class getRelPixels : public ParallelLoopBody
{
public:
getRelPixels(vector<training_sample>* samples_,FacemarkKazemiImpl& object_) :
samples(samples_),
object(object_)
{
}
virtual void operator()( const cv::Range& range) const CV_OVERRIDE
{
for (size_t j = (size_t)range.start; j < (size_t)range.end; ++j){
object.getRelativePixels(((*samples)[j]).current_shape,((*samples)[j]).pixel_coordinates);
}
}
private:
vector<training_sample>* samples;
FacemarkKazemiImpl& object;
};
//This function initialises the training parameters.
bool FacemarkKazemiImpl::setTrainingParameters(String filename){
cout << "Reading Training Parameters " << endl;
FileStorage fs;
fs.open(filename, FileStorage::READ);
if (!fs.isOpened())
{ String error_message = "Error while opening configuration file.Aborting..";
CV_Error(Error::StsBadArg, error_message);
}
int cascade_depth_;
int tree_depth_;
int num_trees_per_cascade_level_;
float learning_rate_;
int oversampling_amount_;
int num_test_coordinates_;
float lambda_;
int num_test_splits_;
fs["cascade_depth"]>> cascade_depth_;
fs["tree_depth"]>> tree_depth_;
fs["num_trees_per_cascade_level"] >> num_trees_per_cascade_level_;
fs["learning_rate"] >> learning_rate_;
fs["oversampling_amount"] >> oversampling_amount_;
fs["num_test_coordinates"] >> num_test_coordinates_;
fs["lambda"] >> lambda_;
fs["num_test_splits"] >> num_test_splits_;
params.cascade_depth = (unsigned long)cascade_depth_;
params.tree_depth = (unsigned long) tree_depth_;
params.num_trees_per_cascade_level = (unsigned long) num_trees_per_cascade_level_;
params.learning_rate = (float) learning_rate_;
params.oversampling_amount = (unsigned long) oversampling_amount_;
params.num_test_coordinates = (unsigned long) num_test_coordinates_;
params.lambda = (float) lambda_;
params.num_test_splits = (unsigned long) num_test_splits_;
fs.release();
cout<<"Parameters loaded"<<endl;
return true;
}
void FacemarkKazemiImpl::getTestCoordinates ()
{
for(unsigned long i = 0; i < params.cascade_depth; ++i){
vector<Point2f> temp;
RNG rng = theRNG();
for(unsigned long j = 0; j < params.num_test_coordinates; ++j)
{
Point2f pt;
pt.x = (float)rng.uniform(minmeanx,maxmeanx);
pt.y = (float)rng.uniform(minmeany,maxmeany);
temp.push_back(pt);
}
loaded_pixel_coordinates.push_back(temp);
}
}
unsigned long FacemarkKazemiImpl:: getNearestLandmark(Point2f pixel)
{
if(meanshape.empty()) {
// throw error if no data (or simply return -1?)
String error_message = "The data is not loaded properly by train function. Aborting...";
CV_Error(Error::StsBadArg, error_message);
}
float dist=float(INT_MAX);
unsigned long index =0;
for(unsigned long i=0;i<meanshape.size();i++){
Point2f pt = meanshape[i]-pixel;
if(sqrt(pt.x*pt.x+pt.y*pt.y)<dist){
dist=sqrt(pt.x*pt.x+pt.y*pt.y);
index = i;
}
}
return index;
}
bool FacemarkKazemiImpl :: getRelativePixels(vector<Point2f> sample,vector<Point2f>& pixel_coordinates,std::vector<int> nearest){
if(sample.size()!=meanshape.size()){
String error_message = "Error while finding relative shape. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
Mat transform_mat;
transform_mat = estimateAffinePartial2D(meanshape, sample);
unsigned long index;
for (unsigned long i = 0;i<pixel_coordinates.size();i++) {
if(!nearest.empty())
index = nearest[i];
index = getNearestLandmark(pixel_coordinates[i]);
pixel_coordinates[i] = pixel_coordinates[i] - meanshape[index];
Mat C = (Mat_<double>(3,1) << pixel_coordinates[i].x, pixel_coordinates[i].y, 0);
if(!transform_mat.empty()){
Mat D =transform_mat*C;
pixel_coordinates[i].x = float((D.at<double>(0,0)));
pixel_coordinates[i].y = float((D.at<double>(1,0)));
}
pixel_coordinates[i] = pixel_coordinates[i] + sample[index];
}
return true;
}
bool FacemarkKazemiImpl::getPixelIntensities(Mat img,vector<Point2f> pixel_coordinates,vector<int>& pixel_intensities,Rect face){
if(pixel_coordinates.size()==0){
String error_message = "No pixel coordinates found. Aborting.....";
CV_Error(Error::StsBadArg, error_message);
}
Mat transform_mat;
convertToActual(face,transform_mat);
Mat dst = img.clone();
Mat C,D;
for(size_t j=0;j<pixel_coordinates.size();j++){
C = (Mat_<double>(3,1) << pixel_coordinates[j].x, pixel_coordinates[j].y, 1);
D = transform_mat*C;
pixel_coordinates[j].x = float(D.at<double>(0,0));
pixel_coordinates[j].y = float(D.at<double>(1,0));
}
int val;
for(unsigned long j=0;j<pixel_coordinates.size();j++){
if(pixel_coordinates[j].x>0&&pixel_coordinates[j].x<img.cols&&pixel_coordinates[j].y>0&&pixel_coordinates[j].y<img.rows){
Vec3b val1 = img.at<Vec3b>((int)pixel_coordinates[j].y,(int)pixel_coordinates[j].x);
val = (int)(val1[0]+val1[1]+val1[2])/3;
}
else
val = 0;
pixel_intensities.push_back(val);
}
return true;
}
vector<regtree> FacemarkKazemiImpl::gradientBoosting(vector<training_sample>& samples,vector<Point2f> pixel_coordinates){
vector<regtree> forest;
vector<Point2f> meanresidual;
meanresidual.resize(samples[0].shapeResiduals.size());
for(unsigned long i=0;i<samples.size();i++){
for(unsigned long j=0;j<samples[i].shapeResiduals.size();j++){
meanresidual[j]=meanresidual[j]+samples[i].shapeResiduals[j];
}
}
for(unsigned long i=0;i<meanresidual.size();i++){
meanresidual[i].x=(meanresidual[i].x)/samples.size();
meanresidual[i].y=(meanresidual[i].y)/samples.size();
}
for(unsigned long i=0;i<samples.size();i++){
for(unsigned long j=0;j<samples[i].shapeResiduals.size();j++)
samples[i].shapeResiduals[j]=samples[i].shapeResiduals[j]-meanresidual[j];
}
for(unsigned long i=0;i<params.num_trees_per_cascade_level;i++){
regtree tree;
buildRegtree(tree,samples,pixel_coordinates);
forest.push_back(tree);
}
return forest;
}
bool FacemarkKazemiImpl::createTrainingSamples(vector<training_sample> &samples,vector<Mat> images,vector< vector<Point2f> > landmarks,vector<Rect> rectangle){
unsigned long in=0;
samples.resize(params.oversampling_amount*images.size());
for(unsigned long i=0;i<images.size();i++){
for(unsigned long j=0;j<params.oversampling_amount;j++){
samples[in].image=images[i];
samples[in].actual_shape = landmarks[i];
samples[in].bound = rectangle[i];
unsigned long rindex=i;
if(in%2==0)
samples[in].current_shape = meanshape;
else{
RNG rng(in);
rindex =(unsigned long)rng.uniform(0,(int)landmarks.size()-1);
samples[in].current_shape = landmarks[rindex];
}
in++;
}
}
parallel_for_(Range(0,(int)samples.size()),getDiffShape(&samples));
return true;
}
void FacemarkKazemiImpl :: writeLeaf(ofstream& os, const vector<Point2f> &leaf)
{
uint64_t size = leaf.size();
os.write((char*)&size, sizeof(size));
os.write((char*)&leaf[0], leaf.size() * sizeof(Point2f));
}
void FacemarkKazemiImpl :: writeSplit(ofstream& os, const splitr& vec)
{
os.write((char*)&vec.index1, sizeof(vec.index1));
os.write((char*)&vec.index2, sizeof(vec.index2));
os.write((char*)&vec.thresh, sizeof(vec.thresh));
uint32_t dummy_ = 0;
os.write((char*)&dummy_, sizeof(dummy_)); // buggy original writer structure alignment
CV_CheckEQ((int)(sizeof(vec.index1) + sizeof(vec.index2) + sizeof(vec.thresh) + sizeof(dummy_)), 24, "Invalid build configuration");
}
void FacemarkKazemiImpl :: writeTree(ofstream &f,regtree tree)
{
string s("num_nodes");
uint64_t len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
uint64_t num_nodes = tree.nodes.size();
f.write((char*)&num_nodes,sizeof(num_nodes));
for(size_t i=0;i<tree.nodes.size();i++){
if(tree.nodes[i].leaf.empty()){
s = string("split");
len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
writeSplit(f,tree.nodes[i].split);
}
else{
s = string("leaf");
len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
writeLeaf(f,tree.nodes[i].leaf);
}
}
}
void FacemarkKazemiImpl :: writePixels(ofstream& f,int index){
f.write((char*)&loaded_pixel_coordinates[index][0], loaded_pixel_coordinates[index].size() * sizeof(Point2f));
}
bool FacemarkKazemiImpl :: saveModel(String filename){
ofstream f(filename.c_str(),ios::binary);
if(!f.is_open()){
String error_message = "Error while opening file to write model. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
if(loaded_forests.size()!=loaded_pixel_coordinates.size()){
String error_message = "Incorrect training data. Aborting....";
CV_Error(Error::StsBadArg, error_message);
}
string s("cascade_depth");
uint64_t len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
uint64_t cascade_size = loaded_forests.size();
f.write((char*)&cascade_size,sizeof(cascade_size));
s = string("pixel_coordinates");
len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
uint64_t num_pixels = loaded_pixel_coordinates[0].size();
f.write((char*)&num_pixels,sizeof(num_pixels));
for(unsigned long i=0;i< loaded_pixel_coordinates.size();i++){
writePixels(f,i);
}
s = string("mean_shape");
uint64_t len1 = s.size();
f.write((char*)&len1, sizeof(len1));
f.write(s.c_str(), len1);
uint64_t mean_shape_size = meanshape.size();
f.write((char*)&mean_shape_size,sizeof(mean_shape_size));
f.write((char*)&meanshape[0], meanshape.size() * sizeof(Point2f));
s = string("num_trees");
len = s.size();
f.write((char*)&len, sizeof(len));
f.write(s.c_str(), len);
uint64_t num_trees = loaded_forests[0].size();
f.write((char*)&num_trees,sizeof(num_trees));
for(unsigned long i=0 ; i<loaded_forests.size() ; i++){
for(unsigned long j=0 ; j<loaded_forests[i].size() ; j++){
writeTree(f,loaded_forests[i][j]);
}
}
return true;
}
void FacemarkKazemiImpl::training(String imageList, String groundTruth){
imageList.clear();
groundTruth.clear();
String error_message = "Less arguments than required";
CV_Error(Error::StsBadArg, error_message);
}
bool FacemarkKazemiImpl::training(vector<Mat>& images, vector< vector<Point2f> >& landmarks,string filename,Size scale,string modelFilename){
if(!setTrainingParameters(filename)){
String error_message = "Error while loading training parameters";
CV_Error(Error::StsBadArg, error_message);
}
vector<Rect> rectangles;
scaleData(landmarks,images,scale);
calcMeanShape(landmarks,images,rectangles);
if(images.size()!=landmarks.size()){
// throw error if no data (or simply return -1?)
String error_message = "The data is not loaded properly. Aborting training function....";
CV_Error(Error::StsBadArg, error_message);
}
vector<training_sample> samples;
getTestCoordinates();
createTrainingSamples(samples,images,landmarks,rectangles);
images.clear();
landmarks.clear();
rectangles.clear();
for(unsigned long i=0;i< params.cascade_depth;i++){
cout<<"Training regressor "<<i<<"..."<<endl;
for (std::vector<training_sample>::iterator it = samples.begin(); it != samples.end(); it++) {
(*it).pixel_coordinates = loaded_pixel_coordinates[i];
}
parallel_for_(Range(0,(int)samples.size()),getRelPixels(&samples,*this));
for (std::vector<training_sample>::iterator it = samples.begin(); it != samples.end(); it++) {
getPixelIntensities((*it).image,(*it).pixel_coordinates,(*it).pixel_intensities,(*it).bound);
}
loaded_forests.push_back(gradientBoosting(samples,loaded_pixel_coordinates[i]));
}
saveModel(modelFilename);
return true;
}
}//cv
}//face
modules/face/test/test_bif.cpp
+71
View File
@@ -0,0 +1,71 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#include "test_precomp.hpp"
namespace opencv_test { namespace {
TEST(CV_Face_BIF, can_create_default) {
cv::Ptr<cv::face::BIF> bif;
EXPECT_NO_THROW(bif = cv::face::BIF::create());
EXPECT_FALSE(bif.empty());
}
TEST(CV_Face_BIF, fails_when_zero_bands) {
EXPECT_ANY_THROW(cv::face::BIF::create(0));
}
TEST(CV_Face_BIF, fails_when_too_many_bands) {
EXPECT_ANY_THROW(cv::face::BIF::create(9));
}
TEST(CV_Face_BIF, fails_when_zero_rotations) {
EXPECT_ANY_THROW(cv::face::BIF::create(8, 0));
}
TEST(CV_Face_BIF, can_compute) {
cv::Mat image(60, 60, CV_32F);
cv::theRNG().fill(image, cv::RNG::UNIFORM, -1, 1);
cv::Ptr<cv::face::BIF> bif = cv::face::BIF::create();
cv::Mat fea;
EXPECT_NO_THROW(bif->compute(image, fea));
EXPECT_EQ(cv::Size(1, 13188), fea.size());
}
}} // namespace
modules/face/test/test_face_align.cpp
+93
View File
@@ -0,0 +1,93 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "test_precomp.hpp"
namespace opencv_test { namespace {
using namespace cv::face;
static bool myDetector( InputArray image, OutputArray ROIs, CascadeClassifier* face_cascade)
{
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade->detectMultiScale( gray, faces, 1.1, 3, 0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
TEST(CV_Face_FacemarkKazemi, can_create_default) {
string cascade_name = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
string configfile_name = cvtest::findDataFile("face/config.xml", true);
CascadeClassifier face_cascade;
EXPECT_TRUE(face_cascade.load(cascade_name));
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
EXPECT_FALSE(facemark.empty());
}
TEST(CV_Face_FacemarkKazemi, can_loadTrainingData) {
string filename = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
string configfile_name = cvtest::findDataFile("face/config.xml", true);
CascadeClassifier face_cascade;
EXPECT_TRUE(face_cascade.load(filename));
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
vector<String> filenames;
filename = cvtest::findDataFile("face/1.txt", true);
filenames.push_back(filename);
filename = cvtest::findDataFile("face/2.txt", true);
filenames.push_back(filename);
vector<String> imagenames;
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
vector<Rect> rectangles;
//Test getData function
EXPECT_NO_THROW(loadTrainingData(filenames,trainlandmarks,imagenames));
vector<Mat> trainimages;
for(unsigned long i=0;i<imagenames.size();i++){
string img = cvtest::findDataFile(imagenames[i], true);
Mat src = imread(img);
EXPECT_TRUE(!src.empty());
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
string modelfilename = "face_landmark_model.dat";
Size scale = Size(460,460);
EXPECT_TRUE(facemark->training(trainimages,Trainlandmarks,configfile_name,scale,modelfilename));
}
TEST(CV_Face_FacemarkKazemi, can_detect_landmarks) {
string cascade_name = cvtest::findDataFile("face/lbpcascade_frontalface_improved.xml", true);
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
Ptr<FacemarkKazemi> facemark;
EXPECT_NO_THROW(facemark = FacemarkKazemi::create(params));
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)myDetector, &face_cascade));
string imgname = cvtest::findDataFile("face/detect.jpg");
string modelfilename = cvtest::findDataFile("face/face_landmark_model.dat",true);
Mat img = imread(imgname);
EXPECT_TRUE(!img.empty());
EXPECT_FALSE(facemark.empty());
EXPECT_NO_THROW(facemark->loadModel(modelfilename));
vector<Rect> faces;
//Detect faces in the current image
EXPECT_TRUE(facemark->getFaces(img,faces));
//vector to store the landmarks of all the faces in the image
vector< vector<Point2f> > shapes;
EXPECT_NO_THROW(facemark->fit(img,faces,shapes));
shapes.clear();
}
}} // namespace
modules/face/test/test_facemark.cpp
+58
View File
@@ -0,0 +1,58 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
#include "test_precomp.hpp"
namespace opencv_test { namespace {
TEST(CV_Face_Facemark, test_utilities) {
string image_file = cvtest::findDataFile("face/david1.jpg", true);
string annotation_file = cvtest::findDataFile("face/david1.pts", true);
string cascade_filename =
cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
std::vector<Point2f> facial_points;
EXPECT_NO_THROW(loadFacePoints(annotation_file,facial_points));
Mat img = imread(image_file);
EXPECT_NO_THROW(drawFacemarks(img, facial_points, Scalar(0,0,255)));
CParams params(cascade_filename);
std::vector<Rect> faces;
EXPECT_TRUE(getFaces(img, faces, &params));
}
}} // namespace
modules/face/test/test_facemark_aam.cpp
+141
View File
@@ -0,0 +1,141 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
/*Usage:
download the opencv_extra from https://github.com/opencv/opencv_extra
and then execute the following commands:
export OPENCV_TEST_DATA_PATH=/home/opencv/opencv_extra/testdata
<build_folder>/bin/opencv_test_face
*/
#include "test_precomp.hpp"
namespace opencv_test { namespace {
static bool customDetector( InputArray image, OutputArray ROIs, CascadeClassifier *face_detector){
Mat gray;
std::vector<Rect> & faces = *(std::vector<Rect>*) ROIs.getObj();
faces.clear();
if(image.channels()>1){
cvtColor(image.getMat(),gray, COLOR_BGR2GRAY);
}else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_detector->detectMultiScale( gray, faces, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30) );
return true;
}
TEST(CV_Face_FacemarkAAM, can_create_default) {
FacemarkAAM::Params params;
Ptr<FacemarkAAM> facemark;
EXPECT_NO_THROW(facemark = FacemarkAAM::create(params));
EXPECT_FALSE(facemark.empty());
}
TEST(CV_Face_FacemarkAAM, can_set_custom_detector) {
string cascade_filename =
cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
CascadeClassifier face_detector;
EXPECT_TRUE(face_detector.load(cascade_filename));
Ptr<FacemarkAAM> facemark = FacemarkAAM::create();
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)customDetector, &face_detector));
}
TEST(CV_Face_FacemarkAAM, test_workflow) {
string i1 = cvtest::findDataFile("face/david1.jpg", true);
string p1 = cvtest::findDataFile("face/david1.pts", true);
string i2 = cvtest::findDataFile("face/david2.jpg", true);
string p2 = cvtest::findDataFile("face/david2.pts", true);
std::vector<string> images_train;
images_train.push_back(i1);
images_train.push_back(i2);
std::vector<String> points_train;
points_train.push_back(p1);
points_train.push_back(p2);
string cascade_filename =
cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
CascadeClassifier face_detector;
EXPECT_TRUE(face_detector.load(cascade_filename));
FacemarkAAM::Params params;
params.n = 1;
params.m = 1;
params.verbose = false;
params.save_model = false;
Ptr<FacemarkAAM> facemark = FacemarkAAM::create(params);
Mat image;
std::vector<Point2f> landmarks;
for(size_t i = 0; i < images_train.size(); i++)
{
image = imread(images_train[i].c_str());
EXPECT_TRUE(loadFacePoints(points_train[i].c_str(),landmarks));
EXPECT_TRUE(landmarks.size()>0);
EXPECT_TRUE(facemark->addTrainingSample(image, landmarks));
}
EXPECT_NO_THROW(facemark->training());
/*------------ Fitting Part ---------------*/
EXPECT_TRUE(facemark->setFaceDetector((cv::face::FN_FaceDetector)customDetector, &face_detector));
string image_filename = cvtest::findDataFile("face/david1.jpg", true);
image = imread(image_filename.c_str());
EXPECT_TRUE(!image.empty());
std::vector<Rect> rects;
std::vector<std::vector<Point2f> > facial_points;
EXPECT_TRUE(facemark->getFaces(image, rects));
EXPECT_TRUE(rects.size()>0);
EXPECT_TRUE(facemark->fit(image, rects, facial_points));
EXPECT_TRUE(facial_points[0].size()>0);
/*------------ Test getData ---------------*/
FacemarkAAM::Data data;
EXPECT_TRUE(facemark->getData(&data));
EXPECT_TRUE(data.s0.size()>0);
}
}} // namespace
modules/face/test/test_facemark_lbf.cpp
+140
View File
@@ -0,0 +1,140 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
This file was part of GSoC Project: Facemark API for OpenCV
Final report: https://gist.github.com/kurnianggoro/74de9121e122ad0bd825176751d47ecc
Student: Laksono Kurnianggoro
Mentor: Delia Passalacqua
*/
/*Usage:
download the opencv_extra from https://github.com/opencv/opencv_extra
and then execute the following commands:
export OPENCV_TEST_DATA_PATH=/home/opencv/opencv_extra/testdata
<build_folder>/bin/opencv_test_face
*/
#include "test_precomp.hpp"
namespace opencv_test { namespace {
CascadeClassifier cascade_detector;
static bool myCustomDetector( InputArray image, OutputArray ROIs, void * config = 0 ){
Mat gray;
std::vector<Rect> & faces = *(std::vector<Rect>*) ROIs.getObj();
faces.clear();
if(config!=0){
//do nothing
}
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
cascade_detector.detectMultiScale( gray, faces, 1.4, 2, CASCADE_SCALE_IMAGE, Size(30, 30) );
return true;
}
TEST(CV_Face_FacemarkLBF, can_create_default) {
FacemarkLBF::Params params;
params.n_landmarks = 68;
Ptr<FacemarkLBF> facemark;
EXPECT_NO_THROW(facemark = FacemarkLBF::create(params));
EXPECT_FALSE(facemark.empty());
}
TEST(CV_Face_FacemarkLBF, can_set_custom_detector) {
string cascade_filename =
cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
EXPECT_TRUE(cascade_detector.load(cascade_filename));
Ptr<FacemarkLBF> facemark = FacemarkLBF::create();
EXPECT_TRUE(facemark->setFaceDetector(myCustomDetector));
}
TEST(CV_Face_FacemarkLBF, test_workflow) {
string i1 = cvtest::findDataFile("face/david1.jpg", true);
string p1 = cvtest::findDataFile("face/david1.pts", true);
string i2 = cvtest::findDataFile("face/david2.jpg", true);
string p2 = cvtest::findDataFile("face/david2.pts", true);
std::vector<string> images_train;
images_train.push_back(i1);
images_train.push_back(i2);
std::vector<String> points_train;
points_train.push_back(p1);
points_train.push_back(p2);
string cascade_filename =
cvtest::findDataFile("cascadeandhog/cascades/lbpcascade_frontalface.xml", true);
FacemarkLBF::Params params;
params.cascade_face = cascade_filename;
params.verbose = false;
params.save_model = false;
Ptr<FacemarkLBF> facemark = FacemarkLBF::create(params);
Mat image;
std::vector<Point2f> landmarks;
for(size_t i=0;i<images_train.size();i++){
image = imread(images_train[i].c_str());
EXPECT_TRUE(loadFacePoints(points_train[i].c_str(),landmarks));
EXPECT_TRUE(landmarks.size()>0);
EXPECT_TRUE(facemark->addTrainingSample(image, landmarks));
}
EXPECT_NO_THROW(facemark->training());
/*------------ Fitting Part ---------------*/
cascade_detector.load(cascade_filename);
facemark->setFaceDetector(myCustomDetector);
string image_filename = cvtest::findDataFile("face/david1.jpg", true);
image = imread(image_filename.c_str());
EXPECT_TRUE(!image.empty());
std::vector<Rect> rects;
std::vector<std::vector<Point2f> > facial_points;
EXPECT_TRUE(facemark->getFaces(image, rects));
EXPECT_TRUE(rects.size()>0);
EXPECT_TRUE(facemark->fit(image, rects, facial_points));
EXPECT_TRUE(facial_points[0].size()>0);
}
}} // namespace
modules/face/test/test_loadsave.cpp
+84
View File
@@ -0,0 +1,84 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#include "test_precomp.hpp"
namespace opencv_test { namespace {
// regression for #1267
// let's make sure, that both Algorithm::save(String) and
// FaceRecognizer::write(String) lead to the same result
void make_test_data(std::vector<cv::Mat> &images, std::vector<int> &labels) {
for (int i=0; i<5; i++) {
cv::Mat m(100,100,CV_8U);
cv::randu(m,0,255);
images.push_back(m);
labels.push_back(i);
}
}
TEST(CV_Face_SAVELOAD, use_save) {
std::vector<cv::Mat> images;
std::vector<int> labels;
make_test_data(images, labels);
cv::Ptr<cv::face::FaceRecognizer> model1 = cv::face::LBPHFaceRecognizer::create();
model1->train(images,labels);
model1->save("fr.xml");
int p1 = model1->predict(images[2]);
cv::Ptr<cv::face::FaceRecognizer> model2 = cv::face::LBPHFaceRecognizer::create();
model2->read("fr.xml");
EXPECT_EQ(model2->empty(), false);
EXPECT_EQ(p1, model2->predict(images[2]));
}
TEST(CV_Face_SAVELOAD, use_write) {
std::vector<cv::Mat> images;
std::vector<int> labels;
make_test_data(images, labels);
cv::Ptr<cv::face::FaceRecognizer> model1 = cv::face::LBPHFaceRecognizer::create();
model1->train(images,labels);
model1->write("fr.xml");
int p1 = model1->predict(images[2]);
cv::Ptr<cv::face::FaceRecognizer> model2 = cv::face::LBPHFaceRecognizer::create();
model2->read("fr.xml");
EXPECT_EQ(model2->empty(), false);
EXPECT_EQ(p1, model2->predict(images[2]));
}
}} // namespace
modules/face/test/test_mace.cpp
+68
View File
@@ -0,0 +1,68 @@
// This file is part of the OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#include "test_precomp.hpp"
#include <fstream>
namespace opencv_test { namespace {
const string FACE_DIR = "face";
const int WINDOW_SIZE = 64;
class MaceTest
{
public:
MaceTest(bool salt);
void run();
protected:
Ptr<MACE> mace;
bool salt;
};
MaceTest::MaceTest(bool use_salt)
{
mace = MACE::create(WINDOW_SIZE);
salt = use_salt;
}
void MaceTest::run()
{
Rect david1 (125,66,58,56);
Rect david2 (132,69,73,74);
Rect detect (199,124,256,274);
string folder = cvtest::TS::ptr()->get_data_path() + FACE_DIR;
Mat train = imread(folder + "/david2.jpg", 0);
Mat tst_p = imread(folder + "/david1.jpg", 0);
Mat tst_n = imread(folder + "/detect.jpg", 0);
vector<Mat> sam_train;
sam_train.push_back( train(Rect(132,69,73,74)) );
sam_train.push_back( train(Rect(130,69,73,72)) );
sam_train.push_back( train(Rect(134,67,73,74)) );
sam_train.push_back( tst_p(Rect(125,66,58,56)) );
sam_train.push_back( tst_p(Rect(123,67,55,58)) );
sam_train.push_back( tst_p(Rect(125,65,58,60)) );
if (salt) mace->salt("it's david"); // "owner's" salt
mace->train(sam_train);
bool self_ok = mace->same(train(david2));
if (salt) mace->salt("this is a test"); // "other's" salt
bool false_A = mace->same(tst_n(detect));
ASSERT_TRUE(self_ok);
ASSERT_FALSE(false_A);
}
TEST(MACE_, unsalted)
{
MaceTest test(false); test.run();
}
TEST(MACE_, salted)
{
MaceTest test(true); test.run();
}
}} // namespace
modules/face/test/test_main.cpp
+41
View File
@@ -0,0 +1,41 @@
/*
By downloading, copying, installing or using the software you agree to this
license. If you do not agree to this license, do not download, install,
copy or use the software.
License Agreement
For Open Source Computer Vision Library
(3-clause BSD License)
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
Third party copyrights are property of their respective owners.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the names of the copyright holders nor the names of the contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
This software is provided by the copyright holders and contributors "as is" and
any express or implied warranties, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose are
disclaimed. In no event shall copyright holders or contributors be liable for
any direct, indirect, incidental, special, exemplary, or consequential damages
(including, but not limited to, procurement of substitute goods or services;
loss of use, data, or profits; or business interruption) however caused
and on any theory of liability, whether in contract, strict liability,
or tort (including negligence or otherwise) arising in any way out of
the use of this software, even if advised of the possibility of such damage.
*/
#include "test_precomp.hpp"
CV_TEST_MAIN("cv")
modules/face/test/test_precomp.hpp
+17
View File
@@ -0,0 +1,17 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
#ifndef __OPENCV_TEST_PRECOMP_HPP__
#define __OPENCV_TEST_PRECOMP_HPP__
#include "opencv2/ts.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/xobjdetect.hpp"
#include "opencv2/face.hpp"
#include "opencv2/face/bif.hpp"
namespace opencv_test {
using namespace cv::face;
}
#endif
modules/face/tutorials/face_landmark/face_landmark_detection.markdown
+91
View File
@@ -0,0 +1,91 @@
Face landmark detection in an image {#tutorial_face_landmark_detection_in_an_image}
===================================
![](images/facereg.jpg)
This application lets you detect landmarks of detected faces in an image. You can detect landmarks of all the faces found in an image
and use them further in various applications like face swapping, face averaging etc.
This functionality is now available in OpenCV.
```
// Command to be typed for running the sample
./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image=/path_to_image/image.jpg
```
### Description of command parameters {tutorial_face_training_parameters}
> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
> * **image** i : (REQUIRED) A path to image in which face landmarks have to be detected.[example - /data/image.jpg]
> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
Understanding code
------------------
![](images/d.png)
This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
face_cascade.load(cascade_name);
Mat img = imread(image);
Ptr<Facemark> facemark = createFacemarkKazemi());
facemark->loadModel(filename);
cout<<"Loaded model"<<endl;
```
The above code creates a CascadeClassifier to detect face regions, and an instance of the face landmark detection class.
We need to load a pretrained model for face landmark detection, and a cascade file for the face detection.
It also loads the image in which landmarks have to be detected.
``` c++
vector<Rect> faces;
resize(img,img,Size(460,460),0,0,INTER_LINEAR_EXACT);
Mat gray;
std::vector<Rect> faces;
if(img.channels()>1){
cvtColor(img.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = img.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
```
After doing some preprocessing, we first have to detect possible face regions (which will be stored in a `vector<Rect>`.
Also, the image is resized to a smaller size as processing speed is faster with small images.
``` c++
vector< vector<Point2f> > shapes;
if (facemark->fit(img,faces,shapes))
{
for ( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
for (unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],5,cv::Scalar(0,0,255),FILLED);
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
waitKey(0);
}
```
It then creates a vector of vector to store shapes for each face detected.
The above code calls the function fit to get shapes of all detected faces in the image
and then draws the rectangles bounding the faces and marks the desired landmarks.
### Detection Results
![](ab.jpg)
![](ab-1.jpg)
modules/face/tutorials/face_landmark/face_landmark_trainer.markdown
+177
View File
@@ -0,0 +1,177 @@
![](face_landmark/images/2.jpg)
Training face landmark detector{#tutorial_face_training_face_landmark_detector}
==============================
This application helps to train your own face landmark detector. You can train your own face landmark detection by just providing the paths for
directory containing the images and files containing their corresponding face landmarks. As this landmark detector was originally trained on
[HELEN dataset](http://www.ifp.illinois.edu/~vuongle2/helen/), the training follows the format of data provided in HELEN dataset.
The dataset consists of .txt files whose first line contains the image name which then follows the annotations.
The format of the file containing annotations should be of following format :
> /directory/images/abc.jpg
> 123.45,345.65
> 321.67,543.89
> .... , ....
> .... , ....
The above format is similar to HELEN dataset which is used for training the model.
```
// Command to be typed for running the sample
./sample_train_landmark_detector -annotations=/home/sukhad/Downloads/code/trainset/ -config=config.xml -face_cascade=lbpcascadefrontalface.xml -model=trained_model.dat -width=460 -height=460
```
## Description of command parameters
> * **annotations** a : (REQUIRED) Path to annotations txt file [example - /data/annotations.txt]
> * **config** c : (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/config.xml]
> * **model** m : (REQUIRED) Path to configuration xml file containing parameters for training.[ example - /data/model.dat]
> * **width** w : (OPTIONAL) The width which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
> * **height** h : (OPTIONAL) The height which you want all images to get to scale the annotations. Large images are slow to process [default = 460]
> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a detector.
## Description of training parameters
The configuration file described above which is used while training contains the training parameters which are required for training.
**The description of parameters is as follows :**
1. **Cascade depth :** This stores the depth of cascade of regressors used for training.
2. **Tree depth :** This stores the depth of trees created as weak learners during gradient boosting.
3. **Number of trees per cascade level :** This stores number of trees required per cascade level.
4. **Learning rate :** This stores the learning rate for gradient boosting.This is required to prevent overfitting using shrinkage.
5. **Oversampling amount :** This stores the oversampling amount for the samples.
6. **Number of test coordinates :** This stores number of test coordinates to be generated as samples to decide for making the split.
7. **Lambda :** This stores the value used for calculating the probabilty which helps to select closer pixels for making the split.
8. **Number of test splits :** This stores the number of test splits to be generated before making the best split.
To get more detailed description about the training parameters you can refer to the [Research paper](https://pdfs.semanticscholar.org/d78b/6a5b0dcaa81b1faea5fb0000045a62513567.pdf).
## Understanding code
![](face_landmark/images/3.jpg)
Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in training.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
vector<String> filenames;
glob(directory,filenames);
```
The above code creates a vector filenames for storing the names of the .txt files.
It gets the filenames of the files in the directory.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces while training the model.
``` c++
vector<String> imagenames;
vector< vector<Point2f> > trainlandmarks,Trainlandmarks;
vector<Mat> trainimages;
loadTrainingData(filenames,trainlandmarks,imagenames);
for(unsigned long i=0;i<300;i++){
string imgname = imagenames[i].substr(0, imagenames[i].size()-1);
string img = directory + string(imgname) + ".jpg";
Mat src = imread(img);
if(src.empty()){
cerr<<string("Image "+img+" not found\n.")<<endl;
continue;
}
trainimages.push_back(src);
Trainlandmarks.push_back(trainlandmarks[i]);
}
```
The above code creates std::vectors to store the images and their corresponding landmarks.
The above code calls a function loadTrainingData to load the landmarks and the images into their respective vectors.
If the dataset you downloaded is of the following format :
```
version: 1
n_points: 68
{
115.167660 220.807529
116.164839 245.721357
120.208690 270.389841
...
}
This is the example of the dataset available at https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
```
Then skip the above code for loading training data and use the following code. This sample is provided as sampleTrainLandmarkDetector2.cpp
in the face module in opencv contrib.
``` c++
std::vector<String> images;
std::vector<std::vector<Point2f> > facePoints;
loadTrainingData(imagesList, annotations, images, facePoints, 0.0);
```
In the above code imagelist and annotations are the file of following format :
```
example of contents for images.txt:
../trainset/image_0001.png
../trainset/image_0002.png
example of contents for annotation.txt:
../trainset/image_0001.pts
../trainset/image_0002.pts
```
These symbolize the names of images and their corresponding annotations.
The above code scales images and landmarks as training on images of smaller size takes less time.
This is because processing larger images requires more time. After scaling data it calculates mean
shape of the data which is used as initial shape while training.
Finally call the following function to perform training :
``` c++
facemark->training(Trainimages,Trainlandmarks,configfile_name,scale,modelfile_name);
```
In the above function scale is passed to scale all images and the corresponding landmarks so that the size of all
images can be reduced as it takes greater time to process large images.
This call to the train function trains the model and stores the trained model file with the given
filename specified.As the training starts successfully you will see something like this :
![](face_landmark/images/train1.png)
**The error rate on trained images depends on the number of images used for training used as follows :**
![](face_landmark/images/train.png)
**The error rate on test images depends on the number of images used for training used as follows :**
![](face_landmark/images/test.png)
modules/face/tutorials/face_landmark/face_landmark_video.markdown
+110
View File
@@ -0,0 +1,110 @@
Face landmark detection in a video{#tutorial_face_landmark_detection_in_video}
===================================
This application lets you detect landmarks of detected faces in a video.This application first detects faces in a current video frame
and then finds their facial landmarks. You just have to pass the video as input.
```
// Command to be typed for running the sample
./sampleDetectLandmarks -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -video=/path_to_video/video.avi
```
Description of command parameters
---------------------------------
> * **model_filename** f : (REQUIRED) A path to binary file storing the trained model which is to be loaded [example - /data/file.dat]
> * **video** v : (REQUIRED) A path to video in which face landmarks have to be detected.[example - /data/video.avi]
> * **face_cascade** c : (REQUIRED) A path to the face cascade xml file which you want to use as a face detector.
### Understanding code
This tutorial will explain the sample code for face landmark detection. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
VideoCapture cap(video);
if(!cap.isOpened()){
cerr<<"Video cannot be loaded. Give correct path"<<endl;
return -1;
}
```
The above code creates a video capture object and then loads the video.
If the video is not loaded properly it prompts the user else the code proceeds.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces.
``` c++
vector<Rect> faces;
vector< vector<Point2f> > shapes;
Mat img;
```
The above code creates a vector to store the detected faces and a vector of vector to store shapes for each
face detected in the current frame.
``` c++
while(1){
faces.clear();
shapes.clear();
cap>>img;
resize(img,img,Size(600,600),0,0,INTER_LINEAR_EXACT);
facemark->getFaces(img,faces);
if(faces.size()==0){
cout<<"No faces found in this frame"<<endl;
}
else{
for( size_t i = 0; i < faces.size(); i++ )
{
cv::rectangle(img,faces[i],Scalar( 255, 0, 0 ));
}
if(facemark->fit(img,faces,shapes))
{
for(unsigned long i=0;i<faces.size();i++){
for(unsigned long k=0;k<shapes[i].size();k++)
cv::circle(img,shapes[i][k],3,cv::Scalar(0,0,255),FILLED);
}
}
}
namedWindow("Detected_shape");
imshow("Detected_shape",img);
if(waitKey(1) >= 0) break;
}
```
The above code then reads each frame and detects faces and the landmarks corresponding to each shape detected.
It then displays the current frame.
After running the above code you will get results something like this
Sample video:
@htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/ZtaV07T90D8" frameborder="0" allowfullscreen></iframe>
@endhtmlonly
modules/face/tutorials/face_landmark/images/2.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 221 KiB

modules/face/tutorials/face_landmark/images/227943776_1.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 95 KiB

modules/face/tutorials/face_landmark/images/230501201_1.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 270 KiB

modules/face/tutorials/face_landmark/images/3.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

modules/face/tutorials/face_landmark/images/ab-1.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

modules/face/tutorials/face_landmark/images/ab.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

modules/face/tutorials/face_landmark/images/d.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 64 KiB

modules/face/tutorials/face_landmark/images/face_swapped.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

modules/face/tutorials/face_landmark/images/facereg.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 45 KiB

modules/face/tutorials/face_landmark/images/test.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

modules/face/tutorials/face_landmark/images/train.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

modules/face/tutorials/face_landmark/images/train1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

modules/face/tutorials/face_landmark/sample_face_swapping.markdown
+147
View File
@@ -0,0 +1,147 @@
Face swapping using face landmark detection{#tutorial_face_swapping_face_landmark_detection}
===========================================
This application lets you swap a face in one image with another face in other image. The application first detects faces in both images and finds its landmarks. Then it swaps the face in first image with in another image. You just have to give paths to the images run the application to swap the two faces.
```
// Command to be typed for running the sample
./sample_face_swapping -file=trained_model.dat -face_cascade=lbpcascadefrontalface.xml -image1=/path_to_image/image1.jpg -image2=/path_to_image/image2.jpg
```
### Description of command parameters
> * **image1** i1 (REQUIRED) Path to the first image file in which you want to apply swapping.
> * **image2** i2 (REQUIRED) Path to the second image file in which you want to apply face swapping.
> * **model** m (REQUIRED) Path to the file containing model to be loaded for face landmark detection.
> * **face_cascade** f (REQUIRED) Path to the face cascade xml file which you want to use as a face detector.
### Understanding the code
This tutorial will explain the sample code for face swapping using OpenCV. Jumping directly to the code :
``` c++
CascadeClassifier face_cascade;
bool myDetector( InputArray image, OutputArray ROIs );
bool myDetector( InputArray image, OutputArray ROIs ){
Mat gray;
std::vector<Rect> faces;
if(image.channels()>1){
cvtColor(image.getMat(),gray,COLOR_BGR2GRAY);
}
else{
gray = image.getMat().clone();
}
equalizeHist( gray, gray );
face_cascade.detectMultiScale( gray, faces, 1.1, 3,0, Size(30, 30) );
Mat(faces).copyTo(ROIs);
return true;
}
```
The facemark API provides the functionality to the user to use their own face detector to be used in face landmark detection.The above code creartes a sample face detector. The above function would be passed to a function pointer in the facemark API.
``` c++
Mat img = imread(image);
face_cascade.load(cascade_name);
FacemarkKazemi::Params params;
params.configfile = configfile_name;
Ptr<Facemark> facemark = FacemarkKazemi::create(params);
facemark->setFaceDetector(myDetector);
```
The above code creates a pointer of the face landmark detection class. The face detector created above has to be passed
as function pointer to the facemark pointer created for detecting faces while training the model.
``` c++
vector<Rect> faces1,faces2;
vector< vector<Point2f> > shape1,shape2;
float ratio1 = (float)img1.cols/(float)img1.rows;
float ratio2 = (float)img2.cols/(float)img2.rows;
resize(img1,img1,Size(640*ratio1,640*ratio1),0,0,INTER_LINEAR_EXACT);
resize(img2,img2,Size(640*ratio2,640*ratio2),0,0,INTER_LINEAR_EXACT);
Mat img1Warped = img2.clone();
facemark->getFaces(img1,faces1);
facemark->getFaces(img2,faces2);
facemark->fit(img1,faces1,shape1);
facemark->fit(img2,faces2,shape2);
```
The above code creates vectors to store the detected faces and a vector of vector to store shapes for each
face detected in both the images.It then detects landmarks of each face detected in both the images.the images are resized
as it is easier to process small images. The images are resized according their actual ratio.
``` c++
vector<Point2f> boundary_image1;
vector<Point2f> boundary_image2;
vector<int> index;
convexHull(Mat(points2),index, false, false);
for(size_t i = 0; i < index.size(); i++)
{
boundary_image1.push_back(points1[index[i]]);
boundary_image2.push_back(points2[index[i]]);
}
```
The above code then finds convex hull to find the boundary points of the face in the image which has to be swapped.
``` c++
vector< vector<int> > triangles;
Rect rect(0, 0, img1Warped.cols, img1Warped.rows);
divideIntoTriangles(rect, boundary_image2, triangles);
for(size_t i = 0; i < triangles.size(); i++)
{
vector<Point2f> triangle1, triangle2;
for(int j = 0; j < 3; j++)
{
triangle1.push_back(boundary_image1[triangles[i][j]]);
triangle2.push_back(boundary_image2[triangles[i][j]]);
}
warpTriangle(img1, img1Warped, triangle1, triangle2);
}
```
Now as we need to warp one face over the other and we need to find affine transform.
Now as the function in OpenCV to find affine transform requires three set of points to calculate
the affine matrix. Also we just need to warp the face instead of the surrounding regions. Hence
we divide the face into triangles so that each triiangle can be easily warped onto the other image.
The function divideIntoTriangles divides the detected faces into triangles.
The function warpTriangle then warps each triangle of one image to other image to swap the faces.
``` c++
vector<Point> hull;
for(size_t i = 0; i < boundary_image2.size(); i++)
{
Point pt((int)boundary_image2[i].x,(int)boundary_image2[i].y);
hull.push_back(pt);
}
Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());
fillConvexPoly(mask,&hull[0],(int)hull.size(), Scalar(255,255,255));
Rect r = boundingRect(boundary_image2);
Point center = (r.tl() + r.br()) / 2;
Mat output;
img1Warped.convertTo(img1Warped, CV_8UC3);
seamlessClone(img1Warped,img2, mask, center, output, NORMAL_CLONE);
imshow("Face_Swapped", output);
```
Even after warping the results somehow look unnatural. Hence to improve the results we apply seamless cloning
to get the desired results as required.
### Results
Consider two images to be used for face swapping as follows :
First image
-----------
![](images/227943776_1.jpg)
Second image
------------
![](images/230501201_1.jpg)
Results after swapping
----------------------
![](images/face_swapped.jpg)
modules/face/tutorials/face_recognition/face_tutorial.markdown
+699
View File
@@ -0,0 +1,699 @@
Face Recognition with OpenCV {#tutorial_face_main}
============================
[TOC]
Introduction {#tutorial_face_intro}
============
[OpenCV (Open Source Computer Vision)](http://opencv.org) is a popular computer vision library
started by [Intel](http://www.intel.com) in 1999. The cross-platform library sets its focus on
real-time image processing and includes patent-free implementations of the latest computer vision
algorithms. In 2008 [Willow Garage](http://www.willowgarage.com) took over support and OpenCV 2.3.1
now comes with a programming interface to C, C++, [Python](http://www.python.org) and
[Android](http://www.android.com). OpenCV is released under a BSD license so it is used in academic
projects and commercial products alike.
OpenCV 2.4 now comes with the very new FaceRecognizer class for face recognition, so you can start
experimenting with face recognition right away. This document is the guide I've wished for, when I
was working myself into face recognition. It shows you how to perform face recognition with
FaceRecognizer in OpenCV (with full source code listings) and gives you an introduction into the
algorithms behind. I'll also show how to create the visualizations you can find in many
publications, because a lot of people asked for.
The currently available algorithms are:
- Eigenfaces (see EigenFaceRecognizer::create)
- Fisherfaces (see FisherFaceRecognizer::create)
- Local Binary Patterns Histograms (see LBPHFaceRecognizer::create)
You don't need to copy and paste the source code examples from this page, because they are available
in the src folder coming with this documentation. If you have built OpenCV with the samples turned
on, chances are good you have them compiled already! Although it might be interesting for very
advanced users, I've decided to leave the implementation details out as I am afraid they confuse new
users.
All code in this document is released under the [BSD
license](http://www.opensource.org/licenses/bsd-license), so feel free to use it for your projects.
Face Recognition {#tutorial_face_facerec}
----------------
Face recognition is an easy task for humans. Experiments in @cite Tu06 have shown, that even one to
three day old babies are able to distinguish between known faces. So how hard could it be for a
computer? It turns out we know little about human recognition to date. Are inner features (eyes,
nose, mouth) or outer features (head shape, hairline) used for a successful face recognition? How do
we analyze an image and how does the brain encode it? It was shown by [David
Hubel](http://en.wikipedia.org/wiki/David_H._Hubel) and [Torsten
Wiesel](http://en.wikipedia.org/wiki/Torsten_Wiesel), that our brain has specialized nerve cells
responding to specific local features of a scene, such as lines, edges, angles or movement. Since we
don't see the world as scattered pieces, our visual cortex must somehow combine the different
sources of information into useful patterns. Automatic face recognition is all about extracting
those meaningful features from an image, putting them into a useful representation and performing
some kind of classification on them.
Face recognition based on the geometric features of a face is probably the most intuitive approach
to face recognition. One of the first automated face recognition systems was described in
@cite Kanade73 : marker points (position of eyes, ears, nose, ...) were used to build a feature vector
(distance between the points, angle between them, ...). The recognition was performed by calculating
the euclidean distance between feature vectors of a probe and reference image. Such a method is
robust against changes in illumination by its nature, but has a huge drawback: the accurate
registration of the marker points is complicated, even with state of the art algorithms. Some of the
latest work on geometric face recognition was carried out in @cite Bru92 . A 22-dimensional feature
vector was used and experiments on large datasets have shown, that geometrical features alone may not
carry enough information for face recognition.
The Eigenfaces method described in @cite TP91 took a holistic approach to face recognition: A facial
image is a point from a high-dimensional image space and a lower-dimensional representation is
found, where classification becomes easy. The lower-dimensional subspace is found with Principal
Component Analysis, which identifies the axes with maximum variance. While this kind of
transformation is optimal from a reconstruction standpoint, it doesn't take any class labels into
account. Imagine a situation where the variance is generated from external sources, let it be light.
The axes with maximum variance do not necessarily contain any discriminative information at all,
hence a classification becomes impossible. So a class-specific projection with a Linear Discriminant
Analysis was applied to face recognition in @cite BHK97 . The basic idea is to minimize the variance
within a class, while maximizing the variance between the classes at the same time.
Recently various methods for a local feature extraction emerged. To avoid the high-dimensionality of
the input data only local regions of an image are described, the extracted features are (hopefully)
more robust against partial occlusion, illumation and small sample size. Algorithms used for a local
feature extraction are Gabor Wavelets (@cite Wiskott97), Discrete Cosinus Transform (@cite Messer06) and
Local Binary Patterns (@cite AHP04). It's still an open research question what's the best way to
preserve spatial information when applying a local feature extraction, because spatial information
is potentially useful information.
Face Database {#tutorial_face_facedb}
-------------
Let's get some data to experiment with first. I don't want to do a toy example here. We are doing
face recognition, so you'll need some face images! You can either create your own dataset or start
with one of the available face databases,
[<http://face-rec.org/databases/>](http://face-rec.org/databases) gives you an up-to-date overview.
Three interesting databases are (parts of the description are quoted from
[<http://face-rec.org>](http://face-rec.org)):
- [AT&T Facedatabase](http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html) The AT&T
Facedatabase, sometimes also referred to as *ORL Database of Faces*, contains ten different
images of each of 40 distinct subjects. For some subjects, the images were taken at different
times, varying the lighting, facial expressions (open / closed eyes, smiling / not smiling) and
facial details (glasses / no glasses). All the images were taken against a dark homogeneous
background with the subjects in an upright, frontal position (with tolerance for some side
movement).
- [Yale Facedatabase A](http://vision.ucsd.edu/content/yale-face-database), also known as
Yalefaces. The AT&T Facedatabase is good for initial tests, but it's a fairly easy database. The
Eigenfaces method already has a 97% recognition rate on it, so you won't see any great
improvements with other algorithms. The Yale Facedatabase A (also known as Yalefaces) is a more
appropriate dataset for initial experiments, because the recognition problem is harder. The
database consists of 15 people (14 male, 1 female) each with 11 grayscale images sized
\f$320 \times 243\f$ pixel. There are changes in the light conditions (center light, left light,
right light), facial expressions (happy, normal, sad, sleepy, surprised, wink) and glasses
(glasses, no-glasses).
The original images are not cropped and aligned. Please look into the @ref face_appendix for a
Python script, that does the job for you.
- [Extended Yale Facedatabase B](http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html) The
Extended Yale Facedatabase B contains 2414 images of 38 different people in its cropped version.
The focus of this database is set on extracting features that are robust to illumination, the
images have almost no variation in emotion/occlusion/... . I personally think, that this dataset
is too large for the experiments I perform in this document. You better use the [AT&T
Facedatabase](http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html) for intial
testing. A first version of the Yale Facedatabase B was used in @cite BHK97 to see how the
Eigenfaces and Fisherfaces method perform under heavy illumination changes. @cite Lee05 used the
same setup to take 16128 images of 28 people. The Extended Yale Facedatabase B is the merge of
the two databases, which is now known as Extended Yalefacedatabase B.
### Preparing the data {#tutorial_face_prepare}
Once we have acquired some data, we'll need to read it in our program. In the demo applications I
have decided to read the images from a very simple CSV file. Why? Because it's the simplest
platform-independent approach I can think of. However, if you know a simpler solution please ping me
about it. Basically all the CSV file needs to contain are lines composed of a filename followed by a
; followed by the label (as *integer number*), making up a line like this:
@code{.csv}
/path/to/image.ext;0
@endcode
Let's dissect the line. /path/to/image.ext is the path to an image, probably something like this if
you are in Windows: C:/faces/person0/image0.jpg. Then there is the separator ; and finally we assign
the label 0 to the image. Think of the label as the subject (the person) this image belongs to, so
same subjects (persons) should have the same label.
Download the AT&T Facedatabase from AT&T Facedatabase and the corresponding CSV file from at.txt,
which looks like this (file is without ... of course):
@code{.csv}
./at/s1/1.pgm;0
./at/s1/2.pgm;0
...
./at/s2/1.pgm;1
./at/s2/2.pgm;1
...
./at/s40/1.pgm;39
./at/s40/2.pgm;39
@endcode
Imagine I have extracted the files to D:/data/at and have downloaded the CSV file to D:/data/at.txt.
Then you would simply need to Search & Replace ./ with D:/data/. You can do that in an editor of
your choice, every sufficiently advanced editor can do this. Once you have a CSV file with valid
filenames and labels, you can run any of the demos by passing the path to the CSV file as parameter:
@code{.sh}
facerec_demo.exe D:/data/at.txt
@endcode
Please, see @ref tutorial_face_appendix_csv for details on creating CSV file.
Eigenfaces {#tutorial_face_eigenfaces}
----------
The problem with the image representation we are given is its high dimensionality. Two-dimensional
\f$p \times q\f$ grayscale images span a \f$m = pq\f$-dimensional vector space, so an image with
\f$100 \times 100\f$ pixels lies in a \f$10,000\f$-dimensional image space already. The question is: Are all
dimensions equally useful for us? We can only make a decision if there's any variance in data, so
what we are looking for are the components that account for most of the information. The Principal
Component Analysis (PCA) was independently proposed by [Karl
Pearson](http://en.wikipedia.org/wiki/Karl_Pearson) (1901) and [Harold
Hotelling](http://en.wikipedia.org/wiki/Harold_Hotelling) (1933) to turn a set of possibly
correlated variables into a smaller set of uncorrelated variables. The idea is, that a
high-dimensional dataset is often described by correlated variables and therefore only a few
meaningful dimensions account for most of the information. The PCA method finds the directions with
the greatest variance in the data, called principal components.
### Algorithmic Description of Eigenfaces method {#tutorial_face_eigenfaces_algo}
Let \f$X = \{ x_{1}, x_{2}, \ldots, x_{n} \}\f$ be a random vector with observations \f$x_i \in R^{d}\f$.
1. Compute the mean \f$\mu\f$
\f[\mu = \frac{1}{n} \sum_{i=1}^{n} x_{i}\f]
2. Compute the the Covariance Matrix S
\f[S = \frac{1}{n} \sum_{i=1}^{n} (x_{i} - \mu) (x_{i} - \mu)^{T}`\f]
3. Compute the eigenvalues \f$\lambda_{i}\f$ and eigenvectors \f$v_{i}\f$ of \f$S\f$
\f[S v_{i} = \lambda_{i} v_{i}, i=1,2,\ldots,n\f]
4. Order the eigenvectors descending by their eigenvalue. The \f$k\f$ principal components are the
eigenvectors corresponding to the \f$k\f$ largest eigenvalues.
The \f$k\f$ principal components of the observed vector \f$x\f$ are then given by:
\f[y = W^{T} (x - \mu)\f]
where \f$W = (v_{1}, v_{2}, \ldots, v_{k})\f$.
The reconstruction from the PCA basis is given by:
\f[x = W y + \mu\f]
where \f$W = (v_{1}, v_{2}, \ldots, v_{k})\f$.
The Eigenfaces method then performs face recognition by:
- Projecting all training samples into the PCA subspace.
- Projecting the query image into the PCA subspace.
- Finding the nearest neighbor between the projected training images and the projected query
image.
Still there's one problem left to solve. Imagine we are given \f$400\f$ images sized \f$100 \times 100\f$
pixel. The Principal Component Analysis solves the covariance matrix \f$S = X X^{T}\f$, where
\f${size}(X) = 10000 \times 400\f$ in our example. You would end up with a \f$10000 \times 10000\f$ matrix,
roughly \f$0.8 GB\f$. Solving this problem isn't feasible, so we'll need to apply a trick. From your
linear algebra lessons you know that a \f$M \times N\f$ matrix with \f$M > N\f$ can only have \f$N - 1\f$
non-zero eigenvalues. So it's possible to take the eigenvalue decomposition \f$S = X^{T} X\f$ of size
\f$N \times N\f$ instead:
\f[X^{T} X v_{i} = \lambda_{i} v{i}\f]
and get the original eigenvectors of \f$S = X X^{T}\f$ with a left multiplication of the data matrix:
\f[X X^{T} (X v_{i}) = \lambda_{i} (X v_{i})\f]
The resulting eigenvectors are orthogonal, to get orthonormal eigenvectors they need to be
normalized to unit length. I don't want to turn this into a publication, so please look into
@cite Duda01 for the derivation and proof of the equations.
### Eigenfaces in OpenCV {#tutorial_face_eigenfaces_use}
For the first source code example, I'll go through it with you. I am first giving you the whole
source code listing, and after this we'll look at the most important lines in detail. Please note:
every source code listing is commented in detail, so you should have no problems following it.
The source code for this demo application is also available in the src folder coming with this
documentation:
@include face/samples/facerec_eigenfaces.cpp
I've used the jet colormap, so you can see how the grayscale values are distributed within the
specific Eigenfaces. You can see, that the Eigenfaces do not only encode facial features, but also
the illumination in the images (see the left light in Eigenface \#4, right light in Eigenfaces \#5):
![image](img/eigenfaces_opencv.png)
We've already seen, that we can reconstruct a face from its lower dimensional approximation. So
let's see how many Eigenfaces are needed for a good reconstruction. I'll do a subplot with
\f$10,30,\ldots,310\f$ Eigenfaces:
@code{.cpp}
// Display or save the image reconstruction at some predefined steps:
for(int num_components = 10; num_components < 300; num_components+=15) {
// slice the eigenvectors from the model
Mat evs = Mat(W, Range::all(), Range(0, num_components));
Mat projection = LDA::subspaceProject(evs, mean, images[0].reshape(1,1));
Mat reconstruction = LDA::subspaceReconstruct(evs, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);
} else {
imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);
}
}
@endcode
10 Eigenvectors are obviously not sufficient for a good image reconstruction, 50 Eigenvectors may
already be sufficient to encode important facial features. You'll get a good reconstruction with
approximately 300 Eigenvectors for the AT&T Facedatabase. There are rule of thumbs how many
Eigenfaces you should choose for a successful face recognition, but it heavily depends on the input
data. @cite Zhao03 is the perfect point to start researching for this:
![image](img/eigenface_reconstruction_opencv.png)
Fisherfaces {#tutorial_face_fisherfaces}
-----------
The Principal Component Analysis (PCA), which is the core of the Eigenfaces method, finds a linear
combination of features that maximizes the total variance in data. While this is clearly a powerful
way to represent data, it doesn't consider any classes and so a lot of discriminative information
*may* be lost when throwing components away. Imagine a situation where the variance in your data is
generated by an external source, let it be the light. The components identified by a PCA do not
necessarily contain any discriminative information at all, so the projected samples are smeared
together and a classification becomes impossible (see
[<http://www.bytefish.de/wiki/pca_lda_with_gnu_octave>](http://www.bytefish.de/wiki/pca_lda_with_gnu_octave)
for an example).
The Linear Discriminant Analysis performs a class-specific dimensionality reduction and was invented
by the great statistician [Sir R. A. Fisher](http://en.wikipedia.org/wiki/Ronald_Fisher). He
successfully used it for classifying flowers in his 1936 paper *The use of multiple measurements in
taxonomic problems* @cite Fisher36 . In order to find the combination of features that separates best
between classes the Linear Discriminant Analysis maximizes the ratio of between-classes to
within-classes scatter, instead of maximizing the overall scatter. The idea is simple: same classes
should cluster tightly together, while different classes are as far away as possible from each other
in the lower-dimensional representation. This was also recognized by
[Belhumeur](http://www.cs.columbia.edu/~belhumeur/), [Hespanha](http://www.ece.ucsb.edu/~hespanha/)
and [Kriegman](http://cseweb.ucsd.edu/~kriegman/) and so they applied a Discriminant Analysis to
face recognition in @cite BHK97 .
### Algorithmic Description of Fisherfaces method {#tutorial_face_fisherfaces_algo}
Let \f$X\f$ be a random vector with samples drawn from \f$c\f$ classes:
\f[\begin{align*}
X & = & \{X_1,X_2,\ldots,X_c\} \\
X_i & = & \{x_1, x_2, \ldots, x_n\}
\end{align*}\f]
The scatter matrices \f$S_{B}\f$ and S\_{W} are calculated as:
\f[\begin{align*}
S_{B} & = & \sum_{i=1}^{c} N_{i} (\mu_i - \mu)(\mu_i - \mu)^{T} \\
S_{W} & = & \sum_{i=1}^{c} \sum_{x_{j} \in X_{i}} (x_j - \mu_i)(x_j - \mu_i)^{T}
\end{align*}\f]
, where \f$\mu\f$ is the total mean:
\f[\mu = \frac{1}{N} \sum_{i=1}^{N} x_i\f]
And \f$\mu_i\f$ is the mean of class \f$i \in \{1,\ldots,c\}\f$:
\f[\mu_i = \frac{1}{|X_i|} \sum_{x_j \in X_i} x_j\f]
Fisher's classic algorithm now looks for a projection \f$W\f$, that maximizes the class separability
criterion:
\f[W_{opt} = \operatorname{arg\,max}_{W} \frac{|W^T S_B W|}{|W^T S_W W|}\f]
Following @cite BHK97, a solution for this optimization problem is given by solving the General
Eigenvalue Problem:
\f[\begin{align*}
S_{B} v_{i} & = & \lambda_{i} S_w v_{i} \nonumber \\
S_{W}^{-1} S_{B} v_{i} & = & \lambda_{i} v_{i}
\end{align*}\f]
There's one problem left to solve: The rank of \f$S_{W}\f$ is at most \f$(N-c)\f$, with \f$N\f$ samples and \f$c\f$
classes. In pattern recognition problems the number of samples \f$N\f$ is almost always samller than the
dimension of the input data (the number of pixels), so the scatter matrix \f$S_{W}\f$ becomes singular
(see @cite RJ91). In @cite BHK97 this was solved by performing a Principal Component Analysis on the
data and projecting the samples into the \f$(N-c)\f$-dimensional space. A Linear Discriminant Analysis
was then performed on the reduced data, because \f$S_{W}\f$ isn't singular anymore.
The optimization problem can then be rewritten as:
\f[\begin{align*}
W_{pca} & = & \operatorname{arg\,max}_{W} |W^T S_T W| \\
W_{fld} & = & \operatorname{arg\,max}_{W} \frac{|W^T W_{pca}^T S_{B} W_{pca} W|}{|W^T W_{pca}^T S_{W} W_{pca} W|}
\end{align*}\f]
The transformation matrix \f$W\f$, that projects a sample into the \f$(c-1)\f$-dimensional space is then
given by:
\f[W = W_{fld}^{T} W_{pca}^{T}\f]
### Fisherfaces in OpenCV {#tutorial_face_fisherfaces_use}
The source code for this demo application is also available in the src folder coming with this
documentation:
@include face/samples/facerec_fisherfaces.cpp
For this example I am going to use the Yale Facedatabase A, just because the plots are nicer. Each
Fisherface has the same length as an original image, thus it can be displayed as an image. The demo
shows (or saves) the first, at most 16 Fisherfaces:
![image](img/fisherfaces_opencv.png)
The Fisherfaces method learns a class-specific transformation matrix, so the they do not capture
illumination as obviously as the Eigenfaces method. The Discriminant Analysis instead finds the
facial features to discriminate between the persons. It's important to mention, that the performance
of the Fisherfaces heavily depends on the input data as well. Practically said: if you learn the
Fisherfaces for well-illuminated pictures only and you try to recognize faces in bad-illuminated
scenes, then method is likely to find the wrong components (just because those features may not be
predominant on bad illuminated images). This is somewhat logical, since the method had no chance to
learn the illumination.
The Fisherfaces allow a reconstruction of the projected image, just like the Eigenfaces did. But
since we only identified the features to distinguish between subjects, you can't expect a nice
reconstruction of the original image. For the Fisherfaces method we'll project the sample image onto
each of the Fisherfaces instead. So you'll have a nice visualization, which feature each of the
Fisherfaces describes:
@code{.cpp}
// Display or save the image reconstruction at some predefined steps:
for(int num_component = 0; num_component < min(16, W.cols); num_component++) {
// Slice the Fisherface from the model:
Mat ev = W.col(num_component);
Mat projection = LDA::subspaceProject(ev, mean, images[0].reshape(1,1));
Mat reconstruction = LDA::subspaceReconstruct(ev, mean, projection);
// Normalize the result:
reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));
// Display or save:
if(argc == 2) {
imshow(format("fisherface_reconstruction_%d", num_component), reconstruction);
} else {
imwrite(format("%s/fisherface_reconstruction_%d.png", output_folder.c_str(), num_component), reconstruction);
}
}
@endcode
The differences may be subtle for the human eyes, but you should be able to see some differences:
![image](img/fisherface_reconstruction_opencv.png)
Local Binary Patterns Histograms {#tutorial_face_lbph}
--------------------------------
Eigenfaces and Fisherfaces take a somewhat holistic approach to face recognition. You treat your
data as a vector somewhere in a high-dimensional image space. We all know high-dimensionality is
bad, so a lower-dimensional subspace is identified, where (probably) useful information is
preserved. The Eigenfaces approach maximizes the total scatter, which can lead to problems if the
variance is generated by an external source, because components with a maximum variance over all
classes aren't necessarily useful for classification (see
[<http://www.bytefish.de/wiki/pca_lda_with_gnu_octave>](http://www.bytefish.de/wiki/pca_lda_with_gnu_octave)).
So to preserve some discriminative information we applied a Linear Discriminant Analysis and
optimized as described in the Fisherfaces method. The Fisherfaces method worked great... at least
for the constrained scenario we've assumed in our model.
Now real life isn't perfect. You simply can't guarantee perfect light settings in your images or 10
different images of a person. So what if there's only one image for each person? Our covariance
estimates for the subspace *may* be horribly wrong, so will the recognition. Remember the Eigenfaces
method had a 96% recognition rate on the AT&T Facedatabase? How many images do we actually need to
get such useful estimates? Here are the Rank-1 recognition rates of the Eigenfaces and Fisherfaces
method on the AT&T Facedatabase, which is a fairly easy image database:
![image](img/at_database_small_sample_size.png)
So in order to get good recognition rates you'll need at least 8(+-1) images for each person and the
Fisherfaces method doesn't really help here. The above experiment is a 10-fold cross validated
result carried out with the facerec framework at:
[<https://github.com/bytefish/facerec>](https://github.com/bytefish/facerec). This is not a
publication, so I won't back these figures with a deep mathematical analysis. Please have a look
into @cite KM01 for a detailed analysis of both methods, when it comes to small training datasets.
So some research concentrated on extracting local features from images. The idea is to not look at
the whole image as a high-dimensional vector, but describe only local features of an object. The
features you extract this way will have a low-dimensionality implicitly. A fine idea! But you'll
soon observe the image representation we are given doesn't only suffer from illumination variations.
Think of things like scale, translation or rotation in images - your local description has to be at
least a bit robust against those things. Just like SIFT, the Local Binary Patterns methodology has
its roots in 2D texture analysis. The basic idea of Local Binary Patterns is to summarize the local
structure in an image by comparing each pixel with its neighborhood. Take a pixel as center and
threshold its neighbors against. If the intensity of the center pixel is greater-equal its neighbor,
then denote it with 1 and 0 if not. You'll end up with a binary number for each pixel, just like
11001111. So with 8 surrounding pixels you'll end up with 2\^8 possible combinations, called *Local
Binary Patterns* or sometimes referred to as *LBP codes*. The first LBP operator described in
literature actually used a fixed 3 x 3 neighborhood just like this:
![image](img/lbp/lbp.png)
### Algorithmic Description of LBPH method {#tutorial_face_lbph_algo}
A more formal description of the LBP operator can be given as:
\f[LBP(x_c, y_c) = \sum_{p=0}^{P-1} 2^p s(i_p - i_c)\f]
, with \f$(x_c, y_c)\f$ as central pixel with intensity \f$i_c\f$; and \f$i_n\f$ being the intensity of the the
neighbor pixel. \f$s\f$ is the sign function defined as:
\f[\begin{equation}
s(x) =
\begin{cases}
1 & \text{if \(x \geq 0\)}\\
0 & \text{else}
\end{cases}
\end{equation}\f]
This description enables you to capture very fine grained details in images. In fact the authors
were able to compete with state of the art results for texture classification. Soon after the
operator was published it was noted, that a fixed neighborhood fails to encode details differing in
scale. So the operator was extended to use a variable neighborhood in @cite AHP04 . The idea is to
align an abritrary number of neighbors on a circle with a variable radius, which enables to capture
the following neighborhoods:
![image](img/lbp/patterns.png)
For a given Point \f$(x_c,y_c)\f$ the position of the neighbor \f$(x_p,y_p), p \in P\f$ can be calculated
by:
\f[\begin{align*}
x_{p} & = & x_c + R \cos({\frac{2\pi p}{P}})\\
y_{p} & = & y_c - R \sin({\frac{2\pi p}{P}})
\end{align*}\f]
Where \f$R\f$ is the radius of the circle and \f$P\f$ is the number of sample points.
The operator is an extension to the original LBP codes, so it's sometimes called *Extended LBP*
(also referred to as *Circular LBP*) . If a points coordinate on the circle doesn't correspond to
image coordinates, the point get's interpolated. Computer science has a bunch of clever
interpolation schemes, the OpenCV implementation does a bilinear interpolation:
\f[\begin{align*}
f(x,y) \approx \begin{bmatrix}
1-x & x \end{bmatrix} \begin{bmatrix}
f(0,0) & f(0,1) \\
f(1,0) & f(1,1) \end{bmatrix} \begin{bmatrix}
1-y \\
y \end{bmatrix}.
\end{align*}\f]
By definition the LBP operator is robust against monotonic gray scale transformations. We can easily
verify this by looking at the LBP image of an artificially modified image (so you see what an LBP
image looks like!):
![image](img/lbp/lbp_yale.jpg)
So what's left to do is how to incorporate the spatial information in the face recognition model.
The representation proposed by Ahonen et. al @cite AHP04 is to divide the LBP image into \f$m\f$ local
regions and extract a histogram from each. The spatially enhanced feature vector is then obtained by
concatenating the local histograms (**not merging them**). These histograms are called *Local Binary
Patterns Histograms*.
### Local Binary Patterns Histograms in OpenCV {#tutorial_face_lbph_use}
The source code for this demo application is also available in the src folder coming with this
documentation:
@include face/samples/facerec_lbph.cpp
Conclusion {#tutorial_face_conclusion}
----------
You've learned how to use the new FaceRecognizer in real applications. After reading the document
you also know how the algorithms work, so now it's time for you to experiment with the available
algorithms. Use them, improve them and let the OpenCV community participate!
Credits {#tutorial_face_credits}
-------
This document wouldn't be possible without the kind permission to use the face images of the *AT&T
Database of Faces* and the *Yale Facedatabase A/B*.
### The Database of Faces {#tutorial_face_credits_db}
__Important: when using these images, please give credit to "AT&T Laboratories, Cambridge."__
The Database of Faces, formerly *The ORL Database of Faces*, contains a set of face images taken
between April 1992 and April 1994. The database was used in the context of a face recognition
project carried out in collaboration with the Speech, Vision and Robotics Group of the Cambridge
University Engineering Department.
There are ten different images of each of 40 distinct subjects. For some subjects, the images were
taken at different times, varying the lighting, facial expressions (open / closed eyes, smiling /
not smiling) and facial details (glasses / no glasses). All the images were taken against a dark
homogeneous background with the subjects in an upright, frontal position (with tolerance for some
side movement).
The files are in PGM format. The size of each image is 92x112 pixels, with 256 grey levels per
pixel. The images are organised in 40 directories (one for each subject), which have names of the
form sX, where X indicates the subject number (between 1 and 40). In each of these directories,
there are ten different images of that subject, which have names of the form Y.pgm, where Y is the
image number for that subject (between 1 and 10).
A copy of the database can be retrieved from:
[<http://www.cl.cam.ac.uk/research/dtg/attarchive/pub/data/att_faces.zip>](http://www.cl.cam.ac.uk/research/dtg/attarchive/pub/data/att_faces.zip).
### Yale Facedatabase A {#tutorial_face_credits_yalea}
*With the permission of the authors I am allowed to show a small number of images (say subject 1 and
all the variations) and all images such as Fisherfaces and Eigenfaces from either Yale Facedatabase
A or the Yale Facedatabase B.*
The Yale Face Database A (size 6.4MB) contains 165 grayscale images in GIF format of 15 individuals.
There are 11 images per subject, one per different facial expression or configuration: center-light,
w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink.
(Source:
[<http://cvc.yale.edu/projects/yalefaces/yalefaces.html>](http://cvc.yale.edu/projects/yalefaces/yalefaces.html))
### Yale Facedatabase B {#tutorial_face_credits_yaleb}
*With the permission of the authors I am allowed to show a small number of images (say subject 1 and
all the variations) and all images such as Fisherfaces and Eigenfaces from either Yale Facedatabase
A or the Yale Facedatabase B.*
The extended Yale Face Database B contains 16128 images of 28 human subjects under 9 poses and 64
illumination conditions. The data format of this database is the same as the Yale Face Database B.
Please refer to the homepage of the Yale Face Database B (or one copy of this page) for more
detailed information of the data format.
You are free to use the extended Yale Face Database B for research purposes. All publications which
use this database should acknowledge the use of "the Exteded Yale Face Database B" and reference
Athinodoros Georghiades, Peter Belhumeur, and David Kriegman's paper, "From Few to Many:
Illumination Cone Models for Face Recognition under Variable Lighting and Pose", PAMI, 2001,
[[bibtex]](http://vision.ucsd.edu/~leekc/ExtYaleDatabase/athosref.html).
The extended database as opposed to the original Yale Face Database B with 10 subjects was first
reported by Kuang-Chih Lee, Jeffrey Ho, and David Kriegman in "Acquiring Linear Subspaces for Face
Recognition under Variable Lighting, PAMI, May, 2005
[[pdf]](http://vision.ucsd.edu/~leekc/papers/9pltsIEEE.pdf)." All test image data used in the
experiments are manually aligned, cropped, and then re-sized to 168x192 images. If you publish your
experimental results with the cropped images, please reference the PAMI2005 paper as well. (Source:
[<http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html>](http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html))
Appendix {#face_appendix}
--------
### Creating the CSV File {#tutorial_face_appendix_csv}
You don't really want to create the CSV file by hand. I have prepared you a little Python script
`create_csv.py` (you find it at `src/create_csv.py` coming with this tutorial) that automatically
creates you a CSV file. If you have your images in hierarchie like this
(`/basepath/<subject>/<image.ext>`):
@code{.sh}
philipp@mango:~/facerec/data/at$ tree
.
|-- s1
| |-- 1.pgm
| |-- ...
| |-- 10.pgm
|-- s2
| |-- 1.pgm
| |-- ...
| |-- 10.pgm
...
|-- s40
| |-- 1.pgm
| |-- ...
| |-- 10.pgm
@endcode
Then simply call `create_csv.py at` , here 'at' being the basepath to the folder, just like this and you could save the
output:
@code{.sh}
philipp@mango:~/facerec/data$ python create_csv.py at
at/s13/2.pgm;0
at/s13/7.pgm;0
at/s13/6.pgm;0
at/s13/9.pgm;0
at/s13/5.pgm;0
at/s13/3.pgm;0
at/s13/4.pgm;0
at/s13/10.pgm;0
at/s13/8.pgm;0
at/s13/1.pgm;0
at/s17/2.pgm;1
at/s17/7.pgm;1
at/s17/6.pgm;1
at/s17/9.pgm;1
at/s17/5.pgm;1
at/s17/3.pgm;1
[...]
@endcode
Here is the script, if you can't find it:
@verbinclude face/samples/etc/create_csv.py
### Aligning Face Images {#tutorial_face_appendix_align}
An accurate alignment of your image data is especially important in tasks like emotion detection,
were you need as much detail as possible. Believe me... You don't want to do this by hand. So I've
prepared you a tiny Python script. The code is really easy to use. To scale, rotate and crop the
face image you just need to call *CropFace(image, eye_left, eye_right, offset_pct, dest_sz)*,
where:
- *eye_left* is the position of the left eye
- *eye_right* is the position of the right eye
- *offset_pct* is the percent of the image you want to keep next to the eyes (horizontal,
vertical direction)
- *dest_sz* is the size of the output image
If you are using the same *offset_pct* and *dest_sz* for your images, they are all aligned at the
eyes.
@verbinclude face/samples/etc/crop_face.py
Imagine we are given [this photo of Arnold
Schwarzenegger](http://en.wikipedia.org/wiki/File:Arnold_Schwarzenegger_edit%28ws%29.jpg), which is
under a Public Domain license. The (x,y)-position of the eyes is approximately *(252,364)* for the
left and *(420,366)* for the right eye. Now you only need to define the horizontal offset, vertical
offset and the size your scaled, rotated & cropped face should have.
Here are some examples:
Configuration | Cropped, Scaled, Rotated Face
--------------------------------|------------------------------------------------------------------
0.1 (10%), 0.1 (10%), (200,200) | ![](tutorials/gender_classification/arnie_10_10_200_200.jpg)
0.2 (20%), 0.2 (20%), (200,200) | ![](tutorials/gender_classification/arnie_20_20_200_200.jpg)
0.3 (30%), 0.3 (30%), (200,200) | ![](tutorials/gender_classification/arnie_30_30_200_200.jpg)
0.2 (20%), 0.2 (20%), (70,70) | ![](tutorials/gender_classification/arnie_20_20_70_70.jpg)
### CSV for the AT&T Facedatabase {#tutorial_face_appendix_attcsv}
@verbinclude face/samples/etc/at.txt
modules/face/tutorials/face_recognition/facerec_video/facerec_video.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 290 KiB

modules/face/tutorials/face_recognition/gender_classification/arnie_10_10_200_200.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.4 KiB

modules/face/tutorials/face_recognition/gender_classification/arnie_20_20_200_200.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.2 KiB

modules/face/tutorials/face_recognition/gender_classification/arnie_20_20_70_70.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 KiB

modules/face/tutorials/face_recognition/gender_classification/arnie_30_30_200_200.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.1 KiB

modules/face/tutorials/face_recognition/gender_classification/clooney_set.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 92 KiB

modules/face/tutorials/face_recognition/gender_classification/fisherface_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

modules/face/tutorials/face_recognition/gender_classification/fisherface_reconstruction_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

modules/face/tutorials/face_recognition/gender_classification/mean.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.9 KiB

modules/face/tutorials/face_recognition/img/at_database_small_sample_size.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

modules/face/tutorials/face_recognition/img/eigenface_reconstruction_opencv.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 171 KiB

modules/face/tutorials/face_recognition/img/eigenfaces_opencv.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 108 KiB

modules/face/tutorials/face_recognition/img/fisherface_reconstruction_opencv.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 111 KiB

modules/face/tutorials/face_recognition/img/fisherfaces_opencv.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 281 KiB

modules/face/tutorials/face_recognition/img/lbp/lbp.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

modules/face/tutorials/face_recognition/img/lbp/lbp_yale.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 83 KiB

modules/face/tutorials/face_recognition/img/lbp/patterns.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

Some files were not shown because too many files have changed in this diff Show More