vendor: OpenCV 5.0.0 snapshot at 755e50675d97db9b7d449d8bd6b09888646f6c6e
This commit is contained in:
Gitea Mirror Bot
@@ -0,0 +1,2 @@
|
||||
set(the_description "Background Segmentation Algorithms")
|
||||
ocv_define_module(bgsegm opencv_core opencv_imgproc opencv_video opencv_geometry WRAP python java objc)
|
||||
@@ -0,0 +1,10 @@
|
||||
Improved Background-Foreground Segmentation Methods
|
||||
===================================================
|
||||
|
||||
This algorithm combines statistical background image estimation and per-pixel Bayesian segmentation. It[1] was introduced by Andrew B. Godbehere, Akihiro Matsukawa, Ken Goldberg in 2012. As per the paper, the system ran a successful interactive audio art installation called "Are We There Yet?" from March 31 - July 31 2011 at the Contemporary Jewish Museum in San Francisco, California.
|
||||
|
||||
It uses first few (120 by default) frames for background modelling. It employs probabilistic foreground segmentation algorithm that identifies possible foreground objects using Bayesian inference. The estimates are adaptive; newer observations are more heavily weighted than old observations to accommodate variable illumination. Several morphological filtering operations like closing and opening are done to remove unwanted noise. You will get a black window during first few frames.
|
||||
|
||||
References
|
||||
----------
|
||||
[1]: A.B. Godbehere, A. Matsukawa, K. Goldberg. Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation. American Control Conference. (2012), pp. 4305–4312
|
||||
@@ -0,0 +1,27 @@
|
||||
@incollection{KB2001,
|
||||
title={An improved adaptive background mixture model for real-time tracking with shadow detection},
|
||||
author={KaewTraKulPong, Pakorn and Bowden, Richard},
|
||||
booktitle={Video-Based Surveillance Systems},
|
||||
pages={135--144},
|
||||
year={2002},
|
||||
publisher={Springer}
|
||||
}
|
||||
|
||||
@inproceedings{Gold2012,
|
||||
title={Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation},
|
||||
author={Godbehere, Andrew B and Matsukawa, Akihiro and Goldberg, Ken},
|
||||
booktitle={American Control Conference (ACC), 2012},
|
||||
pages={4305--4312},
|
||||
year={2012},
|
||||
organization={IEEE}
|
||||
}
|
||||
|
||||
@inproceedings{LGuo2016,
|
||||
author={L. Guo and D. Xu and Z. Qiang},
|
||||
booktitle={2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
|
||||
title={Background Subtraction Using Local SVD Binary Pattern},
|
||||
year={2016},
|
||||
pages={1159-1167},
|
||||
doi={10.1109/CVPRW.2016.148},
|
||||
month={June}
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
/*
|
||||
By downloading, copying, installing or using the software you agree to this
|
||||
license. If you do not agree to this license, do not download, install,
|
||||
copy or use the software.
|
||||
|
||||
|
||||
License Agreement
|
||||
For Open Source Computer Vision Library
|
||||
(3-clause BSD License)
|
||||
|
||||
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
|
||||
Third party copyrights are property of their respective owners.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification,
|
||||
are permitted provided that the following conditions are met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright notice,
|
||||
this list of conditions and the following disclaimer.
|
||||
|
||||
* Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
* Neither the names of the copyright holders nor the names of the contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
|
||||
This software is provided by the copyright holders and contributors "as is" and
|
||||
any express or implied warranties, including, but not limited to, the implied
|
||||
warranties of merchantability and fitness for a particular purpose are
|
||||
disclaimed. In no event shall copyright holders or contributors be liable for
|
||||
any direct, indirect, incidental, special, exemplary, or consequential damages
|
||||
(including, but not limited to, procurement of substitute goods or services;
|
||||
loss of use, data, or profits; or business interruption) however caused
|
||||
and on any theory of liability, whether in contract, strict liability,
|
||||
or tort (including negligence or otherwise) arising in any way out of
|
||||
the use of this software, even if advised of the possibility of such damage.
|
||||
*/
|
||||
|
||||
#ifndef __OPENCV_BGSEGM_HPP__
|
||||
#define __OPENCV_BGSEGM_HPP__
|
||||
|
||||
#include "opencv2/video.hpp"
|
||||
|
||||
#ifdef __cplusplus
|
||||
|
||||
/** @defgroup bgsegm Improved Background-Foreground Segmentation Methods
|
||||
*/
|
||||
|
||||
namespace cv
|
||||
{
|
||||
namespace bgsegm
|
||||
{
|
||||
|
||||
//! @addtogroup bgsegm
|
||||
//! @{
|
||||
|
||||
/** @brief Gaussian Mixture-based Background/Foreground Segmentation Algorithm.
|
||||
|
||||
The class implements the algorithm described in @cite KB2001 .
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorMOG : public BackgroundSubtractor
|
||||
{
|
||||
public:
|
||||
// BackgroundSubtractor interface
|
||||
/** @brief Computes a foreground mask.
|
||||
|
||||
@param image Next video frame of type CV_8UC(n),CV_8SC(n),CV_16UC(n),CV_16SC(n),CV_32SC(n),CV_32FC(n),CV_64FC(n), where n is 1,2,3,4.
|
||||
@param fgmask The output foreground mask as an 8-bit binary image.
|
||||
@param learningRate The value between 0 and 1 that indicates how fast the background model is
|
||||
learnt. Negative parameter value makes the algorithm to use some automatically chosen learning
|
||||
rate. 0 means that the background model is not updated at all, 1 means that the background model
|
||||
is completely reinitialized from the last frame.
|
||||
*/
|
||||
|
||||
CV_WRAP virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
/** @brief Computes a foreground mask and skips known foreground in evaluation.
|
||||
|
||||
@param image Next video frame of type CV_8UC(n),CV_8SC(n),CV_16UC(n),CV_16SC(n),CV_32SC(n),CV_32FC(n),CV_64FC(n), where n is 1,2,3,4.
|
||||
@param fgmask The output foreground mask as an 8-bit binary image.
|
||||
@param knownForegroundMask The mask for inputting already known foreground, allows model to ignore learning known pixels.
|
||||
@param learningRate The value between 0 and 1 that indicates how fast the background model is
|
||||
learnt. Negative parameter value makes the algorithm to use some automatically chosen learning
|
||||
rate. 0 means that the background model is not updated at all, 1 means that the background model
|
||||
is completely reinitialized from the last frame.
|
||||
*/
|
||||
|
||||
CV_WRAP virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
CV_WRAP virtual int getHistory() const = 0;
|
||||
CV_WRAP virtual void setHistory(int nframes) = 0;
|
||||
|
||||
CV_WRAP virtual int getNMixtures() const = 0;
|
||||
CV_WRAP virtual void setNMixtures(int nmix) = 0;
|
||||
|
||||
CV_WRAP virtual double getBackgroundRatio() const = 0;
|
||||
CV_WRAP virtual void setBackgroundRatio(double backgroundRatio) = 0;
|
||||
|
||||
CV_WRAP virtual double getNoiseSigma() const = 0;
|
||||
CV_WRAP virtual void setNoiseSigma(double noiseSigma) = 0;
|
||||
};
|
||||
|
||||
/** @brief Creates mixture-of-gaussian background subtractor
|
||||
|
||||
@param history Length of the history.
|
||||
@param nmixtures Number of Gaussian mixtures.
|
||||
@param backgroundRatio Background ratio.
|
||||
@param noiseSigma Noise strength (standard deviation of the brightness or each color channel). 0
|
||||
means some automatic value.
|
||||
*/
|
||||
CV_EXPORTS_W Ptr<BackgroundSubtractorMOG>
|
||||
createBackgroundSubtractorMOG(int history=200, int nmixtures=5,
|
||||
double backgroundRatio=0.7, double noiseSigma=0);
|
||||
|
||||
|
||||
/** @brief Background Subtractor module based on the algorithm given in @cite Gold2012 .
|
||||
|
||||
Takes a series of images and returns a sequence of mask (8UC1)
|
||||
images of the same size, where 255 indicates Foreground and 0 represents Background.
|
||||
This class implements an algorithm described in "Visual Tracking of Human Visitors under
|
||||
Variable-Lighting Conditions for a Responsive Audio Art Installation," A. Godbehere,
|
||||
A. Matsukawa, K. Goldberg, American Control Conference, Montreal, June 2012.
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorGMG : public BackgroundSubtractor
|
||||
{
|
||||
public:
|
||||
// BackgroundSubtractor interface
|
||||
/** @brief Computes a foreground mask.
|
||||
|
||||
@param image Next video frame of type CV_8UC(n),CV_8SC(n),CV_16UC(n),CV_16SC(n),CV_32SC(n),CV_32FC(n),CV_64FC(n), where n is 1,2,3,4.
|
||||
@param fgmask The output foreground mask as an 8-bit binary image.
|
||||
@param learningRate The value between 0 and 1 that indicates how fast the background model is
|
||||
learnt. Negative parameter value makes the algorithm to use some automatically chosen learning
|
||||
rate. 0 means that the background model is not updated at all, 1 means that the background model
|
||||
is completely reinitialized from the last frame.
|
||||
*/
|
||||
CV_WRAP virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
/** @brief Computes a foreground mask with known foreground mask input.
|
||||
|
||||
@param image Next video frame.
|
||||
@param fgmask The output foreground mask as an 8-bit binary image.
|
||||
@param knownForegroundMask The mask for inputting already known foreground.
|
||||
@param learningRate The value between 0 and 1 that indicates how fast the background model is
|
||||
learnt. Negative parameter value makes the algorithm to use some automatically chosen learning
|
||||
rate. 0 means that the background model is not updated at all, 1 means that the background model
|
||||
is completely reinitialized from the last frame.
|
||||
|
||||
@note This method has a default virtual implementation that throws a "not implemented" error.
|
||||
Foreground masking may not be supported by all background subtractors.
|
||||
*/
|
||||
CV_WRAP virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
CV_WRAP virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE = 0;
|
||||
|
||||
/** @brief Returns total number of distinct colors to maintain in histogram.
|
||||
*/
|
||||
CV_WRAP virtual int getMaxFeatures() const = 0;
|
||||
/** @brief Sets total number of distinct colors to maintain in histogram.
|
||||
*/
|
||||
CV_WRAP virtual void setMaxFeatures(int maxFeatures) = 0;
|
||||
|
||||
/** @brief Returns the learning rate of the algorithm.
|
||||
|
||||
It lies between 0.0 and 1.0. It determines how quickly features are "forgotten" from
|
||||
histograms.
|
||||
*/
|
||||
CV_WRAP virtual double getDefaultLearningRate() const = 0;
|
||||
/** @brief Sets the learning rate of the algorithm.
|
||||
*/
|
||||
CV_WRAP virtual void setDefaultLearningRate(double lr) = 0;
|
||||
|
||||
/** @brief Returns the number of frames used to initialize background model.
|
||||
*/
|
||||
CV_WRAP virtual int getNumFrames() const = 0;
|
||||
/** @brief Sets the number of frames used to initialize background model.
|
||||
*/
|
||||
CV_WRAP virtual void setNumFrames(int nframes) = 0;
|
||||
|
||||
/** @brief Returns the parameter used for quantization of color-space.
|
||||
|
||||
It is the number of discrete levels in each channel to be used in histograms.
|
||||
*/
|
||||
CV_WRAP virtual int getQuantizationLevels() const = 0;
|
||||
/** @brief Sets the parameter used for quantization of color-space
|
||||
*/
|
||||
CV_WRAP virtual void setQuantizationLevels(int nlevels) = 0;
|
||||
|
||||
/** @brief Returns the prior probability that each individual pixel is a background pixel.
|
||||
*/
|
||||
CV_WRAP virtual double getBackgroundPrior() const = 0;
|
||||
/** @brief Sets the prior probability that each individual pixel is a background pixel.
|
||||
*/
|
||||
CV_WRAP virtual void setBackgroundPrior(double bgprior) = 0;
|
||||
|
||||
/** @brief Returns the kernel radius used for morphological operations
|
||||
*/
|
||||
CV_WRAP virtual int getSmoothingRadius() const = 0;
|
||||
/** @brief Sets the kernel radius used for morphological operations
|
||||
*/
|
||||
CV_WRAP virtual void setSmoothingRadius(int radius) = 0;
|
||||
|
||||
/** @brief Returns the value of decision threshold.
|
||||
|
||||
Decision value is the value above which pixel is determined to be FG.
|
||||
*/
|
||||
CV_WRAP virtual double getDecisionThreshold() const = 0;
|
||||
/** @brief Sets the value of decision threshold.
|
||||
*/
|
||||
CV_WRAP virtual void setDecisionThreshold(double thresh) = 0;
|
||||
|
||||
/** @brief Returns the status of background model update
|
||||
*/
|
||||
CV_WRAP virtual bool getUpdateBackgroundModel() const = 0;
|
||||
/** @brief Sets the status of background model update
|
||||
*/
|
||||
CV_WRAP virtual void setUpdateBackgroundModel(bool update) = 0;
|
||||
|
||||
/** @brief Returns the minimum value taken on by pixels in image sequence. Usually 0.
|
||||
*/
|
||||
CV_WRAP virtual double getMinVal() const = 0;
|
||||
/** @brief Sets the minimum value taken on by pixels in image sequence.
|
||||
*/
|
||||
CV_WRAP virtual void setMinVal(double val) = 0;
|
||||
|
||||
/** @brief Returns the maximum value taken on by pixels in image sequence. e.g. 1.0 or 255.
|
||||
*/
|
||||
CV_WRAP virtual double getMaxVal() const = 0;
|
||||
/** @brief Sets the maximum value taken on by pixels in image sequence.
|
||||
*/
|
||||
CV_WRAP virtual void setMaxVal(double val) = 0;
|
||||
};
|
||||
|
||||
/** @brief Creates a GMG Background Subtractor
|
||||
|
||||
@param initializationFrames number of frames used to initialize the background models.
|
||||
@param decisionThreshold Threshold value, above which it is marked foreground, else background.
|
||||
*/
|
||||
CV_EXPORTS_W Ptr<BackgroundSubtractorGMG> createBackgroundSubtractorGMG(int initializationFrames=120,
|
||||
double decisionThreshold=0.8);
|
||||
|
||||
/** @brief Background subtraction based on counting.
|
||||
|
||||
About as fast as MOG2 on a high end system.
|
||||
More than twice faster than MOG2 on cheap hardware (benchmarked on Raspberry Pi3).
|
||||
|
||||
%Algorithm by Sagi Zeevi ( https://github.com/sagi-z/BackgroundSubtractorCNT )
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorCNT : public BackgroundSubtractor
|
||||
{
|
||||
public:
|
||||
// BackgroundSubtractor interface
|
||||
CV_WRAP virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
/** @brief Computes a foreground mask with known foreground mask input.
|
||||
|
||||
@param image Next video frame.
|
||||
@param knownForegroundMask The mask for inputting already known foreground.
|
||||
@param fgmask The output foreground mask as an 8-bit binary image.
|
||||
@param learningRate The value between 0 and 1 that indicates how fast the background model is
|
||||
learnt. Negative parameter value makes the algorithm to use some automatically chosen learning
|
||||
rate. 0 means that the background model is not updated at all, 1 means that the background model
|
||||
is completely reinitialized from the last frame.
|
||||
|
||||
@note This method has a default virtual implementation that throws a "not impemented" error.
|
||||
Foreground masking may not be supported by all background subtractors.
|
||||
*/
|
||||
CV_WRAP virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
CV_WRAP virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE = 0;
|
||||
|
||||
/** @brief Returns number of frames with same pixel color to consider stable.
|
||||
*/
|
||||
CV_WRAP virtual int getMinPixelStability() const = 0;
|
||||
/** @brief Sets the number of frames with same pixel color to consider stable.
|
||||
*/
|
||||
CV_WRAP virtual void setMinPixelStability(int value) = 0;
|
||||
|
||||
/** @brief Returns maximum allowed credit for a pixel in history.
|
||||
*/
|
||||
CV_WRAP virtual int getMaxPixelStability() const = 0;
|
||||
/** @brief Sets the maximum allowed credit for a pixel in history.
|
||||
*/
|
||||
CV_WRAP virtual void setMaxPixelStability(int value) = 0;
|
||||
|
||||
/** @brief Returns if we're giving a pixel credit for being stable for a long time.
|
||||
*/
|
||||
CV_WRAP virtual bool getUseHistory() const = 0;
|
||||
/** @brief Sets if we're giving a pixel credit for being stable for a long time.
|
||||
*/
|
||||
CV_WRAP virtual void setUseHistory(bool value) = 0;
|
||||
|
||||
/** @brief Returns if we're parallelizing the algorithm.
|
||||
*/
|
||||
CV_WRAP virtual bool getIsParallel() const = 0;
|
||||
/** @brief Sets if we're parallelizing the algorithm.
|
||||
*/
|
||||
CV_WRAP virtual void setIsParallel(bool value) = 0;
|
||||
};
|
||||
|
||||
/** @brief Creates a CNT Background Subtractor
|
||||
|
||||
@param minPixelStability number of frames with same pixel color to consider stable
|
||||
@param useHistory determines if we're giving a pixel credit for being stable for a long time
|
||||
@param maxPixelStability maximum allowed credit for a pixel in history
|
||||
@param isParallel determines if we're parallelizing the algorithm
|
||||
*/
|
||||
|
||||
CV_EXPORTS_W Ptr<BackgroundSubtractorCNT>
|
||||
createBackgroundSubtractorCNT(int minPixelStability = 15,
|
||||
bool useHistory = true,
|
||||
int maxPixelStability = 15*60,
|
||||
bool isParallel = true);
|
||||
|
||||
enum LSBPCameraMotionCompensation {
|
||||
LSBP_CAMERA_MOTION_COMPENSATION_NONE = 0,
|
||||
LSBP_CAMERA_MOTION_COMPENSATION_LK
|
||||
};
|
||||
|
||||
/** @brief Implementation of the different yet better algorithm which is called GSOC, as it was implemented during GSOC and was not originated from any paper.
|
||||
|
||||
This algorithm demonstrates better performance on CDNET 2014 dataset compared to other algorithms in OpenCV.
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorGSOC : public BackgroundSubtractor
|
||||
{
|
||||
public:
|
||||
// BackgroundSubtractor interface
|
||||
CV_WRAP virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
CV_WRAP virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
CV_WRAP virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE = 0;
|
||||
};
|
||||
|
||||
/** @brief Background Subtraction using Local SVD Binary Pattern. More details about the algorithm can be found at @cite LGuo2016
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorLSBP : public BackgroundSubtractor
|
||||
{
|
||||
public:
|
||||
// BackgroundSubtractor interface
|
||||
CV_WRAP virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
CV_WRAP virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate=-1) CV_OVERRIDE = 0;
|
||||
|
||||
CV_WRAP virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE = 0;
|
||||
};
|
||||
|
||||
/** @brief This is for calculation of the LSBP descriptors.
|
||||
*/
|
||||
class CV_EXPORTS_W BackgroundSubtractorLSBPDesc
|
||||
{
|
||||
public:
|
||||
static void calcLocalSVDValues(OutputArray localSVDValues, const Mat& frame);
|
||||
|
||||
static void computeFromLocalSVDValues(OutputArray desc, const Mat& localSVDValues, const Point2i* LSBPSamplePoints);
|
||||
|
||||
static void compute(OutputArray desc, const Mat& frame, const Point2i* LSBPSamplePoints);
|
||||
};
|
||||
|
||||
/** @brief Creates an instance of BackgroundSubtractorGSOC algorithm.
|
||||
|
||||
Implementation of the different yet better algorithm which is called GSOC, as it was implemented during GSOC and was not originated from any paper.
|
||||
|
||||
@param mc Whether to use camera motion compensation.
|
||||
@param nSamples Number of samples to maintain at each point of the frame.
|
||||
@param replaceRate Probability of replacing the old sample - how fast the model will update itself.
|
||||
@param propagationRate Probability of propagating to neighbors.
|
||||
@param hitsThreshold How many positives the sample must get before it will be considered as a possible replacement.
|
||||
@param alpha Scale coefficient for threshold.
|
||||
@param beta Bias coefficient for threshold.
|
||||
@param blinkingSupressionDecay Blinking supression decay factor.
|
||||
@param blinkingSupressionMultiplier Blinking supression multiplier.
|
||||
@param noiseRemovalThresholdFacBG Strength of the noise removal for background points.
|
||||
@param noiseRemovalThresholdFacFG Strength of the noise removal for foreground points.
|
||||
*/
|
||||
CV_EXPORTS_W Ptr<BackgroundSubtractorGSOC> createBackgroundSubtractorGSOC(int mc = LSBP_CAMERA_MOTION_COMPENSATION_NONE, int nSamples = 20, float replaceRate = 0.003f, float propagationRate = 0.01f, int hitsThreshold = 32, float alpha = 0.01f, float beta = 0.0022f, float blinkingSupressionDecay = 0.1f, float blinkingSupressionMultiplier = 0.1f, float noiseRemovalThresholdFacBG = 0.0004f, float noiseRemovalThresholdFacFG = 0.0008f);
|
||||
|
||||
/** @brief Creates an instance of BackgroundSubtractorLSBP algorithm.
|
||||
|
||||
Background Subtraction using Local SVD Binary Pattern. More details about the algorithm can be found at @cite LGuo2016
|
||||
|
||||
@param mc Whether to use camera motion compensation.
|
||||
@param nSamples Number of samples to maintain at each point of the frame.
|
||||
@param LSBPRadius LSBP descriptor radius.
|
||||
@param Tlower Lower bound for T-values. See @cite LGuo2016 for details.
|
||||
@param Tupper Upper bound for T-values. See @cite LGuo2016 for details.
|
||||
@param Tinc Increase step for T-values. See @cite LGuo2016 for details.
|
||||
@param Tdec Decrease step for T-values. See @cite LGuo2016 for details.
|
||||
@param Rscale Scale coefficient for threshold values.
|
||||
@param Rincdec Increase/Decrease step for threshold values.
|
||||
@param noiseRemovalThresholdFacBG Strength of the noise removal for background points.
|
||||
@param noiseRemovalThresholdFacFG Strength of the noise removal for foreground points.
|
||||
@param LSBPthreshold Threshold for LSBP binary string.
|
||||
@param minCount Minimal number of matches for sample to be considered as foreground.
|
||||
*/
|
||||
CV_EXPORTS_W Ptr<BackgroundSubtractorLSBP> createBackgroundSubtractorLSBP(int mc = LSBP_CAMERA_MOTION_COMPENSATION_NONE, int nSamples = 20, int LSBPRadius = 16, float Tlower = 2.0f, float Tupper = 32.0f, float Tinc = 1.0f, float Tdec = 0.05f, float Rscale = 10.0f, float Rincdec = 0.005f, float noiseRemovalThresholdFacBG = 0.0004f, float noiseRemovalThresholdFacFG = 0.0008f, int LSBPthreshold = 8, int minCount = 2);
|
||||
|
||||
/** @brief Synthetic frame sequence generator for testing background subtraction algorithms.
|
||||
|
||||
It will generate the moving object on top of the background.
|
||||
It will apply some distortion to the background to make the test more complex.
|
||||
*/
|
||||
class CV_EXPORTS_W SyntheticSequenceGenerator : public Algorithm
|
||||
{
|
||||
private:
|
||||
const double amplitude;
|
||||
const double wavelength;
|
||||
const double wavespeed;
|
||||
const double objspeed;
|
||||
unsigned timeStep;
|
||||
Point2d pos;
|
||||
Point2d dir;
|
||||
Mat background;
|
||||
Mat object;
|
||||
RNG rng;
|
||||
|
||||
public:
|
||||
/** @brief Creates an instance of SyntheticSequenceGenerator.
|
||||
|
||||
@param background Background image for object.
|
||||
@param object Object image which will move slowly over the background.
|
||||
@param amplitude Amplitude of wave distortion applied to background.
|
||||
@param wavelength Length of waves in distortion applied to background.
|
||||
@param wavespeed How fast waves will move.
|
||||
@param objspeed How fast object will fly over background.
|
||||
*/
|
||||
CV_WRAP SyntheticSequenceGenerator(InputArray background, InputArray object, double amplitude, double wavelength, double wavespeed, double objspeed);
|
||||
|
||||
/** @brief Obtain the next frame in the sequence.
|
||||
|
||||
@param frame Output frame.
|
||||
@param gtMask Output ground-truth (reference) segmentation mask object/background.
|
||||
*/
|
||||
CV_WRAP void getNextFrame(OutputArray frame, OutputArray gtMask);
|
||||
};
|
||||
|
||||
/** @brief Creates an instance of SyntheticSequenceGenerator.
|
||||
|
||||
@param background Background image for object.
|
||||
@param object Object image which will move slowly over the background.
|
||||
@param amplitude Amplitude of wave distortion applied to background.
|
||||
@param wavelength Length of waves in distortion applied to background.
|
||||
@param wavespeed How fast waves will move.
|
||||
@param objspeed How fast object will fly over background.
|
||||
*/
|
||||
CV_EXPORTS_W Ptr<SyntheticSequenceGenerator> createSyntheticSequenceGenerator(InputArray background, InputArray object, double amplitude = 2.0, double wavelength = 20.0, double wavespeed = 0.2, double objspeed = 6.0);
|
||||
|
||||
//! @}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
#endif
|
||||
#endif
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"AdditionalImports" : {
|
||||
"*" : [ "\"bgsegm.hpp\"" ]
|
||||
},
|
||||
"func_arg_fix" : {
|
||||
"Bgsegm" : {
|
||||
"createBackgroundSubtractorGSOC" : { "mc" : {"ctype" : "LSBPCameraMotionCompensation"} },
|
||||
"createBackgroundSubtractorLSBP" : { "mc" : {"ctype" : "LSBPCameraMotionCompensation"} }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
#include "opencv2/bgsegm.hpp"
|
||||
#include "opencv2/videoio.hpp"
|
||||
#include "opencv2/highgui.hpp"
|
||||
#include <opencv2/core/utility.hpp>
|
||||
#include <iostream>
|
||||
|
||||
using namespace cv;
|
||||
using namespace cv::bgsegm;
|
||||
|
||||
const String about =
|
||||
"\nA program demonstrating the use and capabilities of different background subtraction algorithms\n"
|
||||
"Using OpenCV version " + String(CV_VERSION) +
|
||||
"\n\nPress 'c' to change the algorithm"
|
||||
"\nPress 'm' to toggle showing only foreground mask or ghost effect"
|
||||
"\nPress 'n' to change number of threads"
|
||||
"\nPress SPACE to toggle wait delay of imshow"
|
||||
"\nPress 'q' or ESC to exit\n";
|
||||
|
||||
const String algos[7] = { "GMG", "CNT", "KNN", "MOG", "MOG2", "GSOC", "LSBP" };
|
||||
|
||||
static Ptr<BackgroundSubtractor> createBGSubtractorByName(const String& algoName)
|
||||
{
|
||||
Ptr<BackgroundSubtractor> algo;
|
||||
if(algoName == String("GMG"))
|
||||
algo = createBackgroundSubtractorGMG(20, 0.7);
|
||||
else if(algoName == String("CNT"))
|
||||
algo = createBackgroundSubtractorCNT();
|
||||
else if(algoName == String("KNN"))
|
||||
algo = createBackgroundSubtractorKNN();
|
||||
else if(algoName == String("MOG"))
|
||||
algo = createBackgroundSubtractorMOG();
|
||||
else if(algoName == String("MOG2"))
|
||||
algo = createBackgroundSubtractorMOG2();
|
||||
else if(algoName == String("GSOC"))
|
||||
algo = createBackgroundSubtractorGSOC();
|
||||
else if(algoName == String("LSBP"))
|
||||
algo = createBackgroundSubtractorLSBP();
|
||||
|
||||
return algo;
|
||||
}
|
||||
|
||||
int main(int argc, char** argv)
|
||||
{
|
||||
CommandLineParser parser(argc, argv, "{@video | vtest.avi | path to a video file}");
|
||||
parser.about(about);
|
||||
parser.printMessage();
|
||||
|
||||
String videoPath = samples::findFile(parser.get<String>(0),false);
|
||||
|
||||
Ptr<BackgroundSubtractor> bgfs = createBGSubtractorByName(algos[0]);
|
||||
|
||||
VideoCapture cap;
|
||||
cap.open(videoPath);
|
||||
|
||||
if (!cap.isOpened())
|
||||
{
|
||||
std::cerr << "Cannot read video. Try moving video file to sample directory." << std::endl;
|
||||
return -1;
|
||||
}
|
||||
|
||||
Mat frame, fgmask, segm;
|
||||
|
||||
int delay = 30;
|
||||
int algo_index = 0;
|
||||
int nthreads = getNumberOfCPUs();
|
||||
bool show_fgmask = false;
|
||||
|
||||
for (;;)
|
||||
{
|
||||
cap >> frame;
|
||||
|
||||
if (frame.empty())
|
||||
{
|
||||
cap.set(CAP_PROP_POS_FRAMES, 0);
|
||||
cap >> frame;
|
||||
}
|
||||
|
||||
bgfs->apply(frame, fgmask);
|
||||
|
||||
if (show_fgmask)
|
||||
segm = fgmask;
|
||||
else
|
||||
{
|
||||
frame.convertTo(segm, CV_8U, 0.5);
|
||||
add(frame, Scalar(100, 100, 0), segm, fgmask);
|
||||
}
|
||||
|
||||
putText(segm, algos[algo_index], Point(10, 30), FONT_HERSHEY_PLAIN, 2.0, Scalar(255, 0, 255), 2, LINE_AA);
|
||||
putText(segm, format("%d threads", nthreads), Point(10, 60), FONT_HERSHEY_PLAIN, 2.0, Scalar(255, 0, 255), 2, LINE_AA);
|
||||
|
||||
imshow("FG Segmentation", segm);
|
||||
|
||||
int c = waitKey(delay);
|
||||
|
||||
if (c == ' ')
|
||||
delay = delay == 30 ? 1 : 30;
|
||||
|
||||
if (c == 'c' || c == 'C')
|
||||
{
|
||||
algo_index++;
|
||||
if ( algo_index > 6 )
|
||||
algo_index = 0;
|
||||
|
||||
bgfs = createBGSubtractorByName(algos[algo_index]);
|
||||
}
|
||||
|
||||
if (c == 'n' || c == 'N')
|
||||
{
|
||||
nthreads++;
|
||||
if ( nthreads > 8 )
|
||||
nthreads = 1;
|
||||
|
||||
setNumThreads(nthreads);
|
||||
}
|
||||
|
||||
if (c == 'm' || c == 'M')
|
||||
show_fgmask = !show_fgmask;
|
||||
|
||||
if (c == 'q' || c == 'Q' || c == 27)
|
||||
break;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
import argparse
|
||||
import cv2 as cv
|
||||
import glob
|
||||
import numpy as np
|
||||
import os
|
||||
import time
|
||||
|
||||
|
||||
# This tool is intended for evaluation of different background subtraction algorithms presented in OpenCV.
|
||||
# Several presets with different settings are available. You can see them below.
|
||||
# This tool measures quality metrics as well as speed.
|
||||
|
||||
|
||||
ALGORITHMS_TO_EVALUATE = [
|
||||
(cv.bgsegm.createBackgroundSubtractorMOG, 'MOG', {}),
|
||||
(cv.bgsegm.createBackgroundSubtractorGMG, 'GMG', {}),
|
||||
(cv.bgsegm.createBackgroundSubtractorCNT, 'CNT', {}),
|
||||
(cv.bgsegm.createBackgroundSubtractorLSBP, 'LSBP-vanilla', {'nSamples': 20, 'LSBPRadius': 4, 'Tlower': 2.0, 'Tupper': 200.0, 'Tinc': 1.0, 'Tdec': 0.05, 'Rscale': 5.0, 'Rincdec': 0.05, 'LSBPthreshold': 8}),
|
||||
(cv.bgsegm.createBackgroundSubtractorLSBP, 'LSBP-speed', {'nSamples': 10, 'LSBPRadius': 16, 'Tlower': 2.0, 'Tupper': 32.0, 'Tinc': 1.0, 'Tdec': 0.05, 'Rscale': 10.0, 'Rincdec': 0.005, 'LSBPthreshold': 8}),
|
||||
(cv.bgsegm.createBackgroundSubtractorLSBP, 'LSBP-quality', {'nSamples': 20, 'LSBPRadius': 16, 'Tlower': 2.0, 'Tupper': 32.0, 'Tinc': 1.0, 'Tdec': 0.05, 'Rscale': 10.0, 'Rincdec': 0.005, 'LSBPthreshold': 8}),
|
||||
(cv.bgsegm.createBackgroundSubtractorLSBP, 'LSBP-camera-motion-compensation', {'mc': 1}),
|
||||
(cv.bgsegm.createBackgroundSubtractorGSOC, 'GSOC', {}),
|
||||
(cv.bgsegm.createBackgroundSubtractorGSOC, 'GSOC-camera-motion-compensation', {'mc': 1})
|
||||
]
|
||||
|
||||
|
||||
def contains_relevant_files(root):
|
||||
return os.path.isdir(os.path.join(root, 'groundtruth')) and os.path.isdir(os.path.join(root, 'input'))
|
||||
|
||||
|
||||
def find_relevant_dirs(root):
|
||||
relevant_dirs = []
|
||||
for d in sorted(os.listdir(root)):
|
||||
d = os.path.join(root, d)
|
||||
if os.path.isdir(d):
|
||||
if contains_relevant_files(d):
|
||||
relevant_dirs += [d]
|
||||
else:
|
||||
relevant_dirs += find_relevant_dirs(d)
|
||||
return relevant_dirs
|
||||
|
||||
|
||||
def load_sequence(root):
|
||||
gt_dir, frames_dir = os.path.join(root, 'groundtruth'), os.path.join(root, 'input')
|
||||
gt = sorted(glob.glob(os.path.join(gt_dir, '*.png')))
|
||||
f = sorted(glob.glob(os.path.join(frames_dir, '*.jpg')))
|
||||
assert(len(gt) == len(f))
|
||||
return gt, f
|
||||
|
||||
|
||||
def evaluate_algorithm(gt, frames, algo, algo_arguments):
|
||||
bgs = algo(**algo_arguments)
|
||||
mask = []
|
||||
t_start = time.time()
|
||||
|
||||
for i in range(len(gt)):
|
||||
frame = np.uint8(cv.imread(frames[i], cv.IMREAD_COLOR))
|
||||
mask.append(bgs.apply(frame))
|
||||
|
||||
average_duration = (time.time() - t_start) / len(gt)
|
||||
average_precision, average_recall, average_f1, average_accuracy = [], [], [], []
|
||||
|
||||
for i in range(len(gt)):
|
||||
gt_mask = np.uint8(cv.imread(gt[i], cv.IMREAD_GRAYSCALE))
|
||||
roi = ((gt_mask == 255) | (gt_mask == 0))
|
||||
if roi.sum() > 0:
|
||||
gt_answer, answer = gt_mask[roi], mask[i][roi]
|
||||
|
||||
tp = ((answer == 255) & (gt_answer == 255)).sum()
|
||||
tn = ((answer == 0) & (gt_answer == 0)).sum()
|
||||
fp = ((answer == 255) & (gt_answer == 0)).sum()
|
||||
fn = ((answer == 0) & (gt_answer == 255)).sum()
|
||||
|
||||
if tp + fp > 0:
|
||||
average_precision.append(float(tp) / (tp + fp))
|
||||
if tp + fn > 0:
|
||||
average_recall.append(float(tp) / (tp + fn))
|
||||
if tp + fn + fp > 0:
|
||||
average_f1.append(2.0 * tp / (2.0 * tp + fn + fp))
|
||||
average_accuracy.append(float(tp + tn) / (tp + tn + fp + fn))
|
||||
|
||||
return average_duration, np.mean(average_precision), np.mean(average_recall), np.mean(average_f1), np.mean(average_accuracy)
|
||||
|
||||
|
||||
def evaluate_on_sequence(seq, summary):
|
||||
gt, frames = load_sequence(seq)
|
||||
category, video_name = os.path.basename(os.path.dirname(seq)), os.path.basename(seq)
|
||||
print('=== %s:%s ===' % (category, video_name))
|
||||
|
||||
for algo, algo_name, algo_arguments in ALGORITHMS_TO_EVALUATE:
|
||||
print('Algorithm name: %s' % algo_name)
|
||||
sec_per_step, precision, recall, f1, accuracy = evaluate_algorithm(gt, frames, algo, algo_arguments)
|
||||
print('Average accuracy: %.3f' % accuracy)
|
||||
print('Average precision: %.3f' % precision)
|
||||
print('Average recall: %.3f' % recall)
|
||||
print('Average F1: %.3f' % f1)
|
||||
print('Average sec. per step: %.4f' % sec_per_step)
|
||||
print('')
|
||||
|
||||
if category not in summary:
|
||||
summary[category] = {}
|
||||
if algo_name not in summary[category]:
|
||||
summary[category][algo_name] = []
|
||||
summary[category][algo_name].append((precision, recall, f1, accuracy))
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='Evaluate all background subtractors using Change Detection 2014 dataset')
|
||||
parser.add_argument('--dataset_path', help='Path to the directory with dataset. It may contain multiple inner directories. It will be scanned recursively.', required=True)
|
||||
parser.add_argument('--algorithm', help='Test particular algorithm instead of all.')
|
||||
|
||||
args = parser.parse_args()
|
||||
dataset_dirs = find_relevant_dirs(args.dataset_path)
|
||||
assert len(dataset_dirs) > 0, ("Passed directory must contain at least one sequence from the Change Detection dataset. There is no relevant directories in %s. Check that this directory is correct." % (args.dataset_path))
|
||||
if args.algorithm is not None:
|
||||
global ALGORITHMS_TO_EVALUATE
|
||||
ALGORITHMS_TO_EVALUATE = filter(lambda a: a[1].lower() == args.algorithm.lower(), ALGORITHMS_TO_EVALUATE)
|
||||
summary = {}
|
||||
|
||||
for seq in dataset_dirs:
|
||||
evaluate_on_sequence(seq, summary)
|
||||
|
||||
for category in summary:
|
||||
for algo_name in summary[category]:
|
||||
summary[category][algo_name] = np.mean(summary[category][algo_name], axis=0)
|
||||
|
||||
for category in summary:
|
||||
print('=== SUMMARY for %s (Precision, Recall, F1, Accuracy) ===' % category)
|
||||
for algo_name in summary[category]:
|
||||
print('%05s: %.3f %.3f %.3f %.3f' % ((algo_name,) + tuple(summary[category][algo_name])))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,41 @@
|
||||
import numpy as np
|
||||
import cv2 as cv
|
||||
import argparse
|
||||
import os
|
||||
|
||||
|
||||
def main():
|
||||
argparser = argparse.ArgumentParser(description='Vizualization of the LSBP/GSOC background subtraction algorithm.')
|
||||
|
||||
argparser.add_argument('-g', '--gt', help='Directory with ground-truth frames', required=True)
|
||||
argparser.add_argument('-f', '--frames', help='Directory with input frames', required=True)
|
||||
argparser.add_argument('-l', '--lsbp', help='Display LSBP instead of GSOC', default=False)
|
||||
args = argparser.parse_args()
|
||||
|
||||
gt = map(lambda x: os.path.join(args.gt, x), os.listdir(args.gt))
|
||||
gt.sort()
|
||||
f = map(lambda x: os.path.join(args.frames, x), os.listdir(args.frames))
|
||||
f.sort()
|
||||
|
||||
gt = np.uint8(map(lambda x: cv.imread(x, cv.IMREAD_GRAYSCALE), gt))
|
||||
f = np.uint8(map(lambda x: cv.imread(x, cv.IMREAD_COLOR), f))
|
||||
|
||||
if not args.lsbp:
|
||||
bgs = cv.bgsegm.createBackgroundSubtractorGSOC()
|
||||
else:
|
||||
bgs = cv.bgsegm.createBackgroundSubtractorLSBP()
|
||||
|
||||
for i in xrange(f.shape[0]):
|
||||
cv.imshow('Frame', f[i])
|
||||
cv.imshow('Ground-truth', gt[i])
|
||||
mask = bgs.apply(f[i])
|
||||
bg = bgs.getBackgroundImage()
|
||||
cv.imshow('BG', bg)
|
||||
cv.imshow('Output mask', mask)
|
||||
k = cv.waitKey(0)
|
||||
if k == 27:
|
||||
break
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,26 @@
|
||||
import cv2 as cv
|
||||
import argparse
|
||||
|
||||
|
||||
def main():
|
||||
argparser = argparse.ArgumentParser(description='Vizualization of the SyntheticSequenceGenerator.')
|
||||
|
||||
argparser.add_argument('-b', '--background', help='Background image.', required=True)
|
||||
argparser.add_argument('-o', '--obj', help='Object image. It must be strictly smaller than background.', required=True)
|
||||
args = argparser.parse_args()
|
||||
|
||||
bg = cv.imread(args.background)
|
||||
obj = cv.imread(args.obj)
|
||||
generator = cv.bgsegm.createSyntheticSequenceGenerator(bg, obj)
|
||||
|
||||
while True:
|
||||
frame, mask = generator.getNextFrame()
|
||||
cv.imshow('Generated frame', frame)
|
||||
cv.imshow('Generated mask', mask)
|
||||
k = cv.waitKey(int(1000.0 / 30))
|
||||
if k == 27:
|
||||
break
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,485 @@
|
||||
/*M///////////////////////////////////////////////////////////////////////////////////////
|
||||
//
|
||||
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
|
||||
//
|
||||
// By downloading, copying, installing or using the software you agree to this license.
|
||||
// If you do not agree to this license, do not download, install,
|
||||
// copy or use the software.
|
||||
//
|
||||
//
|
||||
// License Agreement
|
||||
// For Open Source Computer Vision Library
|
||||
//
|
||||
// Copyright (C) 2000, Intel Corporation, all rights reserved.
|
||||
// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
|
||||
// Third party copyrights are property of their respective owners.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without modification,
|
||||
// are permitted provided that the following conditions are met:
|
||||
//
|
||||
// * Redistribution's of source code must retain the above copyright notice,
|
||||
// this list of conditions and the following disclaimer.
|
||||
//
|
||||
// * Redistribution's in binary form must reproduce the above copyright notice,
|
||||
// this list of conditions and the following disclaimer in the documentation
|
||||
// and/or other materials provided with the distribution.
|
||||
//
|
||||
// * The name of the copyright holders may not be used to endorse or promote products
|
||||
// derived from this software without specific prior written permission.
|
||||
//
|
||||
// This software is provided by the copyright holders and contributors "as is" and
|
||||
// any express or implied warranties, including, but not limited to, the implied
|
||||
// warranties of merchantability and fitness for a particular purpose are disclaimed.
|
||||
// In no event shall the Intel Corporation or contributors be liable for any direct,
|
||||
// indirect, incidental, special, exemplary, or consequential damages
|
||||
// (including, but not limited to, procurement of substitute goods or services;
|
||||
// loss of use, data, or profits; or business interruption) however caused
|
||||
// and on any theory of liability, whether in contract, strict liability,
|
||||
// or tort (including negligence or otherwise) arising in any way out of
|
||||
// the use of this software, even if advised of the possibility of such damage.
|
||||
//
|
||||
//M*/
|
||||
|
||||
#include "precomp.hpp"
|
||||
#include <float.h>
|
||||
#include "opencv2/core/utils/logger.hpp"
|
||||
|
||||
// to make sure we can use these short names
|
||||
#undef K
|
||||
#undef L
|
||||
#undef T
|
||||
|
||||
// This is based on the "An Improved Adaptive Background Mixture Model for
|
||||
// Real-time Tracking with Shadow Detection" by P. KaewTraKulPong and R. Bowden
|
||||
// http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf
|
||||
//
|
||||
// The windowing method is used, but not the shadow detection. I make some of my
|
||||
// own modifications which make more sense. There are some errors in some of their
|
||||
// equations.
|
||||
//
|
||||
|
||||
namespace cv
|
||||
{
|
||||
namespace bgsegm
|
||||
{
|
||||
|
||||
static const int defaultNMixtures = 5;
|
||||
static const int defaultHistory = 200;
|
||||
static const double defaultBackgroundRatio = 0.7;
|
||||
static const double defaultVarThreshold = 2.5*2.5;
|
||||
static const double defaultNoiseSigma = 30*0.5;
|
||||
static const double defaultInitialWeight = 0.05;
|
||||
|
||||
class BackgroundSubtractorMOGImpl CV_FINAL : public BackgroundSubtractorMOG
|
||||
{
|
||||
public:
|
||||
//! the default constructor
|
||||
BackgroundSubtractorMOGImpl()
|
||||
{
|
||||
frameSize = Size(0,0);
|
||||
frameType = 0;
|
||||
|
||||
nframes = 0;
|
||||
nmixtures = defaultNMixtures;
|
||||
history = defaultHistory;
|
||||
varThreshold = defaultVarThreshold;
|
||||
backgroundRatio = defaultBackgroundRatio;
|
||||
noiseSigma = defaultNoiseSigma;
|
||||
name_ = "BackgroundSubtractor.MOG";
|
||||
}
|
||||
// the full constructor that takes the length of the history,
|
||||
// the number of gaussian mixtures, the background ratio parameter and the noise strength
|
||||
BackgroundSubtractorMOGImpl(int _history, int _nmixtures, double _backgroundRatio, double _noiseSigma=0)
|
||||
{
|
||||
frameSize = Size(0,0);
|
||||
frameType = 0;
|
||||
|
||||
nframes = 0;
|
||||
nmixtures = std::min(_nmixtures > 0 ? _nmixtures : defaultNMixtures, 8);
|
||||
history = _history > 0 ? _history : defaultHistory;
|
||||
varThreshold = defaultVarThreshold;
|
||||
backgroundRatio = std::min(_backgroundRatio > 0 ? _backgroundRatio : 0.95, 1.);
|
||||
noiseSigma = _noiseSigma <= 0 ? defaultNoiseSigma : _noiseSigma;
|
||||
}
|
||||
|
||||
//! the update operator
|
||||
virtual void apply(InputArray image, OutputArray fgmask, double learningRate=0) CV_OVERRIDE;
|
||||
|
||||
virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate) CV_OVERRIDE;
|
||||
|
||||
//! re-initiaization method
|
||||
virtual void initialize(Size _frameSize, int _frameType)
|
||||
{
|
||||
frameSize = _frameSize;
|
||||
frameType = _frameType;
|
||||
nframes = 0;
|
||||
|
||||
int nchannels = CV_MAT_CN(frameType);
|
||||
CV_Assert( CV_MAT_DEPTH(frameType) == CV_8U );
|
||||
|
||||
// for each gaussian mixture of each pixel bg model we store ...
|
||||
// the mixture sort key (w/sum_of_variances), the mixture weight (w),
|
||||
// the mean (nchannels values) and
|
||||
// the diagonal covariance matrix (another nchannels values)
|
||||
bgmodel.create( 1, frameSize.height*frameSize.width*nmixtures*(2 + 2*nchannels), CV_32F );
|
||||
bgmodel = Scalar::all(0);
|
||||
}
|
||||
|
||||
virtual void getBackgroundImage(OutputArray) const CV_OVERRIDE
|
||||
{
|
||||
CV_Error( Error::StsNotImplemented, "" );
|
||||
}
|
||||
|
||||
virtual int getHistory() const CV_OVERRIDE { return history; }
|
||||
virtual void setHistory(int _nframes) CV_OVERRIDE { history = _nframes; }
|
||||
|
||||
virtual int getNMixtures() const CV_OVERRIDE { return nmixtures; }
|
||||
virtual void setNMixtures(int nmix) CV_OVERRIDE { nmixtures = nmix; }
|
||||
|
||||
virtual double getBackgroundRatio() const CV_OVERRIDE { return backgroundRatio; }
|
||||
virtual void setBackgroundRatio(double _backgroundRatio) CV_OVERRIDE { backgroundRatio = _backgroundRatio; }
|
||||
|
||||
virtual double getNoiseSigma() const CV_OVERRIDE { return noiseSigma; }
|
||||
virtual void setNoiseSigma(double _noiseSigma) CV_OVERRIDE { noiseSigma = _noiseSigma; }
|
||||
|
||||
virtual void write(FileStorage& fs) const CV_OVERRIDE
|
||||
{
|
||||
fs << "name" << name_
|
||||
<< "history" << history
|
||||
<< "nmixtures" << nmixtures
|
||||
<< "backgroundRatio" << backgroundRatio
|
||||
<< "noiseSigma" << noiseSigma;
|
||||
}
|
||||
|
||||
virtual void read(const FileNode& fn) CV_OVERRIDE
|
||||
{
|
||||
CV_Assert( (String)fn["name"] == name_ );
|
||||
history = (int)fn["history"];
|
||||
nmixtures = (int)fn["nmixtures"];
|
||||
backgroundRatio = (double)fn["backgroundRatio"];
|
||||
noiseSigma = (double)fn["noiseSigma"];
|
||||
}
|
||||
|
||||
protected:
|
||||
Size frameSize;
|
||||
int frameType;
|
||||
Mat bgmodel;
|
||||
int nframes;
|
||||
int history;
|
||||
int nmixtures;
|
||||
double varThreshold;
|
||||
double backgroundRatio;
|
||||
double noiseSigma;
|
||||
String name_;
|
||||
};
|
||||
|
||||
|
||||
template<typename VT> struct MixData
|
||||
{
|
||||
float sortKey;
|
||||
float weight;
|
||||
VT mean;
|
||||
VT var;
|
||||
};
|
||||
|
||||
|
||||
static void process8uC1( const Mat& image, Mat& fgmask, double learningRate,
|
||||
Mat& bgmodel, int nmixtures, double backgroundRatio,
|
||||
double varThreshold, double noiseSigma )
|
||||
{
|
||||
int x, y, k, k1, rows = image.rows, cols = image.cols;
|
||||
float alpha = (float)learningRate, T = (float)backgroundRatio, vT = (float)varThreshold;
|
||||
int K = nmixtures;
|
||||
MixData<float>* mptr = (MixData<float>*)bgmodel.data;
|
||||
|
||||
const float w0 = (float)defaultInitialWeight;
|
||||
const float sk0 = (float)(w0/(defaultNoiseSigma*2));
|
||||
const float var0 = (float)(defaultNoiseSigma*defaultNoiseSigma*4);
|
||||
const float minVar = (float)(noiseSigma*noiseSigma);
|
||||
|
||||
for( y = 0; y < rows; y++ )
|
||||
{
|
||||
const uchar* src = image.ptr<uchar>(y);
|
||||
uchar* dst = fgmask.ptr<uchar>(y);
|
||||
|
||||
if( alpha > 0 )
|
||||
{
|
||||
for( x = 0; x < cols; x++, mptr += K )
|
||||
{
|
||||
float wsum = 0;
|
||||
float pix = src[x];
|
||||
int kHit = -1, kForeground = -1;
|
||||
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
float w = mptr[k].weight;
|
||||
wsum += w;
|
||||
if( w < FLT_EPSILON )
|
||||
break;
|
||||
float mu = mptr[k].mean;
|
||||
float var = mptr[k].var;
|
||||
float diff = pix - mu;
|
||||
float d2 = diff*diff;
|
||||
if( d2 < vT*var )
|
||||
{
|
||||
wsum -= w;
|
||||
float dw = alpha*(1.f - w);
|
||||
mptr[k].weight = w + dw;
|
||||
mptr[k].mean = mu + alpha*diff;
|
||||
var = std::max(var + alpha*(d2 - var), minVar);
|
||||
mptr[k].var = var;
|
||||
mptr[k].sortKey = w/std::sqrt(var);
|
||||
|
||||
for( k1 = k-1; k1 >= 0; k1-- )
|
||||
{
|
||||
if( mptr[k1].sortKey >= mptr[k1+1].sortKey )
|
||||
break;
|
||||
std::swap( mptr[k1], mptr[k1+1] );
|
||||
}
|
||||
|
||||
kHit = k1+1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if( kHit < 0 ) // no appropriate gaussian mixture found at all, remove the weakest mixture and create a new one
|
||||
{
|
||||
kHit = k = std::min(k, K-1);
|
||||
wsum += w0 - mptr[k].weight;
|
||||
mptr[k].weight = w0;
|
||||
mptr[k].mean = pix;
|
||||
mptr[k].var = var0;
|
||||
mptr[k].sortKey = sk0;
|
||||
}
|
||||
else
|
||||
for( ; k < K; k++ )
|
||||
wsum += mptr[k].weight;
|
||||
|
||||
float wscale = 1.f/wsum;
|
||||
wsum = 0;
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
wsum += mptr[k].weight *= wscale;
|
||||
mptr[k].sortKey *= wscale;
|
||||
if( wsum > T && kForeground < 0 )
|
||||
kForeground = k+1;
|
||||
}
|
||||
|
||||
dst[x] = (uchar)(-(kHit >= kForeground));
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
for( x = 0; x < cols; x++, mptr += K )
|
||||
{
|
||||
float pix = src[x];
|
||||
int kHit = -1, kForeground = -1;
|
||||
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
if( mptr[k].weight < FLT_EPSILON )
|
||||
break;
|
||||
float mu = mptr[k].mean;
|
||||
float var = mptr[k].var;
|
||||
float diff = pix - mu;
|
||||
float d2 = diff*diff;
|
||||
if( d2 < vT*var )
|
||||
{
|
||||
kHit = k;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if( kHit >= 0 )
|
||||
{
|
||||
float wsum = 0;
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
wsum += mptr[k].weight;
|
||||
if( wsum > T )
|
||||
{
|
||||
kForeground = k+1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dst[x] = (uchar)(kHit < 0 || kHit >= kForeground ? 255 : 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
static void process8uC3( const Mat& image, Mat& fgmask, double learningRate,
|
||||
Mat& bgmodel, int nmixtures, double backgroundRatio,
|
||||
double varThreshold, double noiseSigma )
|
||||
{
|
||||
int x, y, k, k1, rows = image.rows, cols = image.cols;
|
||||
float alpha = (float)learningRate, T = (float)backgroundRatio, vT = (float)varThreshold;
|
||||
int K = nmixtures;
|
||||
|
||||
const float w0 = (float)defaultInitialWeight;
|
||||
const float sk0 = (float)(w0/(defaultNoiseSigma*2*std::sqrt(3.)));
|
||||
const float var0 = (float)(defaultNoiseSigma*defaultNoiseSigma*4);
|
||||
const float minVar = (float)(noiseSigma*noiseSigma);
|
||||
MixData<Vec3f>* mptr = (MixData<Vec3f>*)bgmodel.data;
|
||||
|
||||
for( y = 0; y < rows; y++ )
|
||||
{
|
||||
const uchar* src = image.ptr<uchar>(y);
|
||||
uchar* dst = fgmask.ptr<uchar>(y);
|
||||
|
||||
if( alpha > 0 )
|
||||
{
|
||||
for( x = 0; x < cols; x++, mptr += K )
|
||||
{
|
||||
float wsum = 0;
|
||||
Vec3f pix(src[x*3], src[x*3+1], src[x*3+2]);
|
||||
int kHit = -1, kForeground = -1;
|
||||
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
float w = mptr[k].weight;
|
||||
wsum += w;
|
||||
if( w < FLT_EPSILON )
|
||||
break;
|
||||
Vec3f mu = mptr[k].mean;
|
||||
Vec3f var = mptr[k].var;
|
||||
Vec3f diff = pix - mu;
|
||||
float d2 = diff.dot(diff);
|
||||
if( d2 < vT*(var[0] + var[1] + var[2]) )
|
||||
{
|
||||
wsum -= w;
|
||||
float dw = alpha*(1.f - w);
|
||||
mptr[k].weight = w + dw;
|
||||
mptr[k].mean = mu + alpha*diff;
|
||||
var = Vec3f(std::max(var[0] + alpha*(diff[0]*diff[0] - var[0]), minVar),
|
||||
std::max(var[1] + alpha*(diff[1]*diff[1] - var[1]), minVar),
|
||||
std::max(var[2] + alpha*(diff[2]*diff[2] - var[2]), minVar));
|
||||
mptr[k].var = var;
|
||||
mptr[k].sortKey = w/std::sqrt(var[0] + var[1] + var[2]);
|
||||
|
||||
for( k1 = k-1; k1 >= 0; k1-- )
|
||||
{
|
||||
if( mptr[k1].sortKey >= mptr[k1+1].sortKey )
|
||||
break;
|
||||
std::swap( mptr[k1], mptr[k1+1] );
|
||||
}
|
||||
|
||||
kHit = k1+1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if( kHit < 0 ) // no appropriate gaussian mixture found at all, remove the weakest mixture and create a new one
|
||||
{
|
||||
kHit = k = std::min(k, K-1);
|
||||
wsum += w0 - mptr[k].weight;
|
||||
mptr[k].weight = w0;
|
||||
mptr[k].mean = pix;
|
||||
mptr[k].var = Vec3f(var0, var0, var0);
|
||||
mptr[k].sortKey = sk0;
|
||||
}
|
||||
else
|
||||
for( ; k < K; k++ )
|
||||
wsum += mptr[k].weight;
|
||||
|
||||
float wscale = 1.f/wsum;
|
||||
wsum = 0;
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
wsum += mptr[k].weight *= wscale;
|
||||
mptr[k].sortKey *= wscale;
|
||||
if( wsum > T && kForeground < 0 )
|
||||
kForeground = k+1;
|
||||
}
|
||||
|
||||
dst[x] = (uchar)(-(kHit >= kForeground));
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
for( x = 0; x < cols; x++, mptr += K )
|
||||
{
|
||||
Vec3f pix(src[x*3], src[x*3+1], src[x*3+2]);
|
||||
int kHit = -1, kForeground = -1;
|
||||
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
if( mptr[k].weight < FLT_EPSILON )
|
||||
break;
|
||||
Vec3f mu = mptr[k].mean;
|
||||
Vec3f var = mptr[k].var;
|
||||
Vec3f diff = pix - mu;
|
||||
float d2 = diff.dot(diff);
|
||||
if( d2 < vT*(var[0] + var[1] + var[2]) )
|
||||
{
|
||||
kHit = k;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if( kHit >= 0 )
|
||||
{
|
||||
float wsum = 0;
|
||||
for( k = 0; k < K; k++ )
|
||||
{
|
||||
wsum += mptr[k].weight;
|
||||
if( wsum > T )
|
||||
{
|
||||
kForeground = k+1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dst[x] = (uchar)(kHit < 0 || kHit >= kForeground ? 255 : 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void BackgroundSubtractorMOGImpl::apply(InputArray _image, OutputArray _fgmask, double learningRate)
|
||||
{
|
||||
Mat image = _image.getMat();
|
||||
bool needToInitialize = nframes == 0 || learningRate >= 1 || image.size() != frameSize || image.type() != frameType;
|
||||
|
||||
if( needToInitialize )
|
||||
initialize(image.size(), image.type());
|
||||
|
||||
CV_Assert( image.depth() == CV_8U );
|
||||
_fgmask.create( image.size(), CV_8U );
|
||||
Mat fgmask = _fgmask.getMat();
|
||||
|
||||
++nframes;
|
||||
learningRate = learningRate >= 0 && nframes > 1 ? learningRate : 1./std::min( nframes, history );
|
||||
CV_Assert(learningRate >= 0);
|
||||
|
||||
if( image.type() == CV_8UC1 )
|
||||
process8uC1( image, fgmask, learningRate, bgmodel, nmixtures, backgroundRatio, varThreshold, noiseSigma );
|
||||
else if( image.type() == CV_8UC3 )
|
||||
process8uC3( image, fgmask, learningRate, bgmodel, nmixtures, backgroundRatio, varThreshold, noiseSigma );
|
||||
else
|
||||
CV_Error( Error::StsUnsupportedFormat, "Only 1- and 3-channel 8-bit images are supported in BackgroundSubtractorMOG" );
|
||||
}
|
||||
|
||||
void BackgroundSubtractorMOGImpl::apply(InputArray _image, InputArray _knownForegroundMask, OutputArray _fgmask, double learningRate){
|
||||
Mat knownForegroundMask = _knownForegroundMask.getMat();
|
||||
if(!_knownForegroundMask.empty())
|
||||
{
|
||||
CV_LOG_WARNING(NULL, "Known Foreground Masking has not been implemented for this specific background subtractor, falling back to subtraction without known foreground");
|
||||
}
|
||||
apply(_image, _fgmask, learningRate);
|
||||
}
|
||||
|
||||
Ptr<BackgroundSubtractorMOG> createBackgroundSubtractorMOG(int history, int nmixtures,
|
||||
double backgroundRatio, double noiseSigma)
|
||||
{
|
||||
return makePtr<BackgroundSubtractorMOGImpl>(history, nmixtures, backgroundRatio, noiseSigma);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
/* End of file. */
|
||||
@@ -0,0 +1,535 @@
|
||||
/*M///////////////////////////////////////////////////////////////////////////////////////
|
||||
//
|
||||
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
|
||||
//
|
||||
// By downloading, copying, installing or using the software you agree to this license.
|
||||
// If you do not agree to this license, do not download, install,
|
||||
// copy or use the software.
|
||||
//
|
||||
//
|
||||
// License Agreement
|
||||
// For Open Source Computer Vision Library
|
||||
//
|
||||
// Copyright (C) 2000, Intel Corporation, all rights reserved.
|
||||
// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
|
||||
// Third party copyrights are property of their respective owners.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without modification,
|
||||
// are permitted provided that the following conditions are met:
|
||||
//
|
||||
// * Redistribution's of source code must retain the above copyright notice,
|
||||
// this list of conditions and the following disclaimer.
|
||||
//
|
||||
// * Redistribution's in binary form must reproduce the above copyright notice,
|
||||
// this list of conditions and the following disclaimer in the documentation
|
||||
// and/or other materials provided with the distribution.
|
||||
//
|
||||
// * The name of the copyright holders may not be used to endorse or promote products
|
||||
// derived from this software without specific prior written permission.
|
||||
//
|
||||
// This software is provided by the copyright holders and contributors "as is" and
|
||||
// any express or implied warranties, including, but not limited to, the implied
|
||||
// warranties of merchantability and fitness for a particular purpose are disclaimed.
|
||||
// In no event shall the Intel Corporation or contributors be liable for any direct,
|
||||
// indirect, incidental, special, exemplary, or consequential damages
|
||||
// (including, but not limited to, procurement of substitute goods or services;
|
||||
// loss of use, data, or profits; or business interruption) however caused
|
||||
// and on any theory of liability, whether in contract, strict liability,
|
||||
// or tort (including negligence or otherwise) arising in any way out of
|
||||
// the use of this software, even if advised of the possibility of such damage.
|
||||
//
|
||||
//M*/
|
||||
|
||||
/*
|
||||
* This class implements a particular BackgroundSubtraction algorithm described in "Visual Tracking of Human Visitors under
|
||||
* Variable-Lighting Conditions for a Responsive Audio Art Installation," A. Godbehere,
|
||||
* A. Matsukawa, K. Goldberg, American Control Conference, Montreal, June 2012.
|
||||
*
|
||||
* Prepared and integrated by Andrew B. Godbehere.
|
||||
*/
|
||||
|
||||
#include "precomp.hpp"
|
||||
#include "opencv2/core/utility.hpp"
|
||||
#include <limits>
|
||||
#include "opencv2/core/utils/logger.hpp"
|
||||
|
||||
namespace cv
|
||||
{
|
||||
namespace bgsegm
|
||||
{
|
||||
|
||||
class BackgroundSubtractorGMGImpl CV_FINAL : public BackgroundSubtractorGMG
|
||||
{
|
||||
public:
|
||||
BackgroundSubtractorGMGImpl()
|
||||
{
|
||||
/*
|
||||
* Default Parameter Values. Override with algorithm "set" method.
|
||||
*/
|
||||
maxFeatures = 64;
|
||||
learningRate = 0.025;
|
||||
numInitializationFrames = 120;
|
||||
quantizationLevels = 16;
|
||||
backgroundPrior = 0.8;
|
||||
decisionThreshold = 0.8;
|
||||
smoothingRadius = 7;
|
||||
updateBackgroundModel = true;
|
||||
minVal_ = maxVal_ = 0;
|
||||
name_ = "BackgroundSubtractor.GMG";
|
||||
}
|
||||
|
||||
~BackgroundSubtractorGMGImpl()
|
||||
{
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate parameters and set up data structures for appropriate image size.
|
||||
* Must call before running on data.
|
||||
* @param frameSize input frame size
|
||||
* @param min minimum value taken on by pixels in image sequence. Usually 0
|
||||
* @param max maximum value taken on by pixels in image sequence. e.g. 1.0 or 255
|
||||
*/
|
||||
void initialize(Size frameSize, double minVal, double maxVal);
|
||||
|
||||
/**
|
||||
* Performs single-frame background subtraction and builds up a statistical background image
|
||||
* model.
|
||||
* @param image Input image
|
||||
* @param fgmask Output mask image representing foreground and background pixels
|
||||
*/
|
||||
virtual void apply(InputArray image, OutputArray fgmask, double learningRate=-1.0) CV_OVERRIDE;
|
||||
virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate) CV_OVERRIDE;
|
||||
|
||||
/**
|
||||
* Releases all inner buffers.
|
||||
*/
|
||||
void release();
|
||||
|
||||
virtual int getMaxFeatures() const CV_OVERRIDE { return maxFeatures; }
|
||||
virtual void setMaxFeatures(int _maxFeatures) CV_OVERRIDE { maxFeatures = _maxFeatures; }
|
||||
|
||||
virtual double getDefaultLearningRate() const CV_OVERRIDE { return learningRate; }
|
||||
virtual void setDefaultLearningRate(double lr) CV_OVERRIDE { learningRate = lr; }
|
||||
|
||||
virtual int getNumFrames() const CV_OVERRIDE { return numInitializationFrames; }
|
||||
virtual void setNumFrames(int nframes) CV_OVERRIDE { numInitializationFrames = nframes; }
|
||||
|
||||
virtual int getQuantizationLevels() const CV_OVERRIDE { return quantizationLevels; }
|
||||
virtual void setQuantizationLevels(int nlevels) CV_OVERRIDE { quantizationLevels = nlevels; }
|
||||
|
||||
virtual double getBackgroundPrior() const CV_OVERRIDE { return backgroundPrior; }
|
||||
virtual void setBackgroundPrior(double bgprior) CV_OVERRIDE { backgroundPrior = bgprior; }
|
||||
|
||||
virtual int getSmoothingRadius() const CV_OVERRIDE { return smoothingRadius; }
|
||||
virtual void setSmoothingRadius(int radius) CV_OVERRIDE { smoothingRadius = radius; }
|
||||
|
||||
virtual double getDecisionThreshold() const CV_OVERRIDE { return decisionThreshold; }
|
||||
virtual void setDecisionThreshold(double thresh) CV_OVERRIDE { decisionThreshold = thresh; }
|
||||
|
||||
virtual bool getUpdateBackgroundModel() const CV_OVERRIDE { return updateBackgroundModel; }
|
||||
virtual void setUpdateBackgroundModel(bool update) CV_OVERRIDE { updateBackgroundModel = update; }
|
||||
|
||||
virtual double getMinVal() const CV_OVERRIDE { return minVal_; }
|
||||
virtual void setMinVal(double val) CV_OVERRIDE { minVal_ = val; }
|
||||
|
||||
virtual double getMaxVal() const CV_OVERRIDE { return maxVal_; }
|
||||
virtual void setMaxVal(double val) CV_OVERRIDE { maxVal_ = val; }
|
||||
|
||||
virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE
|
||||
{
|
||||
backgroundImage.release();
|
||||
}
|
||||
|
||||
virtual void write(FileStorage& fs) const CV_OVERRIDE
|
||||
{
|
||||
fs << "name" << name_
|
||||
<< "maxFeatures" << maxFeatures
|
||||
<< "defaultLearningRate" << learningRate
|
||||
<< "numFrames" << numInitializationFrames
|
||||
<< "quantizationLevels" << quantizationLevels
|
||||
<< "backgroundPrior" << backgroundPrior
|
||||
<< "decisionThreshold" << decisionThreshold
|
||||
<< "smoothingRadius" << smoothingRadius
|
||||
<< "updateBackgroundModel" << (int)updateBackgroundModel;
|
||||
// we do not save minVal_ & maxVal_, since they depend on the image type.
|
||||
}
|
||||
|
||||
virtual void read(const FileNode& fn) CV_OVERRIDE
|
||||
{
|
||||
CV_Assert( (String)fn["name"] == name_ );
|
||||
maxFeatures = (int)fn["maxFeatures"];
|
||||
learningRate = (double)fn["defaultLearningRate"];
|
||||
numInitializationFrames = (int)fn["numFrames"];
|
||||
quantizationLevels = (int)fn["quantizationLevels"];
|
||||
backgroundPrior = (double)fn["backgroundPrior"];
|
||||
smoothingRadius = (int)fn["smoothingRadius"];
|
||||
decisionThreshold = (double)fn["decisionThreshold"];
|
||||
updateBackgroundModel = (int)fn["updateBackgroundModel"] != 0;
|
||||
minVal_ = maxVal_ = 0;
|
||||
frameSize_ = Size();
|
||||
}
|
||||
|
||||
//! Total number of distinct colors to maintain in histogram.
|
||||
int maxFeatures;
|
||||
//! Set between 0.0 and 1.0, determines how quickly features are "forgotten" from histograms.
|
||||
double learningRate;
|
||||
//! Number of frames of video to use to initialize histograms.
|
||||
int numInitializationFrames;
|
||||
//! Number of discrete levels in each channel to be used in histograms.
|
||||
int quantizationLevels;
|
||||
//! Prior probability that any given pixel is a background pixel. A sensitivity parameter.
|
||||
double backgroundPrior;
|
||||
//! Value above which pixel is determined to be FG.
|
||||
double decisionThreshold;
|
||||
//! Smoothing radius, in pixels, for cleaning up FG image.
|
||||
int smoothingRadius;
|
||||
//! Perform background model update
|
||||
bool updateBackgroundModel;
|
||||
|
||||
private:
|
||||
double maxVal_;
|
||||
double minVal_;
|
||||
|
||||
Size frameSize_;
|
||||
int frameNum_;
|
||||
|
||||
String name_;
|
||||
|
||||
Mat_<int> nfeatures_;
|
||||
Mat_<int> colors_;
|
||||
Mat_<float> weights_;
|
||||
};
|
||||
|
||||
|
||||
void BackgroundSubtractorGMGImpl::initialize(Size frameSize, double minVal, double maxVal)
|
||||
{
|
||||
CV_Assert(minVal < maxVal);
|
||||
CV_Assert(maxFeatures > 0);
|
||||
CV_Assert(learningRate >= 0.0 && learningRate <= 1.0);
|
||||
CV_Assert(numInitializationFrames >= 1);
|
||||
CV_Assert(quantizationLevels >= 1 && quantizationLevels <= 255);
|
||||
CV_Assert(backgroundPrior >= 0.0 && backgroundPrior <= 1.0);
|
||||
|
||||
minVal_ = minVal;
|
||||
maxVal_ = maxVal;
|
||||
|
||||
frameSize_ = frameSize;
|
||||
frameNum_ = 0;
|
||||
|
||||
nfeatures_.create(frameSize_);
|
||||
colors_.create(frameSize_.area(), maxFeatures);
|
||||
weights_.create(frameSize_.area(), maxFeatures);
|
||||
|
||||
nfeatures_.setTo(Scalar::all(0));
|
||||
}
|
||||
|
||||
static float findFeature(int color, const int* colors, const float* weights, int nfeatures)
|
||||
{
|
||||
for (int i = 0; i < nfeatures; ++i)
|
||||
{
|
||||
if (color == colors[i])
|
||||
return weights[i];
|
||||
}
|
||||
|
||||
// not in histogram, so return 0.
|
||||
return 0.0f;
|
||||
}
|
||||
|
||||
static void normalizeHistogram(float* weights, int nfeatures)
|
||||
{
|
||||
float total = 0.0f;
|
||||
for (int i = 0; i < nfeatures; ++i)
|
||||
total += weights[i];
|
||||
|
||||
if (total != 0.0f)
|
||||
{
|
||||
for (int i = 0; i < nfeatures; ++i)
|
||||
weights[i] /= total;
|
||||
}
|
||||
}
|
||||
|
||||
static bool insertFeature(int color, float weight, int* colors, float* weights, int& nfeatures, int maxFeatures)
|
||||
{
|
||||
int idx = -1;
|
||||
for (int i = 0; i < nfeatures; ++i)
|
||||
{
|
||||
if (color == colors[i])
|
||||
{
|
||||
// feature in histogram
|
||||
weight += weights[i];
|
||||
idx = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (idx >= 0)
|
||||
{
|
||||
// move feature to beginning of list
|
||||
|
||||
::memmove(colors + 1, colors, idx * sizeof(int));
|
||||
::memmove(weights + 1, weights, idx * sizeof(float));
|
||||
|
||||
colors[0] = color;
|
||||
weights[0] = weight;
|
||||
}
|
||||
else if (nfeatures == maxFeatures)
|
||||
{
|
||||
// discard oldest feature
|
||||
|
||||
::memmove(colors + 1, colors, (nfeatures - 1) * sizeof(int));
|
||||
::memmove(weights + 1, weights, (nfeatures - 1) * sizeof(float));
|
||||
|
||||
colors[0] = color;
|
||||
weights[0] = weight;
|
||||
}
|
||||
else
|
||||
{
|
||||
colors[nfeatures] = color;
|
||||
weights[nfeatures] = weight;
|
||||
|
||||
++nfeatures;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
template <typename T> struct Quantization
|
||||
{
|
||||
static int apply(const void* src_, int x, int cn, double minVal, double maxVal, int quantizationLevels)
|
||||
{
|
||||
const T* src = static_cast<const T*>(src_);
|
||||
src += x * cn;
|
||||
|
||||
unsigned int res = 0;
|
||||
for (int i = 0, shift = 0; i < cn; ++i, ++src, shift += 8)
|
||||
res |= static_cast<int>((*src - minVal) * quantizationLevels / (maxVal - minVal)) << shift;
|
||||
|
||||
return res;
|
||||
}
|
||||
};
|
||||
|
||||
class GMG_LoopBody : public ParallelLoopBody
|
||||
{
|
||||
public:
|
||||
GMG_LoopBody(const Mat& frame, const Mat& fgmask, const Mat_<int>& nfeatures, const Mat_<int>& colors, const Mat_<float>& weights,
|
||||
int maxFeatures, double learningRate, int numInitializationFrames, int quantizationLevels, double backgroundPrior, double decisionThreshold,
|
||||
double maxVal, double minVal, int frameNum, bool updateBackgroundModel) :
|
||||
frame_(frame), fgmask_(fgmask), nfeatures_(nfeatures), colors_(colors), weights_(weights),
|
||||
maxFeatures_(maxFeatures), learningRate_(learningRate), numInitializationFrames_(numInitializationFrames), quantizationLevels_(quantizationLevels),
|
||||
backgroundPrior_(backgroundPrior), decisionThreshold_(decisionThreshold), updateBackgroundModel_(updateBackgroundModel),
|
||||
maxVal_(maxVal), minVal_(minVal), frameNum_(frameNum)
|
||||
{
|
||||
}
|
||||
|
||||
void operator() (const Range& range) const CV_OVERRIDE;
|
||||
|
||||
private:
|
||||
Mat frame_;
|
||||
|
||||
mutable Mat_<uchar> fgmask_;
|
||||
|
||||
mutable Mat_<int> nfeatures_;
|
||||
mutable Mat_<int> colors_;
|
||||
mutable Mat_<float> weights_;
|
||||
|
||||
int maxFeatures_;
|
||||
double learningRate_;
|
||||
int numInitializationFrames_;
|
||||
int quantizationLevels_;
|
||||
double backgroundPrior_;
|
||||
double decisionThreshold_;
|
||||
bool updateBackgroundModel_;
|
||||
|
||||
double maxVal_;
|
||||
double minVal_;
|
||||
int frameNum_;
|
||||
};
|
||||
|
||||
void GMG_LoopBody::operator() (const Range& range) const
|
||||
{
|
||||
typedef int (*func_t)(const void* src_, int x, int cn, double minVal, double maxVal, int quantizationLevels);
|
||||
static const func_t funcs[] =
|
||||
{
|
||||
Quantization<uchar>::apply,
|
||||
Quantization<schar>::apply,
|
||||
Quantization<ushort>::apply,
|
||||
Quantization<short>::apply,
|
||||
Quantization<int>::apply,
|
||||
Quantization<float>::apply,
|
||||
Quantization<double>::apply
|
||||
};
|
||||
|
||||
const func_t func = funcs[frame_.depth()];
|
||||
CV_Assert(func != 0);
|
||||
|
||||
const int cn = frame_.channels();
|
||||
|
||||
for (int y = range.start, featureIdx = y * frame_.cols; y < range.end; ++y)
|
||||
{
|
||||
const uchar* frame_row = frame_.ptr(y);
|
||||
int* nfeatures_row = nfeatures_[y];
|
||||
uchar* fgmask_row = fgmask_[y];
|
||||
|
||||
for (int x = 0; x < frame_.cols; ++x, ++featureIdx)
|
||||
{
|
||||
int nfeatures = nfeatures_row[x];
|
||||
int* colors = colors_[featureIdx];
|
||||
float* weights = weights_[featureIdx];
|
||||
|
||||
int newFeatureColor = func(frame_row, x, cn, minVal_, maxVal_, quantizationLevels_);
|
||||
|
||||
bool isForeground = false;
|
||||
|
||||
if (frameNum_ >= numInitializationFrames_)
|
||||
{
|
||||
// typical operation
|
||||
|
||||
const double weight = findFeature(newFeatureColor, colors, weights, nfeatures);
|
||||
|
||||
// see Godbehere, Matsukawa, Goldberg (2012) for reasoning behind this implementation of Bayes rule
|
||||
const double posterior = (weight * backgroundPrior_) / (weight * backgroundPrior_ + (1.0 - weight) * (1.0 - backgroundPrior_));
|
||||
|
||||
isForeground = ((1.0 - posterior) > decisionThreshold_);
|
||||
|
||||
// update histogram.
|
||||
|
||||
if (updateBackgroundModel_)
|
||||
{
|
||||
for (int i = 0; i < nfeatures; ++i)
|
||||
weights[i] *= (float)(1.0f - learningRate_);
|
||||
|
||||
bool inserted = insertFeature(newFeatureColor, (float)learningRate_, colors, weights, nfeatures, maxFeatures_);
|
||||
|
||||
if (inserted)
|
||||
{
|
||||
normalizeHistogram(weights, nfeatures);
|
||||
nfeatures_row[x] = nfeatures;
|
||||
}
|
||||
}
|
||||
}
|
||||
else if (updateBackgroundModel_)
|
||||
{
|
||||
// training-mode update
|
||||
|
||||
insertFeature(newFeatureColor, 1.0f, colors, weights, nfeatures, maxFeatures_);
|
||||
|
||||
if (frameNum_ == numInitializationFrames_ - 1)
|
||||
normalizeHistogram(weights, nfeatures);
|
||||
}
|
||||
|
||||
fgmask_row[x] = (uchar)(-(schar)isForeground);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void BackgroundSubtractorGMGImpl::apply(InputArray _frame, OutputArray _fgmask, double newLearningRate)
|
||||
{
|
||||
Mat frame = _frame.getMat();
|
||||
|
||||
const int depth = frame.depth();
|
||||
CV_CheckDepth(depth, (depth == CV_8U) || (depth == CV_8S) ||
|
||||
(depth == CV_16U) || (depth == CV_16S) ||
|
||||
(depth == CV_32S) ||
|
||||
(depth == CV_32F) || (depth == CV_64F), "Unsupported depth");
|
||||
CV_CheckGE(frame.channels(), 1, "Unsupported channels");
|
||||
CV_CheckLE(frame.channels(), 4, "Unsupported channels");
|
||||
|
||||
if (newLearningRate != -1.0)
|
||||
{
|
||||
CV_Assert(newLearningRate >= 0.0 && newLearningRate <= 1.0);
|
||||
learningRate = newLearningRate;
|
||||
}
|
||||
|
||||
if (frame.size() != frameSize_)
|
||||
{
|
||||
double minval = minVal_;
|
||||
double maxval = maxVal_;
|
||||
if( minVal_ == 0 && maxVal_ == 0 )
|
||||
{
|
||||
if( depth == CV_8U ) { minval = std::numeric_limits<uint8_t>::min(); maxval = std::numeric_limits<uint8_t>::max(); }
|
||||
else if( depth == CV_8S ) { minval = std::numeric_limits<int8_t>::min(); maxval = std::numeric_limits<int8_t>::max(); }
|
||||
else if( depth == CV_16U ) { minval = std::numeric_limits<uint16_t>::min();maxval = std::numeric_limits<uint16_t>::max();}
|
||||
else if( depth == CV_16S ) { minval = std::numeric_limits<int16_t>::min(); maxval = std::numeric_limits<int16_t>::max(); }
|
||||
else if( depth == CV_32S ) { minval = std::numeric_limits<int32_t>::min(); maxval = std::numeric_limits<int32_t>::max(); }
|
||||
else /* CV_32F or CV_64F */ { minval = 0.0; maxval = 1.0; }
|
||||
}
|
||||
initialize(frame.size(), minval, maxval);
|
||||
}
|
||||
|
||||
_fgmask.create(frameSize_, CV_8UC1);
|
||||
Mat fgmask = _fgmask.getMat();
|
||||
|
||||
GMG_LoopBody body(frame, fgmask, nfeatures_, colors_, weights_,
|
||||
maxFeatures, learningRate, numInitializationFrames, quantizationLevels, backgroundPrior, decisionThreshold,
|
||||
maxVal_, minVal_, frameNum_, updateBackgroundModel);
|
||||
parallel_for_(Range(0, frame.rows), body, frame.total()/(double)(1<<16));
|
||||
|
||||
if (smoothingRadius > 0)
|
||||
{
|
||||
medianBlur(fgmask, fgmask, smoothingRadius);
|
||||
}
|
||||
|
||||
// keep track of how many frames we have processed
|
||||
++frameNum_;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorGMGImpl::apply(InputArray _image, InputArray _knownForegroundMask, OutputArray _fgmask, double newLearningRate){
|
||||
Mat knownForegroundMask = _knownForegroundMask.getMat();
|
||||
if(!_knownForegroundMask.empty())
|
||||
{
|
||||
CV_LOG_WARNING(NULL, "Known Foreground Masking has not been implemented for this specific background subtractor, falling back to subtraction without known foreground");
|
||||
}
|
||||
apply(_image, _fgmask, newLearningRate);
|
||||
}
|
||||
|
||||
void BackgroundSubtractorGMGImpl::release()
|
||||
{
|
||||
frameSize_ = Size();
|
||||
|
||||
nfeatures_.release();
|
||||
colors_.release();
|
||||
weights_.release();
|
||||
}
|
||||
|
||||
|
||||
Ptr<BackgroundSubtractorGMG> createBackgroundSubtractorGMG(int initializationFrames, double decisionThreshold)
|
||||
{
|
||||
Ptr<BackgroundSubtractorGMG> bgfg = makePtr<BackgroundSubtractorGMGImpl>();
|
||||
bgfg->setNumFrames(initializationFrames);
|
||||
bgfg->setDecisionThreshold(decisionThreshold);
|
||||
|
||||
return bgfg;
|
||||
}
|
||||
|
||||
/*
|
||||
///////////////////////////////////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
CV_INIT_ALGORITHM(BackgroundSubtractorGMG, "BackgroundSubtractor.GMG",
|
||||
obj.info()->addParam(obj, "maxFeatures", obj.maxFeatures,false,0,0,
|
||||
"Maximum number of features to store in histogram. Harsh enforcement of sparsity constraint.");
|
||||
obj.info()->addParam(obj, "learningRate", obj.learningRate,false,0,0,
|
||||
"Adaptation rate of histogram. Close to 1, slow adaptation. Close to 0, fast adaptation, features forgotten quickly.");
|
||||
obj.info()->addParam(obj, "initializationFrames", obj.numInitializationFrames,false,0,0,
|
||||
"Number of frames to use to initialize histograms of pixels.");
|
||||
obj.info()->addParam(obj, "quantizationLevels", obj.quantizationLevels,false,0,0,
|
||||
"Number of discrete colors to be used in histograms. Up-front quantization.");
|
||||
obj.info()->addParam(obj, "backgroundPrior", obj.backgroundPrior,false,0,0,
|
||||
"Prior probability that each individual pixel is a background pixel.");
|
||||
obj.info()->addParam(obj, "smoothingRadius", obj.smoothingRadius,false,0,0,
|
||||
"Radius of smoothing kernel to filter noise from FG mask image.");
|
||||
obj.info()->addParam(obj, "decisionThreshold", obj.decisionThreshold,false,0,0,
|
||||
"Threshold for FG decision rule. Pixel is FG if posterior probability exceeds threshold.");
|
||||
obj.info()->addParam(obj, "updateBackgroundModel", obj.updateBackgroundModel,false,0,0,
|
||||
"Perform background model update.");
|
||||
obj.info()->addParam(obj, "minVal", obj.minVal_,false,0,0,
|
||||
"Minimum of the value range (mostly for regression testing)");
|
||||
obj.info()->addParam(obj, "maxVal", obj.maxVal_,false,0,0,
|
||||
"Maximum of the value range (mostly for regression testing)");
|
||||
);
|
||||
*/
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,432 @@
|
||||
/*M///////////////////////////////////////////////////////////////////////////////////////
|
||||
//
|
||||
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
|
||||
//
|
||||
// By downloading, copying, installing or using the software you agree to this license.
|
||||
// If you do not agree to this license, do not download, install,
|
||||
// copy or use the software.
|
||||
//
|
||||
//
|
||||
// License Agreement
|
||||
// (3-clause BSD License)
|
||||
// For BackgroundSubtractorCNT
|
||||
// (Background Subtraction based on Counting)
|
||||
//
|
||||
// Copyright (C) 2016, Sagi Zeevi (www.theimpossiblecode.com), all rights reserved.
|
||||
// Third party copyrights are property of their respective owners.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without modification,
|
||||
// are permitted provided that the following conditions are met:
|
||||
//
|
||||
// * Redistribution's of source code must retain the above copyright notice,
|
||||
// this list of conditions and the following disclaimer.
|
||||
//
|
||||
// * Redistribution's in binary form must reproduce the above copyright notice,
|
||||
// this list of conditions and the following disclaimer in the documentation
|
||||
// and/or other materials provided with the distribution.
|
||||
//
|
||||
// * The name of the copyright holders may not be used to endorse or promote products
|
||||
// derived from this software without specific prior written permission.
|
||||
//
|
||||
// This software is provided by the copyright holders and contributors "as is" and
|
||||
// any express or implied warranties, including, but not limited to, the implied
|
||||
// warranties of merchantability and fitness for a particular purpose are disclaimed.
|
||||
// In no event shall the Intel Corporation or contributors be liable for any direct,
|
||||
// indirect, incidental, special, exemplary, or consequential damages
|
||||
// (including, but not limited to, procurement of substitute goods or services;
|
||||
// loss of use, data, or profits; or business interruption) however caused
|
||||
// and on any theory of liability, whether in contract, strict liability,
|
||||
// or tort (including negligence or otherwise) arising in any way out of
|
||||
// the use of this software, even if advised of the possibility of such damage.
|
||||
//
|
||||
//M*/
|
||||
|
||||
|
||||
#include "precomp.hpp"
|
||||
#include <functional>
|
||||
#include "opencv2/core/utils/logger.hpp"
|
||||
|
||||
namespace cv
|
||||
{
|
||||
namespace bgsegm
|
||||
{
|
||||
|
||||
class BackgroundSubtractorCNTImpl CV_FINAL : public BackgroundSubtractorCNT
|
||||
{
|
||||
public:
|
||||
|
||||
BackgroundSubtractorCNTImpl(int minStability,
|
||||
bool useHistory,
|
||||
int maxStability,
|
||||
bool isParallel);
|
||||
|
||||
// BackgroundSubtractor interface
|
||||
virtual void apply(InputArray image, OutputArray fgmask, double learningRate) CV_OVERRIDE;
|
||||
virtual void apply(InputArray image, InputArray knownForegroundMask, OutputArray fgmask, double learningRate) CV_OVERRIDE;
|
||||
|
||||
virtual void getBackgroundImage(OutputArray backgroundImage) const CV_OVERRIDE;
|
||||
|
||||
int getMinPixelStability() const CV_OVERRIDE;
|
||||
void setMinPixelStability(int value) CV_OVERRIDE;
|
||||
|
||||
int getMaxPixelStability() const CV_OVERRIDE;
|
||||
void setMaxPixelStability(int value) CV_OVERRIDE;
|
||||
|
||||
bool getUseHistory() const CV_OVERRIDE;
|
||||
void setUseHistory(bool value) CV_OVERRIDE;
|
||||
|
||||
bool getIsParallel() const CV_OVERRIDE;
|
||||
void setIsParallel(bool value) CV_OVERRIDE;
|
||||
|
||||
//! the destructor
|
||||
virtual ~BackgroundSubtractorCNTImpl() {}
|
||||
|
||||
private:
|
||||
int minPixelStability;
|
||||
int maxPixelStability;
|
||||
int threshold;
|
||||
bool useHistory;
|
||||
bool isParallel;
|
||||
// These 3 commented expressed in 1 'data' for faster single access
|
||||
// Mat_<int> stability; // data[0] => Candidate for historyStability if pixel is ~same as in prevFrame
|
||||
// Mat_<int> history; // data[1] => Color which got most hits for the past maxPixelStability frames
|
||||
// Mat_<int> historyStability; // data[2] => How many hits this pixel got for the color in history
|
||||
// Mat_<int> background; // data[3] => Current background as detected by algorithm
|
||||
Mat_<Vec4i> data;
|
||||
Mat prevFrame;
|
||||
Mat fgMaskPrev;
|
||||
};
|
||||
|
||||
BackgroundSubtractorCNTImpl::BackgroundSubtractorCNTImpl(int minStability,
|
||||
bool _useHistory,
|
||||
int maxStability,
|
||||
bool _isParallel)
|
||||
: minPixelStability(minStability),
|
||||
maxPixelStability(maxStability),
|
||||
threshold(5),
|
||||
useHistory(_useHistory),
|
||||
isParallel(_isParallel)
|
||||
{
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::getBackgroundImage(OutputArray _backgroundImage) const
|
||||
{
|
||||
CV_Assert(! data.empty());
|
||||
|
||||
_backgroundImage.create(prevFrame.size(), CV_8U); // OutputArray usage requires this step
|
||||
Mat backgroundImage = _backgroundImage.getMat();
|
||||
|
||||
// mixChannels requires same types to mix,
|
||||
// so imixing with tmp Mat and conerting
|
||||
Mat_<int> tmp(prevFrame.rows, prevFrame.cols);
|
||||
int from_bg_model_to_user[] = {3, 0};
|
||||
mixChannels(&data, 1, &tmp, 1, from_bg_model_to_user, 1);
|
||||
tmp.convertTo(backgroundImage, CV_8U);
|
||||
}
|
||||
|
||||
int BackgroundSubtractorCNTImpl::getMinPixelStability() const
|
||||
{
|
||||
return minPixelStability;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::setMinPixelStability(int value)
|
||||
{
|
||||
CV_Assert(value > 0 && value < maxPixelStability);
|
||||
minPixelStability = value;
|
||||
}
|
||||
|
||||
int BackgroundSubtractorCNTImpl::getMaxPixelStability() const
|
||||
{
|
||||
return maxPixelStability;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::setMaxPixelStability(int value)
|
||||
{
|
||||
CV_Assert(value > minPixelStability);
|
||||
maxPixelStability = value;
|
||||
}
|
||||
|
||||
bool BackgroundSubtractorCNTImpl::getUseHistory() const
|
||||
{
|
||||
return useHistory;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::setUseHistory(bool value)
|
||||
{
|
||||
useHistory = value;
|
||||
}
|
||||
|
||||
bool BackgroundSubtractorCNTImpl::getIsParallel() const
|
||||
{
|
||||
return isParallel;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::setIsParallel(bool value)
|
||||
{
|
||||
isParallel = value;
|
||||
}
|
||||
|
||||
class CNTFunctor
|
||||
{
|
||||
public:
|
||||
virtual void operator()(Vec4i &vec, uchar currColor, uchar prevColor, uchar &fgMaskPixelRef) = 0;
|
||||
//! the destructor
|
||||
virtual ~CNTFunctor() {}
|
||||
};
|
||||
|
||||
struct BGSubtractPixel : public CNTFunctor
|
||||
{
|
||||
BGSubtractPixel(int _minPixelStability, int _threshold,
|
||||
const Mat &_frame, const Mat &_prevFrame, Mat &_fgMask)
|
||||
: minPixelStability(_minPixelStability),
|
||||
threshold(_threshold),
|
||||
frame(_frame),
|
||||
prevFrame(_prevFrame),
|
||||
fgMask(_fgMask)
|
||||
{}
|
||||
|
||||
//! the destructor
|
||||
virtual ~BGSubtractPixel() {}
|
||||
|
||||
void operator()(Vec4i &vec, uchar currColor, uchar prevColor, uchar &fgMaskPixelRef) CV_OVERRIDE
|
||||
{
|
||||
int &stabilityRef = vec[0];
|
||||
int &bgImgRef = vec[3];
|
||||
if (abs(currColor - prevColor) < threshold)
|
||||
{
|
||||
++stabilityRef;
|
||||
if (stabilityRef == minPixelStability)
|
||||
{ // bg
|
||||
--stabilityRef;
|
||||
bgImgRef = prevColor;
|
||||
}
|
||||
else
|
||||
{ // fg
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
}
|
||||
else
|
||||
{ // fg
|
||||
stabilityRef = 0;
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
}
|
||||
|
||||
int minPixelStability;
|
||||
int threshold;
|
||||
const Mat &frame;
|
||||
const Mat &prevFrame;
|
||||
Mat &fgMask;
|
||||
};
|
||||
|
||||
struct BGSubtractPixelWithHistory : public CNTFunctor
|
||||
{
|
||||
BGSubtractPixelWithHistory(int _minPixelStability, int _maxPixelStability, int _threshold,
|
||||
const Mat &_frame, const Mat &_prevFrame, Mat &_fgMask)
|
||||
: minPixelStability(_minPixelStability),
|
||||
maxPixelStability(_maxPixelStability),
|
||||
threshold(_threshold),
|
||||
thresholdHistory(30),
|
||||
frame(_frame),
|
||||
prevFrame(_prevFrame),
|
||||
fgMask(_fgMask)
|
||||
{}
|
||||
|
||||
//! the destructor
|
||||
virtual ~BGSubtractPixelWithHistory() {}
|
||||
|
||||
void incrStability(int &histStabilityRef)
|
||||
{
|
||||
if (histStabilityRef < maxPixelStability)
|
||||
{
|
||||
++histStabilityRef;
|
||||
}
|
||||
}
|
||||
|
||||
void decrStability(int &histStabilityRef)
|
||||
{
|
||||
if (histStabilityRef > 0)
|
||||
{
|
||||
--histStabilityRef;
|
||||
}
|
||||
}
|
||||
|
||||
void operator()(Vec4i &vec, uchar currColor, uchar prevColor, uchar &fgMaskPixelRef) CV_OVERRIDE
|
||||
{
|
||||
int &stabilityRef = vec[0];
|
||||
int &historyColorRef = vec[1];
|
||||
int &histStabilityRef = vec[2];
|
||||
int &bgImgRef = vec[3];
|
||||
if (abs(currColor - historyColorRef) < thresholdHistory)
|
||||
{ // No change compared to history - this is maybe a background
|
||||
stabilityRef = 0;
|
||||
incrStability(histStabilityRef);
|
||||
if (histStabilityRef <= minPixelStability)
|
||||
{
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
else
|
||||
{
|
||||
bgImgRef = historyColorRef;
|
||||
}
|
||||
}
|
||||
else if (abs(currColor - prevColor) < threshold)
|
||||
{ // No change compared to prev - this is maybe a background
|
||||
incrStability(stabilityRef);
|
||||
if (stabilityRef > minPixelStability)
|
||||
{ // Stable color - this is maybe a background
|
||||
if (stabilityRef >= histStabilityRef)
|
||||
{
|
||||
historyColorRef = currColor;
|
||||
histStabilityRef = stabilityRef;
|
||||
bgImgRef = historyColorRef;
|
||||
}
|
||||
else
|
||||
{ // Stable but different from stable history - this is a foreground
|
||||
decrStability(histStabilityRef);
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
}
|
||||
else
|
||||
{ // This is FG.
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
}
|
||||
else
|
||||
{ // Color changed - this is defently a foreground
|
||||
stabilityRef = 0;
|
||||
decrStability(histStabilityRef);
|
||||
fgMaskPixelRef = 255;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
int minPixelStability;
|
||||
int maxPixelStability;
|
||||
int threshold;
|
||||
int thresholdHistory;
|
||||
const Mat &frame;
|
||||
const Mat &prevFrame;
|
||||
Mat &fgMask;
|
||||
};
|
||||
|
||||
class CNTInvoker : public ParallelLoopBody
|
||||
{
|
||||
public:
|
||||
CNTInvoker(Mat_<Vec4i> &_data, Mat &_img, Mat &_prevFrame, Mat &_fgMask, CNTFunctor &_functor)
|
||||
: data(_data), img(_img), prevFrame(_prevFrame), fgMask(_fgMask), functor(_functor)
|
||||
{
|
||||
}
|
||||
|
||||
// Iterate rows
|
||||
void operator()(const Range& range) const CV_OVERRIDE
|
||||
{
|
||||
for (int r = range.start; r < range.end; ++r)
|
||||
{
|
||||
Vec4i* row = data.ptr<Vec4i>(r);
|
||||
uchar* frameRow = img.ptr<uchar>(r);
|
||||
uchar* prevFrameRow = prevFrame.ptr<uchar>(r);
|
||||
uchar* fgMaskRow = fgMask.ptr<uchar>(r);
|
||||
for (int c = 0; c < data.cols; ++c)
|
||||
{
|
||||
functor(row[c], frameRow[c], prevFrameRow[c], fgMaskRow[c]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
Mat_<Vec4i> &data;
|
||||
Mat &img;
|
||||
Mat &prevFrame;
|
||||
Mat &fgMask;
|
||||
CNTFunctor &functor;
|
||||
};
|
||||
|
||||
void BackgroundSubtractorCNTImpl::apply(InputArray image, OutputArray _fgmask, double learningRate)
|
||||
{
|
||||
CV_Assert(image.depth() == CV_8U);
|
||||
|
||||
Mat frameIn = image.getMat();
|
||||
if(frameIn.channels() != 1)
|
||||
cvtColor(frameIn, frameIn, COLOR_BGR2GRAY);
|
||||
|
||||
_fgmask.create(image.size(), CV_8U); // OutputArray usage requires this step
|
||||
Mat fgMask = _fgmask.getMat();
|
||||
|
||||
bool needToInitialize = data.empty() || learningRate >= 1 || frameIn.size() != prevFrame.size();
|
||||
|
||||
Mat frame = frameIn.clone();
|
||||
|
||||
if (needToInitialize)
|
||||
{ // Usually done only once
|
||||
data = Mat_<Vec4i>::zeros(frame.rows, frame.cols);
|
||||
prevFrame = frame;
|
||||
|
||||
// mixChannels requires same types to mix,
|
||||
// so imixing with tmp Mat and conerting
|
||||
Mat tmp;
|
||||
prevFrame.convertTo(tmp, CV_32S);
|
||||
int from_gray_to_history_color[] = {0,1};
|
||||
mixChannels(&tmp, 1, &data, 1, from_gray_to_history_color, 1);
|
||||
}
|
||||
|
||||
fgMask = Scalar(0);
|
||||
CNTFunctor *functor;
|
||||
if (useHistory && learningRate)
|
||||
{
|
||||
double scaleMaxStability = 1.0;
|
||||
if (learningRate > 0 && learningRate < 1.0)
|
||||
{
|
||||
scaleMaxStability = learningRate;
|
||||
}
|
||||
functor = new BGSubtractPixelWithHistory(minPixelStability, int(maxPixelStability * scaleMaxStability),
|
||||
threshold, frame, prevFrame, fgMask);
|
||||
}
|
||||
else
|
||||
{
|
||||
functor = new BGSubtractPixel(minPixelStability, threshold*3, frame, prevFrame, fgMask);
|
||||
}
|
||||
|
||||
if (isParallel)
|
||||
{
|
||||
parallel_for_(Range(0, frame.rows),
|
||||
CNTInvoker(data, frame, prevFrame, fgMask, *functor));
|
||||
}
|
||||
else
|
||||
{
|
||||
for (int r = 0; r < data.rows; ++r)
|
||||
{
|
||||
Vec4i* row = data.ptr<Vec4i>(r);
|
||||
uchar* frameRow = frame.ptr<uchar>(r);
|
||||
uchar* prevFrameRow = prevFrame.ptr<uchar>(r);
|
||||
uchar* fgMaskRow = fgMask.ptr<uchar>(r);
|
||||
for (int c = 0; c < data.cols; ++c)
|
||||
{
|
||||
(*functor)(row[c], frameRow[c], prevFrameRow[c], fgMaskRow[c]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
delete functor;
|
||||
|
||||
prevFrame = frame;
|
||||
}
|
||||
|
||||
void BackgroundSubtractorCNTImpl::apply(InputArray _image, InputArray _knownForegroundMask, OutputArray _fgmask, double learningRate){
|
||||
Mat knownForegroundMask = _knownForegroundMask.getMat();
|
||||
if(!_knownForegroundMask.empty())
|
||||
{
|
||||
CV_LOG_WARNING(NULL, "Known Foreground Masking has not been implemented for this specific background subtractor, falling back to subtraction without known foreground");
|
||||
}
|
||||
apply(_image, _fgmask, learningRate);
|
||||
}
|
||||
|
||||
Ptr<BackgroundSubtractorCNT> createBackgroundSubtractorCNT(int minPixelStability, bool useHistory, int maxStability, bool isParallel)
|
||||
{
|
||||
return makePtr<BackgroundSubtractorCNTImpl>(minPixelStability, useHistory, maxStability, isParallel);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
/* End of file. */
|
||||
@@ -0,0 +1,52 @@
|
||||
/*
|
||||
|
||||
By downloading, copying, installing or using the software you agree to this
|
||||
license. If you do not agree to this license, do not download, install,
|
||||
copy or use the software.
|
||||
|
||||
|
||||
License Agreement
|
||||
For Open Source Computer Vision Library
|
||||
(3-clause BSD License)
|
||||
|
||||
Copyright (C) 2013, OpenCV Foundation, all rights reserved.
|
||||
Third party copyrights are property of their respective owners.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification,
|
||||
are permitted provided that the following conditions are met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright notice,
|
||||
this list of conditions and the following disclaimer.
|
||||
|
||||
* Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
* Neither the names of the copyright holders nor the names of the contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
|
||||
This software is provided by the copyright holders and contributors "as is" and
|
||||
any express or implied warranties, including, but not limited to, the implied
|
||||
warranties of merchantability and fitness for a particular purpose are
|
||||
disclaimed. In no event shall copyright holders or contributors be liable for
|
||||
any direct, indirect, incidental, special, exemplary, or consequential damages
|
||||
(including, but not limited to, procurement of substitute goods or services;
|
||||
loss of use, data, or profits; or business interruption) however caused
|
||||
and on any theory of liability, whether in contract, strict liability,
|
||||
or tort (including negligence or otherwise) arising in any way out of
|
||||
the use of this software, even if advised of the possibility of such damage.
|
||||
|
||||
*/
|
||||
|
||||
#ifndef __OPENCV_BGSEGM_PRECOMP_HPP__
|
||||
#define __OPENCV_BGSEGM_PRECOMP_HPP__
|
||||
|
||||
#include <opencv2/bgsegm.hpp>
|
||||
#include <opencv2/video.hpp>
|
||||
#include <opencv2/imgproc.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,221 @@
|
||||
/*M///////////////////////////////////////////////////////////////////////////////////////
|
||||
//
|
||||
// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
|
||||
//
|
||||
// By downloading, copying, installing or using the software you agree to this license.
|
||||
// If you do not agree to this license, do not download, install,
|
||||
// copy or use the software.
|
||||
//
|
||||
//
|
||||
// License Agreement
|
||||
// For Open Source Computer Vision Library
|
||||
//
|
||||
// Copyright (C) 2000, Intel Corporation, all rights reserved.
|
||||
// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
|
||||
// Third party copyrights are property of their respective owners.
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without modification,
|
||||
// are permitted provided that the following conditions are met:
|
||||
//
|
||||
// * Redistribution's of source code must retain the above copyright notice,
|
||||
// this list of conditions and the following disclaimer.
|
||||
//
|
||||
// * Redistribution's in binary form must reproduce the above copyright notice,
|
||||
// this list of conditions and the following disclaimer in the documentation
|
||||
// and/or other materials provided with the distribution.
|
||||
//
|
||||
// * The name of the copyright holders may not be used to endorse or promote products
|
||||
// derived from this software without specific prior written permission.
|
||||
//
|
||||
// This software is provided by the copyright holders and contributors "as is" and
|
||||
// any express or implied warranties, including, but not limited to, the implied
|
||||
// warranties of merchantability and fitness for a particular purpose are disclaimed.
|
||||
// In no event shall the Intel Corporation or contributors be liable for any direct,
|
||||
// indirect, incidental, special, exemplary, or consequential damages
|
||||
// (including, but not limited to, procurement of substitute goods or services;
|
||||
// loss of use, data, or profits; or business interruption) however caused
|
||||
// and on any theory of liability, whether in contract, strict liability,
|
||||
// or tort (including negligence or otherwise) arising in any way out of
|
||||
// the use of this software, even if advised of the possibility of such damage.
|
||||
//
|
||||
//M*/
|
||||
|
||||
/**
|
||||
* @file synthetic_seq.cpp
|
||||
* @author Vladislav Samsonov <vvladxx@gmail.com>
|
||||
* @brief Synthetic frame sequence generator for testing background subtraction algorithms.
|
||||
*
|
||||
*/
|
||||
|
||||
#include "precomp.hpp"
|
||||
|
||||
namespace cv
|
||||
{
|
||||
namespace bgsegm
|
||||
{
|
||||
namespace
|
||||
{
|
||||
|
||||
inline int clamp(int x, int l, int u) {
|
||||
return ((x) < (l)) ? (l) : (((x) > (u)) ? (u) : (x));
|
||||
}
|
||||
|
||||
inline int within(int a, int b, int c) {
|
||||
return (((a) <= (b)) && ((b) <= (c))) ? 1 : 0;
|
||||
}
|
||||
|
||||
void bilinearInterp(uchar* dest, double x, double y, unsigned bpp, const uchar** values) {
|
||||
x = std::fmod(x, 1.0);
|
||||
y = std::fmod(y, 1.0);
|
||||
|
||||
if (x < 0.0)
|
||||
x += 1.0;
|
||||
if (y < 0.0)
|
||||
y += 1.0;
|
||||
|
||||
for (unsigned i = 0; i < bpp; i++) {
|
||||
double m0 = (1.0 - x) * values[0][i] + x * values[1][i];
|
||||
double m1 = (1.0 - x) * values[2][i] + x * values[3][i];
|
||||
dest[i] = (uchar) ((1.0 - y) * m0 + y * m1);
|
||||
}
|
||||
}
|
||||
|
||||
// Static background is a way too easy test. We will add distortion to it.
|
||||
void waveDistortion(const uchar* src, uchar* dst, int width, int height, int bypp, double amplitude, double wavelength, double phase) {
|
||||
const uchar zeroes[4] = {0, 0, 0, 0};
|
||||
const long rowsiz = width * bypp;
|
||||
const double xhsiz = (double) width / 2.0;
|
||||
const double yhsiz = (double) height / 2.0;
|
||||
double xscale, yscale;
|
||||
|
||||
if (xhsiz < yhsiz) {
|
||||
xscale = yhsiz / xhsiz;
|
||||
yscale = 1.0;
|
||||
}
|
||||
else if (xhsiz > yhsiz) {
|
||||
xscale = 1.0;
|
||||
yscale = xhsiz / yhsiz;
|
||||
}
|
||||
else {
|
||||
xscale = 1.0;
|
||||
yscale = 1.0;
|
||||
}
|
||||
|
||||
wavelength *= 2;
|
||||
|
||||
for (int y = 0; y < height; y++) {
|
||||
uchar* dest = dst;
|
||||
|
||||
for (int x = 0; x < width; x++) {
|
||||
const double dx = x * xscale;
|
||||
const double dy = y * yscale;
|
||||
const double d = sqrt (dx * dx + dy * dy);
|
||||
const double amnt = amplitude * sin(((d / wavelength) * (2.0 * M_PI) + phase));
|
||||
const double needx = (amnt + dx) / xscale;
|
||||
const double needy = (amnt + dy) / yscale;
|
||||
const int xi = clamp(int(needx), 0, width - 2);
|
||||
const int yi = clamp(int(needy), 0, height - 2);
|
||||
|
||||
const uchar* p = src + rowsiz * yi + xi * bypp;
|
||||
|
||||
const int x1_in = within(0, xi, width - 1);
|
||||
const int y1_in = within(0, yi, height - 1);
|
||||
const int x2_in = within(0, xi + 1, width - 1);
|
||||
const int y2_in = within(0, yi + 1, height - 1);
|
||||
const uchar* values[4];
|
||||
|
||||
if (x1_in && y1_in)
|
||||
values[0] = p;
|
||||
else
|
||||
values[0] = zeroes;
|
||||
|
||||
if (x2_in && y1_in)
|
||||
values[1] = p + bypp;
|
||||
else
|
||||
values[1] = zeroes;
|
||||
|
||||
if (x1_in && y2_in)
|
||||
values[2] = p + rowsiz;
|
||||
else
|
||||
values[2] = zeroes;
|
||||
|
||||
if (x2_in && y2_in)
|
||||
values[3] = p + bypp + rowsiz;
|
||||
else
|
||||
values[3] = zeroes;
|
||||
|
||||
bilinearInterp(dest, needx, needy, bypp, values);
|
||||
dest += bypp;
|
||||
}
|
||||
|
||||
dst += rowsiz;
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
SyntheticSequenceGenerator::SyntheticSequenceGenerator(InputArray _background, InputArray _object, double _amplitude, double _wavelength, double _wavespeed, double _objspeed)
|
||||
: amplitude(_amplitude), wavelength(_wavelength), wavespeed(_wavespeed), objspeed(_objspeed), timeStep(0) {
|
||||
_background.getMat().copyTo(background);
|
||||
_object.getMat().copyTo(object);
|
||||
|
||||
if (background.channels() == 1) {
|
||||
cvtColor(background, background, COLOR_GRAY2BGR);
|
||||
}
|
||||
|
||||
if (object.channels() == 1) {
|
||||
cvtColor(object, object, COLOR_GRAY2BGR);
|
||||
}
|
||||
|
||||
CV_Assert(background.channels() == 3);
|
||||
CV_Assert(object.channels() == 3);
|
||||
CV_Assert(background.size().width > object.size().width);
|
||||
CV_Assert(background.size().height > object.size().height);
|
||||
|
||||
background.convertTo(background, CV_8U);
|
||||
object.convertTo(object, CV_8U);
|
||||
|
||||
pos.x = (background.size().width - object.size().width) / 2;
|
||||
pos.y = (background.size().height - object.size().height) / 2;
|
||||
|
||||
const double phi = rng.uniform(0.0, CV_2PI);
|
||||
dir.x = std::cos(phi);
|
||||
dir.y = std::sin(phi);
|
||||
}
|
||||
|
||||
void SyntheticSequenceGenerator::getNextFrame(OutputArray _frame, OutputArray _gtMask) {
|
||||
CV_Assert(!background.empty() && !object.empty());
|
||||
const Size sz = background.size();
|
||||
|
||||
_frame.create(sz, CV_8UC3);
|
||||
Mat frame = _frame.getMat();
|
||||
|
||||
CV_Assert(background.isContinuous() && frame.isContinuous());
|
||||
|
||||
waveDistortion(background.ptr(), frame.ptr(), sz.width, sz.height, 3, amplitude, wavelength, double(timeStep) * wavespeed);
|
||||
|
||||
const Size objSz = object.size();
|
||||
|
||||
object.copyTo(frame(Rect(Point2i(pos), objSz)));
|
||||
|
||||
while (pos.x + dir.x * objspeed < 0 || pos.x + dir.x * objspeed >= sz.width - objSz.width || pos.y + dir.y * objspeed < 0 || pos.y + dir.y * objspeed >= sz.height - objSz.height) {
|
||||
const double phi = rng.uniform(0.0, CV_2PI);
|
||||
dir.x = std::cos(phi);
|
||||
dir.y = std::sin(phi);
|
||||
}
|
||||
|
||||
_gtMask.create(sz, CV_8U);
|
||||
Mat gtMask = _gtMask.getMat();
|
||||
gtMask.setTo(cv::Scalar::all(0));
|
||||
gtMask(Rect(Point2i(pos), objSz)) = 255;
|
||||
|
||||
pos += dir * objspeed;
|
||||
++timeStep;
|
||||
}
|
||||
|
||||
Ptr<SyntheticSequenceGenerator> createSyntheticSequenceGenerator(InputArray background, InputArray object, double amplitude, double wavelength, double wavespeed, double objspeed) {
|
||||
return makePtr<SyntheticSequenceGenerator>(background, object, amplitude, wavelength, wavespeed, objspeed);
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
// This file is part of OpenCV project.
|
||||
// It is subject to the license terms in the LICENSE file found in the top-level directory
|
||||
// of this distribution and at http://opencv.org/license.html.
|
||||
//
|
||||
// Author: andrewgodbehere
|
||||
#include "test_precomp.hpp"
|
||||
|
||||
namespace opencv_test { namespace {
|
||||
|
||||
/**
|
||||
* This test checks the following:
|
||||
* (i) BackgroundSubtractorGMG can operate with matrices of various types and sizes
|
||||
* (ii) Training mode returns empty fgmask
|
||||
* (iii) End of training mode, and anomalous frame yields every pixel detected as FG
|
||||
*/
|
||||
typedef testing::TestWithParam<std::tuple<perf::MatDepth,int>> bgsubgmg_allTypes;
|
||||
TEST_P(bgsubgmg_allTypes, accuracy)
|
||||
{
|
||||
const int depth = get<0>(GetParam());
|
||||
const int ncn = get<1>(GetParam());
|
||||
const int mtype = CV_MAKETYPE(depth, ncn);
|
||||
const int width = 64;
|
||||
const int height = 64;
|
||||
RNG& rng = TS::ptr()->get_rng();
|
||||
|
||||
Ptr<BackgroundSubtractorGMG> fgbg = createBackgroundSubtractorGMG();
|
||||
ASSERT_TRUE(fgbg != nullptr) << "Failed to call createBackgroundSubtractorGMG()";
|
||||
|
||||
/**
|
||||
* Set a few parameters
|
||||
*/
|
||||
fgbg->setSmoothingRadius(7);
|
||||
fgbg->setDecisionThreshold(0.7);
|
||||
fgbg->setNumFrames(120);
|
||||
|
||||
/**
|
||||
* Generate bounds for the values in the matrix for each type
|
||||
*/
|
||||
double maxd = 0, mind = 0;
|
||||
|
||||
/**
|
||||
* Max value for simulated images picked randomly in upper half of type range
|
||||
* Min value for simulated images picked randomly in lower half of type range
|
||||
*/
|
||||
if (depth == CV_8U)
|
||||
{
|
||||
uchar half = UCHAR_MAX/2;
|
||||
maxd = (unsigned char)rng.uniform(half+32, UCHAR_MAX);
|
||||
mind = (unsigned char)rng.uniform(0, half-32);
|
||||
}
|
||||
else if (depth == CV_8S)
|
||||
{
|
||||
maxd = (char)rng.uniform(32, CHAR_MAX);
|
||||
mind = (char)rng.uniform(CHAR_MIN, -32);
|
||||
}
|
||||
else if (depth == CV_16U)
|
||||
{
|
||||
ushort half = USHRT_MAX/2;
|
||||
maxd = (unsigned int)rng.uniform(half+32, USHRT_MAX);
|
||||
mind = (unsigned int)rng.uniform(0, half-32);
|
||||
}
|
||||
else if (depth == CV_16S)
|
||||
{
|
||||
maxd = rng.uniform(32, SHRT_MAX);
|
||||
mind = rng.uniform(SHRT_MIN, -32);
|
||||
}
|
||||
else if (depth == CV_32S)
|
||||
{
|
||||
maxd = rng.uniform(32, INT_MAX);
|
||||
mind = rng.uniform(INT_MIN, -32);
|
||||
}
|
||||
else
|
||||
{
|
||||
ASSERT_TRUE( (depth == CV_32F)||(depth == CV_64F) ) << "Unsupported depth";
|
||||
const double harf = 0.5;
|
||||
const double bias = 0.125; // = 32/256 (Like CV_8U)
|
||||
maxd = rng.uniform(harf + bias, 1.0);
|
||||
mind = rng.uniform(0.0, harf - bias );
|
||||
}
|
||||
|
||||
fgbg->setMinVal(mind);
|
||||
fgbg->setMaxVal(maxd);
|
||||
|
||||
Mat simImage(height, width, mtype);
|
||||
Mat fgmask;
|
||||
|
||||
const Mat fullbg(height, width, CV_8UC1, cv::Scalar(0)); // all background.
|
||||
|
||||
const int numLearningFrames = 120;
|
||||
for (int i = 0; i < numLearningFrames; ++i)
|
||||
{
|
||||
/**
|
||||
* Genrate simulated "image" for any type. Values always confined to upper half of range.
|
||||
*/
|
||||
rng.fill(simImage, RNG::UNIFORM, (mind + maxd)*0.5, maxd);
|
||||
|
||||
/**
|
||||
* Feed simulated images into background subtractor
|
||||
*/
|
||||
fgbg->apply(simImage,fgmask);
|
||||
|
||||
EXPECT_EQ(cv::norm(fgmask, fullbg, NORM_INF), 0) << "foreground mask should be entirely background during training";
|
||||
}
|
||||
//! generate last image, distinct from training images
|
||||
rng.fill(simImage, RNG::UNIFORM, mind, maxd);
|
||||
fgbg->apply(simImage,fgmask);
|
||||
|
||||
const Mat fullfg(height, width, CV_8UC1, cv::Scalar(255)); // all foreground.
|
||||
EXPECT_EQ(cv::norm(fgmask, fullfg, NORM_INF), 0) << "foreground mask should be entirely foreground finally";
|
||||
}
|
||||
|
||||
INSTANTIATE_TEST_CASE_P(/**/,
|
||||
bgsubgmg_allTypes,
|
||||
testing::Combine(
|
||||
testing::Values(CV_8U, CV_8S, CV_16U, CV_16S, CV_32S, CV_32F, CV_64F),
|
||||
testing::Values(1,2,3,4)));
|
||||
|
||||
}} // namespace
|
||||
@@ -0,0 +1,140 @@
|
||||
// This file is part of OpenCV project.
|
||||
// It is subject to the license terms in the LICENSE file found in the top-level directory
|
||||
// of this distribution and at http://opencv.org/license.html.
|
||||
#include "test_precomp.hpp"
|
||||
#include <set>
|
||||
|
||||
namespace opencv_test { namespace {
|
||||
|
||||
static string getDataDir() { return TS::ptr()->get_data_path(); }
|
||||
|
||||
static string getLenaImagePath() { return getDataDir() + "shared/lena.png"; }
|
||||
|
||||
// Simple synthetic illumination invariance test
|
||||
TEST(BackgroundSubtractor_LSBP, IlluminationInvariance)
|
||||
{
|
||||
RNG rng;
|
||||
Mat input(100, 100, CV_32FC3);
|
||||
|
||||
rng.fill(input, RNG::UNIFORM, 0.0f, 0.1f);
|
||||
|
||||
Mat lsv1, lsv2;
|
||||
cv::bgsegm::BackgroundSubtractorLSBPDesc::calcLocalSVDValues(lsv1, input);
|
||||
input *= 10;
|
||||
cv::bgsegm::BackgroundSubtractorLSBPDesc::calcLocalSVDValues(lsv2, input);
|
||||
|
||||
ASSERT_LE(cv::norm(lsv1, lsv2), 0.04f);
|
||||
}
|
||||
|
||||
TEST(BackgroundSubtractor_LSBP, Correctness)
|
||||
{
|
||||
Mat input(3, 3, CV_32FC3);
|
||||
|
||||
float n = 0;
|
||||
for (int i = 0; i < 3; ++i)
|
||||
for (int j = 0; j < 3; ++j) {
|
||||
input.at<Point3f>(i, j) = Point3f(n, n, n);
|
||||
++n;
|
||||
}
|
||||
|
||||
Mat lsv;
|
||||
bgsegm::BackgroundSubtractorLSBPDesc::calcLocalSVDValues(lsv, input);
|
||||
|
||||
EXPECT_LE(std::abs(lsv.at<float>(1, 1) - 0.0903614f), 0.001f);
|
||||
|
||||
input = 1;
|
||||
bgsegm::BackgroundSubtractorLSBPDesc::calcLocalSVDValues(lsv, input);
|
||||
|
||||
EXPECT_LE(std::abs(lsv.at<float>(1, 1) - 0.0f), 0.001f);
|
||||
}
|
||||
|
||||
TEST(BackgroundSubtractor_LSBP, Discrimination)
|
||||
{
|
||||
Point2i LSBPSamplePoints[32];
|
||||
for (int i = 0; i < 32; ++i) {
|
||||
const double phi = i * CV_2PI / 32.0;
|
||||
LSBPSamplePoints[i] = Point2i(int(4 * std::cos(phi)), int(4 * std::sin(phi)));
|
||||
}
|
||||
|
||||
Mat lena = imread(getLenaImagePath());
|
||||
Mat lsv;
|
||||
|
||||
lena.convertTo(lena, CV_32FC3);
|
||||
|
||||
bgsegm::BackgroundSubtractorLSBPDesc::calcLocalSVDValues(lsv, lena);
|
||||
|
||||
Scalar mean, var;
|
||||
meanStdDev(lsv, mean, var);
|
||||
|
||||
EXPECT_GE(mean[0], 0.02);
|
||||
EXPECT_LE(mean[0], 0.04);
|
||||
EXPECT_GE(var[0], 0.03);
|
||||
|
||||
Mat desc;
|
||||
bgsegm::BackgroundSubtractorLSBPDesc::computeFromLocalSVDValues(desc, lsv, LSBPSamplePoints);
|
||||
Size sz = desc.size();
|
||||
std::set<int> distinctive_elements;
|
||||
|
||||
for (int i = 0; i < sz.height; ++i)
|
||||
for (int j = 0; j < sz.width; ++j)
|
||||
distinctive_elements.insert(desc.at<int>(i, j));
|
||||
|
||||
EXPECT_GE(distinctive_elements.size(), 35000U);
|
||||
}
|
||||
|
||||
static double scoreBitwiseReduce(const Mat& mask, const Mat& gtMask, uchar v1, uchar v2) {
|
||||
Mat result;
|
||||
cv::bitwise_and(mask == v1, gtMask == v2, result);
|
||||
return cv::countNonZero(result);
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
static double evaluateBGSAlgorithm(Ptr<T> bgs) {
|
||||
Mat background = imread(getDataDir() + "shared/fruits.png");
|
||||
Mat object = imread(getDataDir() + "shared/baboon.png");
|
||||
cv::resize(object, object, Size(100, 100), 0, 0, INTER_LINEAR_EXACT);
|
||||
Ptr<bgsegm::SyntheticSequenceGenerator> generator = bgsegm::createSyntheticSequenceGenerator(background, object);
|
||||
|
||||
double f1_mean = 0;
|
||||
unsigned total = 0;
|
||||
|
||||
for (int frameNum = 1; frameNum <= 400; ++frameNum) {
|
||||
Mat frame, gtMask;
|
||||
generator->getNextFrame(frame, gtMask);
|
||||
|
||||
Mat mask;
|
||||
bgs->apply(frame, mask);
|
||||
|
||||
Size sz = frame.size();
|
||||
EXPECT_EQ(sz, gtMask.size());
|
||||
EXPECT_EQ(gtMask.size(), mask.size());
|
||||
EXPECT_EQ(mask.type(), gtMask.type());
|
||||
EXPECT_EQ(mask.type(), CV_8U);
|
||||
|
||||
// We will give the algorithm some time for the proper background model inference.
|
||||
// Almost all background subtraction algorithms have a problem with cold start and require some time for background model initialization.
|
||||
// So we will not count first part of the frames in the score.
|
||||
if (frameNum > 300) {
|
||||
const double tp = scoreBitwiseReduce(mask, gtMask, 255, 255);
|
||||
const double fp = scoreBitwiseReduce(mask, gtMask, 255, 0);
|
||||
const double fn = scoreBitwiseReduce(mask, gtMask, 0, 255);
|
||||
|
||||
if (tp + fn + fp > 0) {
|
||||
const double f1_score = 2.0 * tp / (2.0 * tp + fn + fp);
|
||||
f1_mean += f1_score;
|
||||
++total;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
f1_mean /= total;
|
||||
return f1_mean;
|
||||
}
|
||||
|
||||
TEST(BackgroundSubtractor_LSBP, Accuracy)
|
||||
{
|
||||
EXPECT_GE(evaluateBGSAlgorithm(bgsegm::createBackgroundSubtractorGSOC()), 0.9);
|
||||
EXPECT_GE(evaluateBGSAlgorithm(bgsegm::createBackgroundSubtractorLSBP()), 0.25);
|
||||
}
|
||||
|
||||
}} // namespace
|
||||
@@ -0,0 +1,6 @@
|
||||
// This file is part of OpenCV project.
|
||||
// It is subject to the license terms in the LICENSE file found in the top-level directory
|
||||
// of this distribution and at http://opencv.org/license.html.
|
||||
#include "test_precomp.hpp"
|
||||
|
||||
CV_TEST_MAIN("cv")
|
||||
@@ -0,0 +1,16 @@
|
||||
// This file is part of OpenCV project.
|
||||
// It is subject to the license terms in the LICENSE file found in the top-level directory
|
||||
// of this distribution and at http://opencv.org/license.html.
|
||||
#ifndef __OPENCV_TEST_PRECOMP_HPP__
|
||||
#define __OPENCV_TEST_PRECOMP_HPP__
|
||||
|
||||
#include "opencv2/ts.hpp"
|
||||
#include "opencv2/imgproc.hpp"
|
||||
#include "opencv2/imgcodecs.hpp"
|
||||
#include "opencv2/bgsegm.hpp"
|
||||
|
||||
namespace opencv_test {
|
||||
using namespace cv::bgsegm;
|
||||
}
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,133 @@
|
||||
Background Subtraction {#tutorial_bgsegm_bg_subtraction}
|
||||
======================
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
In this chapter,
|
||||
|
||||
- We will familiarize with the background subtraction methods available in OpenCV.
|
||||
|
||||
Basics
|
||||
------
|
||||
|
||||
Background subtraction is a major preprocessing step in many vision-based applications. For
|
||||
example, consider the case of a visitor counter where a static camera takes the number of visitors
|
||||
entering or leaving the room, or a traffic camera extracting information about the vehicles etc. In
|
||||
all these cases, first you need to extract the person or vehicles alone. Technically, you need to
|
||||
extract the moving foreground from static background.
|
||||
|
||||
If you have an image of background alone, like an image of the room without visitors, image of the road
|
||||
without vehicles etc, it is an easy job. Just subtract the new image from the background. You get
|
||||
the foreground objects alone. But in most of the cases, you may not have such an image, so we need
|
||||
to extract the background from whatever images we have. It becomes more complicated when there are
|
||||
shadows of the vehicles. Since shadows also move, simple subtraction will mark that also as
|
||||
foreground. It complicates things.
|
||||
|
||||
Several algorithms were introduced for this purpose.
|
||||
In the following, we will have a look at two algorithms from the `bgsegm` module.
|
||||
|
||||
### BackgroundSubtractorMOG
|
||||
|
||||
It is a Gaussian Mixture-based Background/Foreground Segmentation Algorithm. It was introduced in
|
||||
the paper "An Improved Adaptive Background Mixture Model for Realtime Tracking with Shadow
|
||||
Detection" by P. KaewTraKulPong and R. Bowden in 2001. It uses a method to model each background
|
||||
pixel by a mixture of K Gaussian distributions (K = 3 to 5). The weights of the mixture represent
|
||||
the time proportions that those colours stay in the scene. The probable background colours are the
|
||||
ones which stay longer and more static.
|
||||
|
||||
While coding, we need to create a background object using the function,
|
||||
**cv.bgsegm.createBackgroundSubtractorMOG()**. It has some optional parameters like length of history,
|
||||
number of gaussian mixtures, threshold etc. It is all set to some default values. Then inside the
|
||||
video loop, use backgroundsubtractor.apply() method to get the foreground mask.
|
||||
|
||||
See a simple example below:
|
||||
@code{.py}
|
||||
import numpy as np
|
||||
import cv2 as cv
|
||||
|
||||
cap = cv.VideoCapture('vtest.avi')
|
||||
|
||||
fgbg = cv.bgsegm.createBackgroundSubtractorMOG()
|
||||
|
||||
while(1):
|
||||
ret, frame = cap.read()
|
||||
|
||||
fgmask = fgbg.apply(frame)
|
||||
|
||||
cv.imshow('frame',fgmask)
|
||||
k = cv.waitKey(30) & 0xff
|
||||
if k == 27:
|
||||
break
|
||||
|
||||
cap.release()
|
||||
cv.destroyAllWindows()
|
||||
@endcode
|
||||
( All the results are shown at the end for comparison).
|
||||
|
||||
@note Documentation on the newer method **cv.createBackgroundSubtractorMOG2()** can be found here: @ref tutorial_background_subtraction
|
||||
|
||||
### BackgroundSubtractorGMG
|
||||
|
||||
This algorithm combines statistical background image estimation and per-pixel Bayesian segmentation.
|
||||
It was introduced by Andrew B. Godbehere, Akihiro Matsukawa, and Ken Goldberg in their paper "Visual
|
||||
Tracking of Human Visitors under Variable-Lighting Conditions for a Responsive Audio Art
|
||||
Installation" in 2012. As per the paper, the system ran a successful interactive audio art
|
||||
installation called “Are We There Yet?” from March 31 - July 31 2011 at the Contemporary Jewish
|
||||
Museum in San Francisco, California.
|
||||
|
||||
It uses first few (120 by default) frames for background modelling. It employs probabilistic
|
||||
foreground segmentation algorithm that identifies possible foreground objects using Bayesian
|
||||
inference. The estimates are adaptive; newer observations are more heavily weighted than old
|
||||
observations to accommodate variable illumination. Several morphological filtering operations like
|
||||
closing and opening are done to remove unwanted noise. You will get a black window during first few
|
||||
frames.
|
||||
|
||||
It would be better to apply morphological opening to the result to remove the noises.
|
||||
@code{.py}
|
||||
import numpy as np
|
||||
import cv2 as cv
|
||||
|
||||
cap = cv.VideoCapture('vtest.avi')
|
||||
|
||||
kernel = cv.getStructuringElement(cv.MORPH_ELLIPSE,(3,3))
|
||||
fgbg = cv.bgsegm.createBackgroundSubtractorGMG()
|
||||
|
||||
while(1):
|
||||
ret, frame = cap.read()
|
||||
|
||||
fgmask = fgbg.apply(frame)
|
||||
fgmask = cv.morphologyEx(fgmask, cv.MORPH_OPEN, kernel)
|
||||
|
||||
cv.imshow('frame',fgmask)
|
||||
k = cv.waitKey(30) & 0xff
|
||||
if k == 27:
|
||||
break
|
||||
|
||||
cap.release()
|
||||
cv.destroyAllWindows()
|
||||
@endcode
|
||||
Results
|
||||
-------
|
||||
|
||||
**Original Frame**
|
||||
|
||||
Below image shows the 200th frame of a video
|
||||
|
||||
|
||||
|
||||
**Result of BackgroundSubtractorMOG**
|
||||
|
||||
|
||||
|
||||
**Result of BackgroundSubtractorGMG**
|
||||
|
||||
Noise is removed with morphological opening.
|
||||
|
||||
|
||||
|
||||
Additional Resources
|
||||
--------------------
|
||||
|
||||
Exercises
|
||||
---------
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 6.6 KiB |
@@ -0,0 +1,6 @@
|
||||
Tutorials for bgsegm module {#tutorial_table_of_content_bgsegm}
|
||||
===============================================================
|
||||
|
||||
- @subpage tutorial_bgsegm_bg_subtraction
|
||||
|
||||
In several applications, we need to extract foreground for further operations like object tracking. Background Subtraction is a well-known method in those cases.
|
||||
Reference in New Issue
Block a user