CMPUT414 Lab Excercise 1
Due date Jan 25, 2020
1 Preliminary Technologies of Background Subtraction 80%
• Reading and Displaying Image Frames: The following code can be used to read and
display the image sequence of the video “highway”. Please understand it and run it on
your computer. A screen shot of the computer will be enough.
1 c l e a r a l l
2 c l o s e a l l
3 c l c
4
5 im pa = ’ . / d a t a / highway / i n p u t / ’ ;
6 i m f t = ’ j p g ’ ;
7
8 [ f i l e s d a t a ] = l o a d D a t a p l u s ( im pa , i m f t ) ;
9 d a t a = do ub l e ( d a t a ) ;
10
11 [ row im column im b y t e i m f r a m e s i m ] = s i z e ( d a t a ) ;
12
13 f o r i = 1 : f r a m e s i m
14 im = d a t a ( : , : , : , i ) ;
15
16 d i s p l a y M a t r i x I m a g e ( i , 1 , 2 , im , im )
17 % d i s p l a y M a t r i x I m a g e ( i , 1 , 3 , im , im , im )
18 % d i s p l a y M a t r i x I m a g e ( i , 2 , 2 , im , im , im , im )
19
20 end
• Capturing Background Image:
A easy method for capturing background im-
age is averaging the value of pixels over time.
Mathematically, it is shown as follows:
Ibk(x,y) = 1N
N∑
t=1
It(x,y) (1)
where It(x,y) denotes the value of the pixel lo-
cated at (x,y) in frames t and N is the number
of frames in the entire video sequence. Write
a Matlab program to find the background
image of video “highway” according to Eq.
1. If you do every thing correctly, the image
should look exactly like the image on right.
1
• Subtracting Background: Now you have the background image and frames from the video;
subtract them to show the result of background subtraction. The subtraction could be a
minus operation, which is shown as follows:
Pt(x,y) = |It(x,y)−Ibk(x,y)| (2)
where Ibk(x,y) is the background image captured by Eq. 1 and Pt(x,y) is the proximity
image of frame t. Write a Matlab program to compute the proximity image of Frame
1000 by Eq. 2. If you do every thing correctly, the result should look like following:
Figure 1: From left to right: Frame of 1000, background image and proximity image.
• Thresholding: The proximity image is compared with a threshold value T , which is a user
parameter, for binary classification of background subtraction. Once the proximity value
is smaller than the T , the corresponding pixel is labelled as background (black point),
otherwise, it is labelled as foreground (white point). Mathematically, it can be represented
as: M(x,y) = g(Pt(x,y),T )
g(Pt(x,y),T ) =
{
255, Pt(x,y) > T
0, otherwise ,
whereM(x,y) denotes the binary mask of the background image. Write a Matlab pro-
gram to compute binary mask. If you do every thing correctly, the binary mask should
look like Fig. 2.
Figure 2: The binary mask following background subtraction.
2 Background Subtraction 20%
You already have a good understanding of background subtraction. Unfortunately, the per-
formance of the binary mask above is not very good, there are some noise points on the plant
in the video. Therefore, the task of this section is improving the accuracy, you can use any
method you like. Once you achieve over 90% in Fm (F-Measure) value for Frame of 1000,
the frame shown in the Fig. 1, you will get full marks for this section. You can check this
webpage (http://jacarini.dinf.usherbrooke.ca/resultEvaluation/) to understand how to calculate
the Fm value, and there is also an example to capture the Fm value of the binary mask shown
in Fig. 2.
1 c l e a r a l l
2 c l o s e a l l
3 c l c
4
5 g t p a = ’ . / d a t a / highway / g r o u n d t r u t h / ’ ;
6 g t f t = ’ png ’ ;
7
8
9 [ f i l e s d a t a ] = l o a d D a t a p l u s ( g t p a , g t f t ) ;
10
11 gt im = d a t a ( : , : , : , 1000) ;
12 fgim = d ou b le ( imread ( ’ . / fg im . png ’ ) ) ;
13
14
15 [ TP FP FN TN] = e v a l u a t i o n e n t r y ( fgim , g t im ) ;
16
17 Re = TP / ( TP + FN)
18 Pr = TP / ( TP + FP )
19 Fm = (2* Pr *Re ) / ( Pr + Re )
The following papers may inspire you on how to improve accuracy. Good luck!
• Heikkila, Marko, and Matti Pietikainen. ”A texture-based method for modeling the background and detect-
ing moving objects.” IEEE transactions on pattern analysis and machine intelligence 28.4 (2006): 657-662.
• Liao, Shengcai, et al. ”Modeling pixel process with scale invariant local patterns for background subtraction
in complex scenes.” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE,
2010.
• Lin, Liang, et al. ”Complex background subtraction by pursuing dynamic spatio-temporal models.” IEEE
Trans. Image Processing 23.7 (2014): 3191-3202.
• Wang, Yi, Zhiming Luo, and Pierre-Marc Jodoin. ”Interactive deep learning method for segmenting moving
objects.” Pattern Recognition Letters 96 (2017): 66-75.
• Babaee, Mohammadreza, Duc Tung Dinh, and Gerhard Rigoll. ”A deep convolutional neural network for
video sequence background subtraction.” Pattern Recognition 76 (2018): 635-649.
• https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=background+subtraction&btnG=