CSC3067
Video Analytics and Machine Learning
Week 1 - Introduction
Introduction to video surveillance
Image and Video Processing
Computer Vision Applications
Generic object classifier
Module Overview
1. Introduction to Video-Surveillance and Computer Vision
PART 1: IMAGE AND VIDEO PROCESSING
2. Image and video acquisition and characteristics
3. Data Preprocessing: Point Operations
• Brightness enhancement
• Contrast enhancement
4. Data Preprocessing: Neighbourhood Operations
• Filtering and Noise reduction.
• Convolution Medical Imaging
Applications
Video Surveillance
5. Image Segmentation
Defence
• Brightness segmentation
• Template Matching
6. Video Segmentation: Motion Estimation
• Background Subtraction
• Background Mixture Models,
• Optical Flow
Traffic Monitoring
Applications
7. Video Segmentation: Tracking Sport Analysis
• Kalman Filter
• Particle Filter
• Tracking by Detection
Module Overview
PART 2: MACHINE LEARNING AND PATTERN RECOGNITION
8. Machine Learning I
• Type of problems: Verification, detection and identification
• Nearest Neighbour Classifier
• Linear Discriminants
• SVM
• Boosting
Applications OCR
• Random Forest
• Neural Networks
• Intro to Deep Learning
9. Feature extraction I
• Simple Features
• Colour Extraction and Histograms
• Edge extraction
Pedestrian detection
• Rectangular Filters
Applications Biometrics
• SIFT
Activity recognition
• HOG
• Bag of Words
Module Overview
PART 2: MACHINE LEARNING AND PATTERN RECOGNITION
10. Automatic Feature Extraction I
• Dimensionality reduction
• PCA
• LDA
• Active Shape Models Pose Estimation
Applications
• Active Appearance Models Face Recognition
11. Evaluation
• Evaluation Metrics
• Experimental setups
Module Overview
1. Introduction to Video-Surveillance and Computer Vision
PART 1: IMAGE AND VIDEO PROCESSING
2. Image and video acquisition and characteristics
3. Data Preprocessing: Point Operations
• Brightness enhancement
• Contrast enhancement
4. Data Preprocessing: Neighbourhood Operations
• Filtering and Noise reduction.
• Convolution
5. Image Segmentation
• Brightness segmentation
• Template Matching
6. Video Segmentation: Motion Estimation
• Background Subtraction
• Background Mixture Models,
• Optical Flow
7. Video Segmentation: Tracking
• Kalman Filter
• Particle Filter
• Tracking by Detection
Image/Video System
SENSOR
N
IMAGE ANALYSIS
SENSOR O
WORKSTATION
I
T
A A
T
M
A
R
D
O
F
IMAGE ANALYSIS
N
SENSOR
WORKSTATION
I
SENSOR
Video-Surveillance
•
It is a process where video cameras are deployed
in order to monitor the behaviour, activities or
other change information of people for the purpose
of influencing, directing or protecting
Video-Surveillance
•
Surveillance means:
•
Monitoring, observing and listening
to individuals’ movements,
conversations, and other activities or
communications
•
Recording anything monitored,
observed or listened to in the course
of surveillance
•
Using a surveillance device.
•
Finding unidentified person or
abnormal behaviour.
Why Video-Surveillance?
•
The purpose of police and other public authority
surveillance activity is to:
•
Ensure community safety (prevent crime)
•
Secure evidence so offenders may be brought before the
courts
•
Gather intelligence on criminal or terrorist activity and
threats to the public
•
Criminals are quick to exploit new technologies
•
Keep them one step ahead of the law
•
Surveillance capabilities to apprehend these increasingly
sophisticated offenders.
Categories
•
Recording: collecting
information for investigation
and evidence purposes
•
Passive: an employee
monitors a few screens while
working on other tasks
•
Active: automatically
monitoring an area for
assisting security officers
The Analyst
•
Analyst uses image enhancement software
and hardware
•
Makes extraction of
information from the
image easier
•
Advance software and
hardware reduces the
technical skills of the
analyst
Image Analyst Workstation
• Highly
qualified
analyst
A/D
camera host
framegrabber
computer
high-resolution monitor
scanner
image printer
Image processing hard disk
hardware optical disk
Video Surveillance Workstation
• Medium
qualified
analyst
Video Surveillance Workstation
Cyber-Physical Security
•
Cyber-security is the measurements taken to protect
computer assets
•
However how can you
prevent physical attacks?
•
Weakest point is between the screen and the chair
•
Phishing
•
Inside intruder
•
Biometrics
Defence Reconnaissance
•
Situational awareness
•
Target detection
•
Target Tracking
•
Target Recognition
Infrared Camera
Camera
Defence Reconnaissance
•
Show how the image system used in the
defence application fits into the generic
scheme.
Surveillance/Reconnaissance
•
Target detection
•
Mission can last for 4 hours
•
Efficiency drops after 15 mins!
•
Automated classifier can pre-screen video
•
Cues analyst when it has detected something
•
Analyst makes final decision
Surveillance/Reconnaissance
•
Target tracking
•
Once a target is detected, analyst tries
to identify it, i.e. friend or foe
•
Analyst needs to keep moving target in
the centre of the sensor field-of-view
•
Difficult to manoeuvre sensor and to
concentrate on identifying target
Surveillance/Reconnaissance
•
With automated tracking, analyst simply clicks
on the screen at the target position
•
Tracker takes over keeping target in centre of
FOV
•
Analyst hands-free, can concentrate on
identifying moving target
Medical Diagnosis
Prostate Cancer Diagnosis
•
40.000 men are diagnosed every year
•
10,000 men die every year.
•
Methods of diagnosis
•
Prostate specific antigen (PSA) blood test.
•
Needle biopsy.
•
Analysed under microscope by a
pathologist.
Prostate Cancer Diagnosis
•
Biopsy Analysis
•
Stroma (muscular
normal tissue).
•
Cancer (abnormal
tissue development).
•
Pathologist
•
textures
•
structures
Prostate Cancer Diagnosis
•
Uses software
tool to enhance
images.
•
Information
produced:
•
Patient is healthy
•
Patient has
cancer
Cancer Diagnosis
Cancer Diagnosis
•
To analyse a complete slide takes six minutes
•
Too many slides!
•
Automatic classifier pre-screens the slides
•
Extracts interesting slides for analyst to look at.
•
Once again, analyst makes final decision.
Many Other Applications
•
Medical diagnosis
•
Industrial inspection
•
Security, civil surveillance
(CCTV)
•
Defence reconnaissance and
intelligence
•
Ambient assisting living
•
Entertainment
•
Sport analysis
•
Virtual and augmented reality
•
Scientific data processing
Sport Application
Why use automated processing?
•
Sometimes it must be done in real time
•
Video surveillance
•
It is impossible and inefficient to store all the data
•
Analysis of images is often boring and tiring.
•
Terabytes and terabytes of data
•
Leads to reduced efficiency of analyst.
•
Analysts are expensive.
•
Automated processing can assist analyst by performing some
of their functions (Final decision taken by analyst):
•
Prescreening of medical images
•
Visualization of scientific data
•
Alarm triggering to security officer
Why is computer vision so
difficult?
•
Computer vision is useful for automatizing many tasks
•
However there are very few applications fully automated being used
in your every day life or in industry
•
Others coming:
•
Autonomous driving, etc…
Why is computer vision so
difficult?
Currently
• •
Functionalities: Challenges:
• •
Background extraction Real-time
• •
Moving object detection Consistence
• •
Tracking Reliability/robustness
•
Event analysis
• •
One person scenario Illumination changes
•
•
Crowed scenes
Simple activity recognition
•
• Occlusion
Suspicious behaviour detection
•
• Different pose/view point changes
Video retrieval/summarisation •
Low resolution at a distance
•
Video database management •
Non-overlapping cameras
•
•
Controlled environments Appearance changes
•
Ambiguous definition of suspicious
behaviour
•
Large amount of data
Object classifier
Human detector
Action recognition
GENERIC
CLASSIFICATION
SYSTEM
Generic Classification System
Identify to which of a set of categories the
data belong
DATA
INFORMATION
Surveillance: friend or foe
Medical Diagnosis: healthy or cancer
Video games: punch or kick
Fall detector: standing or fall
Quality control: right or faulty
Generic automated system
Enhanced
Image
Image
Data
Data
Image Pre-
Segmentation
Acquisition processing
Binary
Image
Data
Feature
Descriptions
Feature
Information Classification
Extraction
v =[25pix
100pix]
Preprocessing
•
Enhances the image
•
Normalise all images
•
Values
•
Alignment
•
Techniques
•
Contrast enhancement
•
Equalisation
•
Noise reduction
Segmentation
•
Separates image into
objects and
background.
•
First stage binarisation
•
Postprocessing to
‘clean up’ thresholded
binary image.
Feature Extraction
Segmented Image
Feature
v =[P A]
Extraction
What is classification?
•
Classification is the process of assigning a category to an
object in an image from a set of discriminative feature v.
•
v is assigned to some given class C , C , …, C .
1 2 N
C C C
1 2 3
v C =VAN
Classification
3
Defence Reconnaissance
•
Show how the image system used in the
defence application fits into the generic
scheme.
• Situational awareness
• Target detection
• Target Tracking
• Target Recognition
What did we cover today?
•
Video-surveillance
•
Computer vision applications
•
Automatic image/video-processing
•
Machine Learning pipeline
