A list of Jupyter notebooks and functions for Computer Vision tasks, using OpenCV, Pillow and Tensorflow
😎
Cheat Sheet: How to debug a Neural Network for Computer Vision problems in Tensorflow
- Masking & Image Manipulation
- Block views & Pooling (Max/Mean/Median)
- Contour Detection
- Convex Hull
- Edge detection using Roberts, Sobel and Canny edge detectors
- Image Descriptors
- SIFT(Scale Invariant Feature Transform): for detecting blobs (a region of an image that greatly differs from its surrounding areas).
- SURF: for detecting blobs. Improved from SIFT and uses Fast Hessian algo
- DAISY descriptors
- HOG descriptor (Histogram of Oriented Gradients) with non-max suppression NMS (
hog_detect_people.py
) - Harris: for detecting corners (
harrisCornerDetection.py
) - FAST (Features from Accelerated Segment Test): for detecting corners
- BRIEF: for detecting blobs
- ORB (Oriented FAST and Rotated BRIEF): for detecting a combination of corners and blobs, uses both FAST and BRIEF (
orb_knn.py
)
- Algo for matching features
- Brute-force matching (
cv2.BFMatcher
class, using KNN and ratio test) (orb_knn.py
) - FLANN-based (Fast Library for Approximate Nearest Neighbors) matching (
flann.py
&flann_homography.py
)
- Brute-force matching (
- Denoising Filters
- Total variation filter: based on the principal that signals with noise have high total variation
- Bilateral filter: good at preserving edges
- Wavelet denoising filter: good at preserving image quality
- Morphological reconstruction
- Erosion (to find holes in image)
- Dilation (to find peaks in image)
- Segmentation and Transformation
- Global (e.g. otsu thresholding) vs local thresholding (e.g. cv.adaptiveThreshold): Thresholding: convert grayscale images to binary, or generally to segment objects from the background
- RAG (Region Adjacency Graph): Used to segment areas of interest from the image. Each region in image is represented as a graph node in RAG, and weight of edge = difference between average colors of pixels in each region
- Watershed algos (Classic vs. Compact): Treats a grayscale image as a topographical map and finds lines between pixels of equal brightness. These lines are then used to segment the image into regions
- Transformation algorithms: warp, swirl from skimage
- Structural similarity index & MSE: measure how two images are different from each other
- Dimension Reduction
- Dictionary Learning
- Convolution kernels
- Autoencoders
Files can the found in the folder Toolkit
- High Pass Filter(HPF) and Low Pass Filter (LPF) (
hpf.py
) - Canny edge detection(
canny.py
) - Find contours (
contours.py
) - Try alls threshold methods, e.g. itsu, isodata, mean, min (
try_all_threshold.py
) - RAG thresholding(
rag_thresholding.py
) - Segmentation with Watershed algos (
watershed_classic.py
andwatershed_compact.py
) - Rotate, scale and translate the image (
warp.py
) - Add noise to image(
add_noise.py
) - Find similarity between images(MSE, Structural Similarity Index)(
ssim.py
) - Histogram comparison (
histogram_comparison.py
using thecompareHist
function from opencv) - Detecting lines with
HoughLines
andHoughLinesP
, circles withHoughCircles
(lineDetection.py
,circleDetection.py
). Detecting other shapes can be done via combiningcv2.findContours
andcv2.approxPolyDP
- Foreground segmentation with
GrabCut
- Haar face detection
haarFaceDetection.py
- Face recognition:
generateImages.py
andfaceRecognition.py
(Eigenfaces, Fisherfaces,Local Binary Patterns Histograms) - Homography, i.e. find images that contain a specific icon (
icon_matcher
folder) - Non-max supression, used for detection with sliding windows where one object may get detected multiple times
non_max_suppression.py
- Customised object detector with SIFT, Bag of Words(BoW), SVM, sliding window and non-max suppression
detector_car_svm.py
anddetector_car_bow_sliding_window.py
- Save and load an SVM detector with
svm.save()
andsvm.load()
Files can the found in the folder Toolkit Video
- Object tracking techniques:
- Background subtraction
- Basic motion detection using background subtraction
basic_motion_detection.py
- MOG background subtractor
mog.py
- KNN background subtractor
knn.py
- GMG background subtractor
gmg.py
- Basic motion detection using background subtraction
- Histogram back-projection with MeanShift or CamShift
meanshift.py
,camshift.py
- Background subtraction
- Kalman filters
kalman.py
,kalman_pedestrian_tracking.py
Files can the found in the folder Toolkit Neural Network
- Simple neural network
simple_neural_net.py
,neural_net_multiple_features.py
- Recognizing handwritten MNIST digits with neural network
neural_net_MNIST.py
. Runtest_neural_net_MNIST.py
to see the neural net's accuracy - Use the model built from MNIST data on new data
detect_and_classify_digits.py
- Ways to improve neural net performance:
- Experiment with the size of your training dataset, the number of hidden nodes, and the number of epochs until you find a peak level of accuracy
- Modify
neural_net_MNIST.create_ann
function so that it supports more than one hidden layer - Try different activation functions. We have used
cv2.ml.ANN_MLP_SIGMOID_SYM
, but it isn't the only option; the others includecv2.ml.ANN_MLP_IDENTITY
,cv2.ml.ANN_MLP_GAUSSIAN
,cv2.ml.ANN_MLP_RELU
, andcv2.ml.ANN_MLP_LEAKYRELU
- Try different training methods. We have used
cv2.ml.ANN_MLP_BACKPROP
. The other options includecv2.ml.ANN_MLP_RPROP
andcv2.ml.ANN_MLP_ANNEAL
- Save and load neural network models
save_and_load_neural_net.py
- Load a deep learning model for tensorflow
load_tf_model.py
- Detect and classify objects with 3rd party neural net: mobileNet + Single Shot Detector
detect_objects_neural_net.py
- Detect and classify faces with 3rd party neural nets:
detect_faces_neural_net.py
- Face detection using the Caffe model
res10_300x300_ssd_iter_140000
- Age and gender detection using the Caffe model
age_net
andgender_net
- Face detection using the Caffe model
Files can the found in the folder toolkit_film_filters
- Emulate the following 4 types of films using curves
- Kodak Portra, a family of films that is optimized for portraits and weddings
class BGRPortraCurveFilter
infilters.py
- Fuji Provia, a family of general-purpose films
class BGRProviaCurveFilter
infilters.py
- Fuji Velvia, a family of films that is optimized for landscapes
class BGRVelviaCurveFilter
infilters.py
- Cross-processing, a nonstandard film processing technique, sometimes used to produce a grungy look in fashion and band photography
- Kodak Portra, a family of films that is optimized for portraits and weddings
- Edge detection (e.g. Sobel, Canny). May need to convert to grayscale first
- Segment detection (e.g. RAG, watershed, GrabCut)
- Transformation(rotation, scale, crop,distanceTransform)
- Apply Gaussian blur to remove noise and make the darkness of image more uniform
- Apply threshold to make image stand out from the background, and erosion to make contours free of irregularities
- Feature extraction
- Feature matching
- Brute Force
- FLANN-based with KNN and ratio test
- Haar cascade classifiers
- Facial recognition: Eigenfaces, Fisherfaces, Local Binary Pattern Histograms (LBPHs)