Nima Navab
  • Projects
    • All Projects
    • In/Decline
    • in-between
    • sum
    • deathwhiff
    • fade out, fade in
    • genocide memorial
    • @fear
    • flux 1.0
    • control freak
    • Liquid Light
    • Cloud Chamber
  • Research
    • Atmospheres
    • topological media lab
    • LEAP @ Concordia
  • Blog
    • Current blog
    • 2014 spatial theory blog
  • Bio

OpenCV Library

3/11/2016

 
What is CV (Computer Vision)?

"
Computer vision is a field that includes methods for acquiring, processing, analyzing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions." (source)

What is OpenCV (Open Source Computer Vision Library)?


"OpenCV is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications...

The library has more than 2500 optimized algorithms... These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc. OpenCV has more than 47 thousand people of user community and estimated number of downloads exceeding 7 million...

It has C++, C, Python, Java and MATLAB interfaces and supports Windows, Linux, Android and Mac OS. OpenCV leans mostly towards real-time vision applications and takes advantage of MMX and SSE instructions when available... OpenCV is written natively in C++ and has a templated interface that works seamlessly with STL containers." (source)
Picture
"Making Things See" by Greg Bordenstein (click for source)
'OpenCV for Processing' (Do not mistake for 'OpenCV')

"OpenCV for Processing is based on OpenCV's official Java bindings. It attempts to provide convenient wrappers for common OpenCV functions that are friendly to beginners and feel familiar to the Processing environment." (source)

What can it do:
  1. Face Detection
  2. Brightness Contrast
  3. Filter Images
  4. Find Contours
  5. Find Edges
  6. Find Lines
  7. Brightest Point
  8. Region Of Interest
  9. Image Diff
  10. Dilation And Erosion Thin
  11. Working With Color Images
  12. Background Subtraction
  13. Color Channels
  14. Find Histogram
  15. Hue Range Selection
  16. Calibration Demo
  17. Histogram Skin Detection
  18. Depth From Stereo
  19. Warp Perspective
  20. Marker Detection

LINK TO LIBRARY

*IMPORTANT NOTE:
OpenCV will not run on 32bit (OS 10.6+) OpenCV will not work with Processing 3

cv.jit Computer Vision for Jitter (Max/ Msp)

Picture
Picture
Statistics
  • cv.jit.mean Calculates the mean value of a matrix over time.
  • cv.jit.ravg Calculates the running average of a matrix over time. cv.jit.sum Sums all the pixels in a plane, works for any type/dim.
  • cv.jit.variance Estimates the variance of a matrix over time. cv.jit.stddev Estimates the standard deviation of a matrix over time.

Motion Analysis
  • cv.jit.opticalflow Estimates the optical flow using various algorithms.
  • cv.jit.LKflow Estimates the optical flow using the Lucas-Kanade technique.
  • cv.jit.HSflow Estimates the optical flow using the Horn-Schunk technique.
  • cv.jit.track Track the position of up to 255 individual pixels.
  • cv.jit.features2track Initialize cv.jit.track to easiest pixels to track.
  • cv.jit.framesub Difference between consecutive frames.
  • cv.jit.shift Region tracking using the MeanShift and CAMShift algorithms.
  • cv.jit.touches Track multiple regions at a time. (Optimized for multi-touch interfaces.)

Binary Images
  • cv.jit.threshold Adaptive thresholding.
  • cv.jit.canny Extract binary edges from a greyscale image.
  • cv.jit.binedge Returns only edge pixels.
  • cv.jit.dilate Turns a pixel ON if at least one neighbour is ON.
  • cv.jit.erode Removes the edge pixels from an image. c
  • v.jit.open Erode followed by dilate.
  • cv.jit.close Dilate followed by erode.

Image Segmentation
  • cv.jit.floodfill Isolates a single connected component.
  • cv.jit.label Gives each connected component a unique value.
  • cv.jit.blobs.bounds Find bounding boxes for each connected component.
  • cv.jit.blobs.centroids Find center of mass for each connected component.
  • cv.jit.blobs.direction Find direction each connected component points to.
  • cv.jit.blobs.elongation Calculate elongation for each connected component.
  • cv.jit.blobs.moments Calculate moments of inertia for each connected component.
  • cv.jit.blobs.orientation Measure angle of main axis for each connected component.
  • cv.jit.blobs.recon Carry out pattern recognition on each connected components.
  • cv.jit.blobs.sort Re-arrange labels so that each connected component keeps the same from frame to frame.

Shape Analysis
  • cv.jit.mass Returns the number of non-zero pixels.
  • cv.jit.centroids Calculates the center of mass for an image.
  • cv.jit.moments Computes various invariant shape descriptors.
  • cv.jit.orientation Calculates a shape’s main axis.
  • cv.jit.direction Calculates the direction a shape points to.
  • cv.jit.perimeter Counts the number of edge pixels.
  • cv.jit.elongation Estimates how thin a shape is.
  • cv.jit.circularity Estimates how compact a shape is.
  • cv.jit.undergrad Performs simple pattern recognition.
  • cv.jit.learn Performs pattern analyis and recognition on an incoming list.
  • cv.jit.faces Finds human faces in an image.
  • cv.jit.features Finds areas of high contrast, pixels that are easy to track.
  • cv.jit.lines Finds straight lines.
  • cv.jit.hough Compute Hough space. (by Christopher P. Baker and Mateusz Herczka.)
  • cv.jit.hough2lines Find straight lines in Hough space.
  • cv.jit.snake Fit a point sequence to image edges.

Miscelaneous
  • cv.jit.grab Cross-platform wrapper for jit.qt.grab/jit.dx.grab.
  • cv.jit.changetype Change the type of a matrix without changing other attributes.
  • cv.jit.resize Anti-aliased matrix resize.
  • cv.jit.cartopol Treats the data in two matrices as cartesian coordinates and translates to polar data. cv.jit.poltocar …and vice versa.

Drawing and display
  • cv.jit.track.draw Visualize output of cv.jit.track.
  • cv.jit.lines.draw Visualize output of cv.jit.lines.
  • cv.jit.features.draw Visualize output of cv.jit.features.
  • cv.jit.faces.draw Visualize output of cv.jit.faces.
  • cv.jit.centroids.draw Visualize output of cv.jit.centroids.
  • cv.jit.blobs.orient.draw Visualize output of cv.jit.blobs.orientation.
  • cv.jit.blobs.elongation.draw Visualize output of cv.jit.blobs.elongation.
  • cv.jit.blobs.direction.draw Visualize output of cv.jit.blobs.direction.
  • cv.jit.blobs.centroids.draw Visualize output of cv.jit.blobs.centroids.
  • cv.jit.blobs.bounds.draw Visualize output of cv.jit.blobs.bounds.
  • cv.jit.blobs.color Visualize output of cv.jit.label.
  • cv.jit.shift.draw Drawing utility for cv.jit.shift.
  • cv.jit.flow.draw Display optical flow using hue and saturation.
  • cv.jit.touches.draw Drawing utility for cv.jit.touches.

Obsolete objects
  • cv.jit.covariance Computes the covariance matrix of a vector.
  • cv.jit.mahalanobis Computes the Mahalanobis metric.
  • cv.jit.hmean Calculates the harmonic mean over time.
  • cv.jit.gmean Calculates the geometric mean over time.
  • cv.jit.trackpoints Display utility for cv.jit.track.
  • cv.jit.trackgroup Manager utility for cv.jit.track.
  • cv.jit.shapeinfo Wrapper for cv.jit.moments.

LINK TO LIBRARY
*IMPORTANT NOTE:
Patch requires Kinnect 1, Max/ Msp, Jitter, CNMAT Externals, tapTools, freenectlib, oscP5

Tangible Physics Engine Prototype Setting & Processing Pipeline

Picture
Picture
Physical Setup:
  • Kinect hooked up on top the table proving RGB normal video plane matrix, IR matrix and depth map
  • Short throw projecting on top of the table

Mapping:
  • Projection mapping using syphon client mapped through MadMapper

Max/ Jitter:
  • Kinect hooked up using jit.freenect (freenect kinect library for Max)
  • Freenect in mode 1 (depth map) set at 5” depth cut off from table surface (like a scan)
  • Background subtraction + slide down to only see new objects on the surface
  • Using computer vision library’s jit.blobs.centroid, iterate through every blob’s x, y, z position and mass
  • For every frame scale, prepend and route tracking info via Open Sound Control protocol (OSC library
  • for Max by CNMAT), achieved through identifying a port on local host

In Processing:
  • Using OSCp5 library in processing receive OSC message and control the location of attractor, repellor,
  • generator and etc. as part of the tangible physics engine/ particle system.

Comments are closed.

    BLOG:
    Research & Creation

    An archive of process:
    including ideas, inspirations, sketches, references, etc.

    Archives:

    November 2016
    October 2016
    April 2016
    March 2016
    December 2015
    November 2015
    October 2015
    September 2015
    May 2015
    January 2015

    Categories

    All
    Ideas
    Inspirations
    Materials
    Photo Journalism
    Pneumatics
    Process Work
    Prototyping
    Research
    Responsive Environments
    Scenerios
    Sensors
    Site
    Spatial Design
    Spatial Theory
    TML
    Workshops

Powered by Create your own unique website with customizable templates.