a sample result

I use an early filter approach with small trick. The program displays the original image to the user, but actually processing the filtered version in the learning stage.

The input is filtered before processing the finding stage, and then overlay red start on the original image.

It's obvious that filtering eliminates some false positives, but it also eliminates some overlapping bees.

Again, the learning samples are important for the second stage. I can stack up the learning samples, however, I am not sure whether I should compress them to limit the computation. For now, every learning process starts a new file and abandons all the old data.

Also I think I should head to some kind of texture recognition for the overlapping bees.

median filtered

I am a little not sure where to insert the filtering stage. Early filtering makes the human user more difficult to recognize the object, but late filtering makes the program more difficult to recognize the object.


Window size 6 pixels overlap 2 pixels.
Window size 8 pixels overlap 3 pixels.





compare to the original image:

Nvidia SIGGRAPH 2009 image processing slide

Some of my classmates may be interested on this slide about image processing on GPU:

http://www.slideshare.net/NVIDIA/advances-in-gpu-based-image-processing

corr2 is too expensive

I used to use small input images for the second stage of finding bees. When I switched to original size(1600x1200) the program took minutes to compute the correlation between windows and the samples. This time I changed to use sum(abs(win - sample)) and threshold to different values of positive and negative samples.

positive threshold : negative

0.6 : 1.0
http://farm5.static.flickr.com/4072/4346608685_3b71a707ca_o.png


0.5 : 2.0
http://farm3.static.flickr.com/2790/4346608677_584043b1ed_o.png


0.8 : 2.5
http://farm3.static.flickr.com/2688/4346641775_1578dc4368_o.png


0.8 : 2.0
http://farm3.static.flickr.com/2680/4346641767_f43140b968_o.png


0.8 : 1.5
http://farm3.static.flickr.com/2793/4346641763_2bf8929ac8_o.png


0.8 : 1.0
http://farm3.static.flickr.com/2683/4346641759_81a2639058_o.png


0.7 : 1.0
http://farm5.static.flickr.com/4020/4346641757_152a94b63e_o.png


0.5 : 1.5
http://farm5.static.flickr.com/4047/4346641755_4f6edc29a3_o.png



In sampling phase I intentionally selected bees in a dirty background, for which affected bees in a clean background.

problem of multiply bees and a dirty backdrop

This time I pick bees that are in the dirty background and overlay on each other, and try to match them on a similar image. A lot of false positives appear on the respond.



This image cooperates with negative supervise learning.

A window has one score if it has correlation less than 0.5 with the negative samples, greater than 0.9 correlation with the positive samples. Only those with 3 scores are selected.

update

user selected samples by learning.m

  

responds from finding.m

histograms are not comparable when the value of each bin differs. Matlab hist does not provide fixed value range unless user specified. Since I rewrite the histogram function myself and it returns fixed bin size(33) and fixed value range(0:255/33) the histograms are fine.

The blkproc function does not work as I mention in the last post.

This is a useful link for blkproc: 

http://www.oit.uci.edu/dcslib/matlab/matlab-v53/help/toolbox/images/blkproc.html

I fix my code to use blkproc(img, [15 15], [8 8], myhist) instead of [30 30] [15 15] windows and border.

An image of 310*310 is used, ceil(310/15) = 21, the windows form a 21*21 grid, and each window overlaps by 8 pixels on its four sides.

to reconstruct the index from a 1D positions,    

x = (mod(po(i),21)+1)*15-8;  y = (po(i) / 21 + 1)*15-8;

Negative training set is still not enforced in this test. I spend some time to play with the negative result while forgot to load the positive sample back before doing the finding though. r*s spread all over the place.

from learning to finding

This is a brief description of the project work flow:

The computer learns interested objects, in our case, bees, from human inputs. The "knowledge" of bee recognition is stored as descriptors in a data structure. On an input image, the computer calculate descriptors in the same fashion as the learning phase, and then finds correlations between the input image and the "knowledge". A highly correlated input indicates that the input is highly possible to be the interested object(bee). As the computer learns more about what can be a bee and what should not be a bee, it can distinguish better and better of the input image. 

learning.m is a user interactive script for the program to learn from human: "what is a bee and what isn't". User can double click on the bees (30 for each execution) and the program will store the histogram of color (HOC) and histogram of gradient(HOG) in a [99 30] data structure called positiveResult. Instead of loop over each pixel to accumulate the histogram, his_fast.m uses find function to loop over each bin. Each histogram has only 33 bins vs each image has thousands of pixels.  his_fast.m improves its performance significantly by shorten the for-loop. Only A* B* channels and gradient magnitude are used. L channel is ignored and RGB is converted to gray scale before calculation of gradient magnitude. Therefore positiveResult(:, i) is a descriptor of the ith bee. 

 

finding.m detects bees in a given image. The program divide the input image into 30 by 30 small windows and overlap by 15 pixels. For each small window it calculates the HOC and HOG information (his_image [4620 3] for an input image of 273*397), and then compares them with the positiveResult via corr2. his_image(((i-1)*33 + 1):33*i, :) returns a [33 3] descriptor for each window.

Somehow elements in positiveResult have high correlations with unrelated windows. For an instance of "cropped-2-20081212-091900.jpg", posiveResult(:, 1) and flatten version of image(1:33, :) has correlation of 0.86. This is a problem to be resolved.

I threshold to corr2 returns greater than 0.9, and the window is picked only when 3 or more positiveResult agree.

This is a plot for the 75th window compare to 2nd, 3rd, 9th, and 10th positiveResult.

 

What's next:

A negativeResult can be done in a similar fashion in order to eliminate false positives.

Windows indexing should be built.

Bigger size of positiveResult should be built.

References:

[1] Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf

[2] Stanley Bileschi, Lior Wolf, Image representations beyond histograms of gradients: The role of Gestalt descriptors, http://www.mit.edu/~bileschi/papers/gestalt.pdf