How to spearate the output of vl_hog() into 6*6*31 blocks and use reshape() to convert every block to a row vector?

2 views (last 30 days)
How to spearate the output of vl_hog() into 6*6*31 blocks,
use reshape() to convert every block to a row vector,
save each vector to features_neg, and increase idx by 1
in the following code:
step=feature_params.hog_cell_size;
idx=0;
for i=1:num_images
im=imread([non_face_scn_path '/' image_files(i).name]);
im=single(im);
hog = vl_hog(im, feature_parames.hog_cell_size, 'verbose');
for m=1:step:size(im,1)
for n=1:step:size(im,2)
%%%%% spearate the output of vl_hog() into 6*6*31 blocks,
%%%%% use reshape() to convert every block to a row vector,
%%%%% save each vector to features_neg, and increase idx by 1
%%%%%
features_neg(idx,:)=reshape(hog,1, []);
idx=idx+1
  2 Comments
Walter Roberson
Walter Roberson on 26 Nov 2020
Do you mean you want each block to be 6 x 6 x 31 ? Or do you mean that you want the code to figure out what block size to use such that you can fit 6 blocks down by 6 blocks across by 31 blocks tall ? Does hog start out as 3 dimensional, or does it start out as a vector? What do you want done if the size of the first dimension is not an exact multiple of 6?
For example if hog is input as 512 x 512, then how many blocks of what size are you expecting as output?
Pooyan Mobtahej
Pooyan Mobtahej on 26 Nov 2020
It is face detection using sliding window and i am trying to fill the function for get random negative features. The size is diffrenet and we use the followng code for sliding window and I want to fill the part that I mentioned, I put my code but don't know if it is correct, you can take a look:
% Starter code prepared by James Hays, Brown University
% This function should return negative training examples (non-faces) from
% any images in 'non_face_scn_path'. Images should be converted to
% grayscale, because the positive training data is only available in
% grayscale. For best performance, you should sample random negative
% examples at multiple scales.
function features_neg = get_random_negative_features(non_face_scn_path, feature_params, num_samples)
% 'non_face_scn_path' is a string. This directory contains many images
% which have no faces in them.
% 'feature_params' is a struct, with fields
% feature_params.template_size (probably 36), the number of pixels
% spanned by each train / test template and
% feature_params.hog_cell_size (default 6), the number of pixels in each
% HoG cell. template size should be evenly divisible by hog_cell_size.
% Smaller HoG cell sizes tend to work better, but they make things
% slower because the feature dimensionality increases and more
% importantly the step size of the classifier decreases at test time.
% 'num_samples' is the number of random negatives to be mined, it's not
% important for the function to find exactly 'num_samples' non-face
% features, e.g. you might try to sample some number from each image, but
% some images might be too small to find enough.
% 'features_neg' is N by D matrix where N is the number of non-faces and D
% is the template dimensionality, which would be
% (feature_params.template_size / feature_params.hog_cell_size)^2 * 31
% if you're using the default vl_hog parameters
% Useful functions:
% vl_hog, HOG = VL_HOG(IM, CELLSIZE)
% http://www.vlfeat.org/matlab/vl_hog.html (API)
% http://www.vlfeat.org/overview/hog.html (Tutorial)
% rgb2gray
image_files = dir(fullfile(non_face_scn_path, '*.jpg'));
num_images = length(image_files);
% placeholder to be deleted
features_neg = rand(100, (feature_params.template_size / feature_params.hog_cell_size)^2 * 31);
step=feature_params.hog_cell_size;
idx=0;
for i=1:num_images
im=imread([non_face_scn_path '/' image_files(i).name]);
%%%%% please follow the steps below to compute HOG features for each image.
%%%%% 1. use single() function to convert input image to SINGLE class;
%%%%% 2. call vl_hog() function, two parameters: one is the image, the
%%%%% other is the feature_parames.hog_cell_size;
%%%%%
im=single(im);
hog = vl_hog(im, feature_parames.hog_cell_size, 'verbose');
for m=1:step:size(im,1)
for n=1:step:size(im,2)
%%%%% spearate the output of vl_hog() into 6*6*31 blocks,
%%%%% use reshape() to convert every block to a row vector,
%%%%% save each vector to features_neg, and increase idx by 1
%%%%% Your code here!
features_neg(idx,:)=reshape(hog, [1, 6*6*31]);
idx=idx+1
end
end
disp(['idex is: ' num2str(idx)]);
disp(['case is: ' num2str(i)]);
end
idx

Sign in to comment.

Accepted Answer

Walter Roberson
Walter Roberson on 26 Nov 2020
Edited: Walter Roberson on 26 Nov 2020
If you have a cell array of blocks, then
vector_blocks_cell = cellfun(@(B) reshape(B,1,[]), non_vector_blocks_cell, 'uniform', 0);
  10 Comments
Pooyan Mobtahej
Pooyan Mobtahej on 27 Nov 2020
I am trying to modify face detection project which I will attach the project code for you to check! The i shall get positive and negative random featuresusing HOG and then do SVM classification so please check if I am doing write in codes besides the one that I mentioned
Project:
% Sliding window face detection with linear SVM.
% All code by James Hays, except for pieces of evaluation code from Pascal
% VOC toolkit. Images from CMU+MIT face database, CalTech Web Face
% Database, and SUN scene database.
% Code structure:
% project.m <--- You code parts of this
% + get_positive_features.m <--- You code this
% + get_random_negative_features.m <--- You code this
% [classifier training] <--- You code this
% + report_accuracy.m
% + run_detector.m <--- You code this
% + non_max_supr_bbox.m
% + evaluate_all_detections.m
% + VOCap.m
% + visualize_detections_by_image.m
% + visualize_detections_by_image_no_gt.m
% + visualize_detections_by_confidence.m
% Other functions. You don't need to use any of these unless you're trying
% to modify or build a test set:
% Training and Testing data related functions:
% test_scenes/visualize_cmumit_database_landmarks.m
% test_scenes/visualize_cmumit_database_bboxes.m
% test_scenes/cmumit_database_points_to_bboxes.m %This function converts
% from the original MIT+CMU test set landmark points to Pascal VOC
% annotation format (bounding boxes).
% caltech_faces/caltech_database_points_to_crops.m %This function extracts
% training crops from the Caltech Web Face Database. The crops are
% intentionally large to contain most of the head, not just the face. The
% test_scene annotations are likewise scaled to contain most of the head.
% set up paths to VLFeat functions.
% See http://www.vlfeat.org/matlab/matlab.html for VLFeat Matlab documentation
% This should work on 32 and 64 bit versions of Windows, MacOS, and Linux
close all
clear all
%%%%% JZ: You may need to change the directory below to the directory where you installed VLFeat
run('/Users/pooyan/Documents/computer Vision/vlfeat-0.9.21 3/toolbox/vl_setup')
[~,~,~] = mkdir('visualizations');
%%%%% JZ: You don't need to change anything below (line 47-52)
data_path = '/Users/pooyan/Documents/projectcv/data/'; %change if you want to work with a network copy
train_path_pos = fullfile(data_path, 'caltech_faces/Caltech_CropFaces'); %Positive training examples. 36x36 head crops
non_face_scn_path = fullfile(data_path, 'train_non_face_scenes'); %We can mine random or hard negatives from here
test_scn_path = fullfile(data_path,'test_scenes/test_jpg'); %CMU+MIT test scenes
label_path = fullfile(data_path,'test_scenes/ground_truth_bboxes.txt'); %the ground truth face locations in the test set
%test_scn_path = fullfile(data_path,'test_scenes/test_class'); %Bonus scenes
%label_path = fullfile(data_path,'test_scenes/ground_truth_class_bboxes.txt'); %the ground truth face locations in the test set
%The faces are 36x36 pixels, which works fine as a template size. You could
%add other fields to this struct if you want to modify HoG default
%parameters such as the number of orientations, but that does not help
%performance in our limited test.
feature_params = struct('template_size', 36, 'hog_cell_size', 6);
% Step 1. Load positive training crops and random negative examples
%%%%% JZ: Please open get_positive_features.m file and complete it.
features_pos = get_positive_features( train_path_pos, feature_params );
%%%% JZ: you may change num_negative_examples to see the performance.
num_negative_examples = 10000; %Higher will work strictly better, but you should start with 10000 for debugging
%%%% JZ: Please open get_random_negative_features.m and complete it.
features_neg = get_random_negative_features( non_face_scn_path, feature_params, num_negative_examples);
% step 2. Train Classifier
% Use vl_svmtrain on your training features to get a linear classifier
% specified by 'w' and 'b'
% [w b] = vl_svmtrain(X, Y, lambda)
% http://www.vlfeat.org/sandbox/matlab/vl_svmtrain.html
% 'lambda' is an important parameter, try many values. Small values seem to
% work best e.g. 0.0001, but you can try other values
%
% (1) num_examples is a variable that defines the number of positive examples (face) and negative examples (non-face).
% (2) randomly select num_examples positive examples and num_examples negative examples
% (3) F is the training dataset containing positive and negative examples. Totol 2*num_examples examples.
% (4) Label is the class of each examples in F. Positive examples have class 1 and negative examples have class -1.
num_examples = length(features_pos);
selectedcase = randperm(length(features_pos),num_examples);
features_pos=features_pos(selectedcase,:);
selectedcase = randperm(length(features_neg),num_examples);
features_neg=features_neg(selectedcase,:);
Label=[ones(length(features_pos),1); ones(length(features_neg),1)*-1]';
F=[features_pos; features_neg]';
%%%%% YOUR CODE HERE FOR SVMTRAIN!
%X = [features_pos',features_neg'];
%Y = [ones(size(features_pos,1),1);-ones(size(features_neg,1),1)];
lambda=0.0001;
[w b] = vl_svmtrain(F, Labels, lambda);
%% step 3. Examine learned classifier
%%%%% Don't change this step!
% You don't need to modify anything in this section. The section first
% evaluates _training_ error, which isn't ultimately what we care about,
% but it is a good sanity check. Your training error should be very low.
fprintf('Initial classifier performance on train data:\n')
confidences = [features_pos; features_neg]*w + b;
label_vector = [ones(size(features_pos,1),1); -1*ones(size(features_neg,1),1)];
[tp_rate, fp_rate, tn_rate, fn_rate] = report_accuracy( confidences, label_vector );
% Visualize how well separated the positive and negative examples are at
% training time. Sometimes this can idenfity odd biases in your training
% data, especially if you're trying hard negative mining. This
% visualization won't be very meaningful with the placeholder starter code.
non_face_confs = confidences( label_vector < 0);
face_confs = confidences( label_vector > 0);
figure(2);
plot(sort(face_confs), 'g'); hold on
plot(sort(non_face_confs),'r');
plot([0 size(non_face_confs,1)], [0 0], 'b');
hold off;
% Visualize the learned detector. This would be a good thing to include in
% your writeup!
n_hog_cells = sqrt(length(w) / 31); %specific to default HoG parameters
imhog = vl_hog('render', single(reshape(w, [n_hog_cells n_hog_cells 31])), 'verbose') ;
figure(3); imagesc(imhog) ; colormap gray; set(3, 'Color', [.988, .988, .988])
pause(0.1) %let's ui rendering catch up
hog_template_image = frame2im(getframe(3));
% getframe() is unreliable. Depending on the rendering settings, it will
% grab foreground windows instead of the figure in question. It could also
% return a partial image.
imwrite(hog_template_image, 'visualizations/hog_template.png')
%% Step 4. Run detector on test set.
% YOU CODE 'run_detector'. Make sure the outputs are properly structured!
% They will be interpreted in Step 6 to evaluate and visualize your
% results. See run_detector.m for more details.
%%%% JZ: Please open run_detector.m file and complete it.
[bboxes, confidences, image_ids] = run_detector(test_scn_path, w, b, feature_params);
% run_detector will have (at least) two parameters which can heavily
% influence performance -- how much to rescale each step of your multiscale
% detector, and the threshold for a detection. If your recall rate is low
% and your detector still has high precision at its highest recall point,
% you can improve your average precision by reducing the threshold for a
% positive detection.
%% Step 5. Evaluate and Visualize detections
% These functions require ground truth annotations, and thus can only be
% run on the CMU+MIT face test set. Use visualize_detectoins_by_image_no_gt
% for testing on extra images (it is commented out below).
% Don't modify anything in 'evaluate_detections'!
[gt_ids, gt_bboxes, gt_isclaimed, tp, fp, duplicate_detections] = ...
evaluate_detections(bboxes, confidences, image_ids, label_path);
visualize_detections_by_image(bboxes, confidences, image_ids, tp, fp, test_scn_path, label_path)
% visualize_detections_by_image_no_gt(bboxes, confidences, image_ids, test_scn_path)
% visualize_detections_by_confidence(bboxes, confidences, image_ids, test_scn_path, label_path);
% performance to aim for
% random (stater code) 0.001 AP
% single scale ~ 0.2 to 0.4 AP
% multiscale, 6 pixel step ~ 0.83 AP
% multiscale, 4 pixel step ~ 0.89 AP
% multiscale, 3 pixel step ~ 0.92 AP
Positive Feature:
% Starter code prepared by James Hays, Brown University
% This function should return all positive training examples (faces) from
% 36x36 images in 'train_path_pos'. Each face should be converted into a
% HoG template according to 'feature_params'. For improved performance, try
% mirroring or warping the positive training examples.
function features_pos = get_positive_features(train_path_pos, feature_params)
% 'train_path_pos' is a string. This directory contains 36x36 images of
% faces
% 'feature_params' is a struct, with fields
% feature_params.template_size (probably 36), the number of pixels
% spanned by each train / test template and
% feature_params.hog_cell_size (default 6), the number of pixels in each
% HoG cell. template size should be evenly divisible by hog_cell_size.
% Smaller HoG cell sizes tend to work better, but they make things
% slower because the feature dimensionality increases and more
% importantly the step size of the classifier decreases at test time.
% 'features_pos' is N by D matrix where N is the number of faces and D
% is the template dimensionality, which would be
% (feature_params.template_size / feature_params.hog_cell_size)^2 * 31
% if you're using the default vl_hog parameters
% Useful functions:
% vl_hog, HOG = VL_HOG(IM, CELLSIZE)
% http://www.vlfeat.org/matlab/vl_hog.html (API)
% http://www.vlfeat.org/overview/hog.html (Tutorial)
% rgb2gray
image_files = dir( fullfile(train_path_pos, '*.jpg') ); %Caltech Faces stored as .jpg
num_images = length(image_files); % number of images in the dataset
idx=0;
for i=1:num_images
im=imread([train_path_pos '/' image_files(i).name]);
%%%%% please follow the steps below to compute HOG features for each image.
%%%%% 1. use single() function to convert input image to SINGLE class;
%%%%% 2. call vl_hog() function, two parameters: one is the image, the
%%%%% other is the feature_parames.hog_cell_size;
%%%%% 3. use reshape() function to convert the output of vl_hog() to a
%%%%% row vector;
%%%%% 4. add the row vector to features_pos and increase idx by 1;
%%%%% 5. use fliplr() function to flip the input image and repeat steps
%%%%% 2, 3, 4;
%%%%% YOUR CODE HERE!
im=single(im);
hog = vl_hog(im, feature_parames.hog_cell_size, 'verbose') ;
%idx=idx+1;
features_pos(idx,:)=reshape(hog,1,[]);
idx=idx+1;
im=fliplr(im);
hog = vl_hog(im, feature_parames.hog_cell_size, 'verbose') ;
features_pos(idx,:)=reshape(hog,1,[]);
end
Negative:
% Starter code prepared by James Hays, Brown University
% This function should return negative training examples (non-faces) from
% any images in 'non_face_scn_path'. Images should be converted to
% grayscale, because the positive training data is only available in
% grayscale. For best performance, you should sample random negative
% examples at multiple scales.
function features_neg = get_random_negative_features(non_face_scn_path, feature_params, num_samples)
% 'non_face_scn_path' is a string. This directory contains many images
% which have no faces in them.
% 'feature_params' is a struct, with fields
% feature_params.template_size (probably 36), the number of pixels
% spanned by each train / test template and
% feature_params.hog_cell_size (default 6), the number of pixels in each
% HoG cell. template size should be evenly divisible by hog_cell_size.
% Smaller HoG cell sizes tend to work better, but they make things
% slower because the feature dimensionality increases and more
% importantly the step size of the classifier decreases at test time.
% 'num_samples' is the number of random negatives to be mined, it's not
% important for the function to find exactly 'num_samples' non-face
% features, e.g. you might try to sample some number from each image, but
% some images might be too small to find enough.
% 'features_neg' is N by D matrix where N is the number of non-faces and D
% is the template dimensionality, which would be
% (feature_params.template_size / feature_params.hog_cell_size)^2 * 31
% if you're using the default vl_hog parameters
% Useful functions:
% vl_hog, HOG = VL_HOG(IM, CELLSIZE)
% http://www.vlfeat.org/matlab/vl_hog.html (API)
% http://www.vlfeat.org/overview/hog.html (Tutorial)
% rgb2gray
image_files = dir(fullfile(non_face_scn_path, '*.jpg'));
num_images = length(image_files);
% placeholder to be deleted
features_neg = rand(100, (feature_params.template_size / feature_params.hog_cell_size)^2 * 31);
step=feature_params.hog_cell_size;
idx=0;
for i=1:num_images
im=imread([non_face_scn_path '/' image_files(i).name]);
%%%%% please follow the steps below to compute HOG features for each image.
%%%%% 1. use single() function to convert input image to SINGLE class;
%%%%% 2. call vl_hog() function, two parameters: one is the image, the
%%%%% other is the feature_parames.hog_cell_size;
%%%%% Your code here!
im=single(im);
hog = vl_hog(im, feature_parames.hog_cell_size, 'verbose');
for m=1:step:size(im,1)
for n=1:step:size(im,2)
%%%%% spearate the output of vl_hog() into 6*6*31 blocks,
%%%%% use reshape() to convert every block to a row vector,
%%%%% save each vector to features_neg, and increase idx by 1
%%%%% Your code here!
features_neg(idx,:)=reshape(hog, [1, 6*6*31]);
idx=idx+1
end
end
disp(['idex is: ' num2str(idx)]);
disp(['case is: ' num2str(i)]);
end
idx

Sign in to comment.

More Answers (2)

Pooyan Mobtahej
Pooyan Mobtahej on 28 Nov 2020
i have modified as follows and got error again :
idx=0;
for i=1:num_images
im=imread([non_face_scn_path '/' image_files(i).name]);
%%%%% JZ: please follow the steps below to compute HOG features for each image.
%%%%% 1. use single() function to convert input image to SINGLE class;
%%%%% 2. call vl_hog() function, two parameters: one is the image, the
%%%%% other is the feature_parames.hog_cell_size;
%%%%% JZ: Your code here!
im=single(im);
hog = vl_hog(im, feature_params.hog_cell_size);
for m=1:step:size(im,1)
for n=1:step:size(im,2)
%%%%% JZ: spearate the output of vl_hog() into 6*6*31 blocks,
%%%%% use reshape() to convert every block to a row vector,
%%%%% save each vector to features_neg, and increase idx by 1
%%%%% JZ: Your code here!
idx=idx+1
% feature_neg(idx,:)=reshape(hog,6,6,[]);
% features_neg(idx,:) = hog(:);
feature_neg(idx,:) = reshape(hog(m-step+1:m, n-step+1:n, :),[],1);
%idx=idx+1
  1 Comment
Walter Roberson
Walter Roberson on 28 Nov 2020
Your m starts at 1 and your n starts at 1. Suppose step is 6. Then you try to index hog as hog(1-6+1:1, 1-6+1:1, :) which would be hog(-4:1, -4:1, :) which would use invalid negative indices.
I told you to use + not - and the order was reversed hog(m:m+step-1, n:n+step-1, :) which would be hog(1:1+6-1, 1:1+6-1,:) which would be hog(1:6, 1:6, :)
for m=1:step:size(im,1)
You need to stop at the point where m+step-1 does not exceed size(im,1) .
m+step-1 <= size(im,1)
so m <= size(im,1) - step + 1

Sign in to comment.


Pooyan Mobtahej
Pooyan Mobtahej on 28 Nov 2020
I revised that
im=single(im);
hog = vl_hog(im, feature_params.hog_cell_size);
for m=1:step:size(im,1)
for n=1:step:size(im,2)
%%%%% JZ: spearate the output of vl_hog() into 6*6*31 blocks,
%%%%% use reshape() to convert every block to a row vector,
%%%%% save each vector to features_neg, and increase idx by 1
%%%%% JZ: Your code here!
idx=idx+1
% feature_neg(idx,:)=reshape(hog,6,6,[]);
% features_neg(idx,:) = hog(:);
feature_neg(idx,:) = reshape(hog(m:m+step-1, n:n+step-1, :),[],1);
%idx=idx+1
end
end
disp(['idex is: ' num2str(idx)]);
disp(['case is: ' num2str(i)]);
end
idx
still this error:
Index in position 2 exceeds array bounds (must not exceed 43).
Error in get_random_negative_features (line 66)
feature_neg(idx,:) = reshape(hog(m:m+step-1, n:n+step-1, :),[],1);
Error in project (line 71)
features_neg = get_random_negative_features( non_face_scn_path, feature_params, num_negative_examples);
  2 Comments
Walter Roberson
Walter Roberson on 28 Nov 2020
As I wrote,
You need to stop at the point where m+step-1 does not exceed size(im,1) .
And I posted the logic that shows you how to calculate the upper bound.
Pooyan Mobtahej
Pooyan Mobtahej on 29 Nov 2020
can once again explain how to :
stop at the point where m+step-1 does not exceed size(im,1) .
should I modify loop? or anything in reshape part?

Sign in to comment.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!