Returning an array of colors from a double image

I am trying to write a function that takes a type double image as input and returns an array of the colors in that image. The returned colors are supposed to be in a matrix form. The colors in my existing image are red, green, blue, white, and yellow. I can't get my head around this. Any suggestions?

[Voss]

colors = unique(reshape(im,[],3),'rows');

where im is your double image.

[DGM]

See also:

https://www.mathworks.com/matlabcentral/answers/517876-colour-pattern-reading-using-only-lab-or-hsv

https://www.mathworks.com/matlabcentral/answers/454146-how-to-detect-each-color-square-of-an-image-to-crop-as-individual-images

Answer 1

[Voss]

% create an image with colors r,g,b,w,y
im = ones(2,4,3);
im(1,1,:) = [1 0 0];
im(1,3,:) = [0 1 0];
im(2,2,:) = [0 0 1];
im(2,4,:) = [1 1 0];
imshow(im);

% get the set of unique colors in the image:
colors = unique(reshape(im,[],3),'rows')

colors = 5×3

0 0 1 0 1 0 1 0 0 1 1 0 1 1 1

Answer 2

[Image Analyst]

rgbImage = imread('peppers.png');
% Call the function:
colors = GetUniqueColors(rgbImage)

% Define the function:
function colors = GetUniqueColors(rgbImage)
[r, g, b] = imsplit(rgbImage);
colors = unique([r(:), g(:), b(:)], "rows")
end

[Mark]

This only returns rgb, how can I retrun yellow and white also ?

[Image Analyst]

All colors have 3 components: R, G, and B.

For pure yellow R would be 255, G would be 255, and G would be zero.

White would have all 3 components be 255.

Or if the image is double replace 255 by 1.

If yellow and white were actually in your image, my code would definitely return them. Are you sure they are there? Attach your image.

Your question said it would take an existing image and return the colors in an array, which is what I did. If you want, upload your existing test image so I can run the function on it and show you that it does return white and yellow.

[Image Analyst]

The image has significant noise in it. Do you want the noise colors to be different than the noise-free colors. For example if there is a yellow pixel that has values [1,1,0] and another one that is [0.99,0.98, 0.01] (which is also yellow but a slightly different shade of yellow due to the noise) do you consider that to be a different color or not?

Do you know how many colors to expect in the image and what their ranges are? If so you could use kmeans to tell it there are, say, 6 color classes. Demo is attached. Or use rgb2ind().

Do you want to denoise the image first?

Will all your images be nearly perfect computer graphics like this, with a few pure colors, or will you have any general photos, like a vacation snapshot, that have thousands of colors?

[Mark]

yes I need to denoise the image first , that I know of , and yes all my images are like this one with just these colours . I am just trying to return a matrix of all the colors present by writing a function to extract the array of colors in my images, which are red,green,blue, white and yellow. The output should be somehow like this [[ g y y w],[b w g g],.........] and so on

[Image Analyst]

I'm not sure what to analyze. If you know in advance the colors are r,g,b,y,w, and black, why do we need to find them from the denoised image? If all your images have all those colors, then we know what they will be.

And I don't understand "[[ g y y w],[b w g g],.........]". Do you know that the colors will be one of the 6 but you need to find out which one of the 6 colors is inside each of the 16 tiles? So essentially you don't really want to know what colors are present but you want to know which pure reference color is in each tile? If so, find the tiles, and find the color difference between the tile's mean color and each of your 6 reference colors and the color with the smallest color difference is the color that that tile is. Pretty much like @DGM did, though if you didn't have such a perfect computer graphic image, but one that might be an actual camera image that might possibly be rotated slightly, the labeled image would have to be reordered.

[DGM]

Yeah, I totally ignored the task of registration. That's what the fiducials are for, but I figured registration can be a separate problem. That, and I'm pretty sure there are more robust ways than just relying on the label image to fall into the correct order.

[Image Analyst]

What I do in this case is to get the tile centroids and then use kmeans on the x and y centroids separately. Then sort the centroids in the order you want and then reassign the label numbers in the labeled image. I could make up a demo - it seems like it's a fairly common need.

[DGM]

I was thinking to generate a set of tile centers with respect to the fiducials and then when presented with a new image, transform those coordinates to get the new ordered locations of the tile centers. That would allow +/- 45 deg of misalignment before things get ambiguous.

[Mark]

The output matrix should be in the form of strings that is what i mean by [green, yellow, white, blue,.....], so the output array should be an array of the colors in the picture , I guess using Matlab cell array would be the best solution to store that , but I am not sure how to return the output of the colors in my image as strings . so the problem here is to write a function which detects colors and returns them in a matlab cell array as strings of the colors.

[DGM]

That's what my second example does.

[Image Analyst]

I think @DGM means to replace

colornames = {'w','r','g','b','y'};

by

colornames = {"white", "red", "gr","blue", "yellow"};

Answer 3

[DGM]

You can leverage rgb2ind()'s minimum variance quantization to get a best-fit color table of specified length.

A = imread('https://www.mathworks.com/matlabcentral/answers/uploaded_files/917239/image.png');
[~,CT] = rgb2ind(A,6) % get a color table of at most 6 colors

CT = 6×3

0.0392 0.0392 0.0392 0.2039 0.1686 0.9569 0.9569 0.0549 0.0392 0.8471 0.9569 0.0588 0.9569 0.9569 0.9569 0.0863 0.9569 0.2235

Bear in mind that since these colors were originally close to the extremes of the data range, truncation means that the addition of zero-mean gaussian noise will indeed shift the mean colors of the image, even if the noise mean is zero. I should point out that it's pretty clear the blue, green and yellow patches weren't on their corners to begin with.

If you know that you only want primary + secondary + neutral colors, you can just round the result.

CTrounded = round(CT)

CTrounded = 6×3

0 0 0 0 0 1 1 0 0 1 1 0 1 1 1 0 1 0

Otherwise, you can try to renormalize the values to correct for the inward shift caused by the noise. This assumes that the colors in the image nominally spanned the data range before the noise was added.

CTnormalized = mat2gray(CT)

CTnormalized = 6×3

0 0 0 0.1795 0.1410 1.0000 1.0000 0.0171 0 0.8803 1.0000 0.0214 1.0000 1.0000 1.0000 0.0513 1.0000 0.2009

Answer 4

[DGM]

Oh okay I totally misunderstood the question. Round 2:

A = imread('patchchart.png');

patchmask = rgb2gray(A)>40;
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
patchmask = imclearborder(patchmask); % get rid of outer white region
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)

% segment the image
[L N] = bwlabel(patchmask);

% get average color in each mask region
patchcolors = zeros(N,3);
for p = 1:N % step through patches
    patchmk = L==p;
    Apatch = A(patchmk(:,:,[1 1 1]));
    patchcolors(p,:) = mean(reshape(Apatch,[],3),1);
end
patchcolors = patchcolors./255; % normalize

% specify a correlated list of colors and color names
colornames = {'w','r','g','b','y'};
colorrefs = [1 1 1; 1 0 0; 0 1 0; 0 0 1; 1 1 0];

% find color distances in RGB
D = patchcolors - permute(colorrefs,[3 2 1]);
D = squeeze(sum(D.^2,2));

% find index of closest match for each patch
[~,idx] = min(D,[],2);

% look up color names
patchnames = reshape(colornames(idx),4,4)

patchnames = 4×4 cell array
    {'b'}    {'y'}    {'w'}    {'y'}
    {'y'}    {'w'}    {'w'}    {'r'}
    {'w'}    {'y'}    {'r'}    {'r'}
    {'g'}    {'w'}    {'w'}    {'r'}

Alternatively, instead of doing the distance minimization the long way, you could just use rgb2ind() to do that work:

% find index of closest match for each patch
idx = rgb2ind(permute(patchcolors,[1 3 2]),colorrefs) + 1;

% look up color names
patchnames = reshape(colornames(idx),4,4)

patchnames = 4×4 cell array
    {'b'}    {'y'}    {'w'}    {'y'}
    {'y'}    {'w'}    {'w'}    {'r'}
    {'w'}    {'y'}    {'r'}    {'r'}
    {'g'}    {'w'}    {'w'}    {'r'}

[Mark]

can I have the same output but with a type double image not a unit8 , Like is it applicable to have the same result if my image is of type double ? because i tried to the same thing you are doing on a type double image but it is not returning the same results for this image , although i put a threshold on the patchmask=rgb2grey(A)>0.15625, any suggestions on how to solve that?

[DGM]

Assuming that your double image is unit-scale (0-1), all you have to do is adjust the threshold and remove the denormalization line.

A = imread('patchchart.png');
A = im2double(A); % <- UNIT-SCALE DOUBLE

patchmask = rgb2gray(A)>0.16; % <- NOTE THRESHOLD 
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
patchmask = imclearborder(patchmask); % get rid of outer white region
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)

% segment the image
[L N] = bwlabel(patchmask);

% get average color in each mask region
patchcolors = zeros(N,3);
for p = 1:N % step through patches
    patchmk = L==p;
    Apatch = A(patchmk(:,:,[1 1 1]));
    patchcolors(p,:) = mean(reshape(Apatch,[],3),1);
end
% patchcolors = patchcolors./255 <- THIS IS NOT NEEDED

% specify a correlated list of colors and color names
colornames = {'w','r','g','b','y'};
colorrefs = [1 1 1; 1 0 0; 0 1 0; 0 0 1; 1 1 0];

% find index of closest match for each patch
idx = rgb2ind(permute(patchcolors,[1 3 2]),colorrefs) + 1;

% look up color names
patchnames = reshape(colornames(idx),4,4)

[Mark]

Thanks alot it works, but now I am trying to use it with a noisy image of type double and giving me the following error :

Error using reshape

Number of elements must not change. Use [] as one of the size inputs to automatically calculate the appropriate size for that dimension.

Error in findColours (line 23)

colours = reshape(colornames(idx),4,4);

Any Suggestions on why is that happening?

[DGM]

More than likely the segmentation is failing to isolate the blocks from each other, resulting in there being fewer than 16 detected objects. Alternatively, there may be spurious objects which haven't been removed, resulting in more than 16 detected objects. Things that might help:

• adjusting the threshold value
• adjusting the blob sizes in the calls to bwareaopen()
• preprocessing with a median filter

An example of the new noisier image would help know what needs to be changed.

[Image Analyst]

Are your images always aligned/registered so that you can use a fixed mask. That would be easiest. Or do your squares move around so they're in a different location every time.

And like @DGM said (and you should know by now), attach the image you're having trouble with. It's hard to diagnose why you're not finding exactly 16 patches without the image.

[Mark]

Here is an example of a noisy image I am trying to apply this code to , it gave me the error mentioned above.

[DGM]

Here's this. I had to make the patch ordering a bit more robust in order to get them in the right order. This still isn't enough to keep them in order if there's any significant rotation, but it's enough if the patch sizes vary a bit.

I also didn't use rgb2ind() here. There's something screwy going on internally that's making it not want to process the color table correctly as a whole. The results it gives appear to be dependent on the table order, regardless of dithering options. I don't recall how it does the quantization, but I don't think that really matters since the given method works fine.

A = imread('patchchart2.png');
A = medfilt3(A,[7 7 1]); % median filter to suppress noise
A = imadjust(A,stretchlim(A,0.05)); % increase contrast

patchmask = rgb2gray(A)>40;
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
patchmask = imclearborder(patchmask); % get rid of outer white region
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)

% segment the image
[L N] = bwlabel(patchmask);

% get average color in each mask region
patchcolors = zeros(N,3);
for p = 1:N % step through patches
    patchmk = L==p;
    Apatch = A(patchmk(:,:,[1 1 1]));
    patchcolors(p,:) = mean(reshape(Apatch,[],3),1);
end
patchcolors = patchcolors./255;

% try to snap the centers to a grid
S = regionprops(patchmask,'centroid');
C = vertcat(S.Centroid);
climits = [min(C,[],1); max(C,[],1)];
C = round((C-climits(1,:))./range(climits,1)*3 + 1);

% reorder color samples
idx = sub2ind([4 4],C(:,2),C(:,1));
patchcolors(idx,:) = patchcolors;

% specify a correlated list of colors and color names
colornames = {'w','r','g','b','y'};
colorrefs = [1 1 1; 1 0 0; 0 1 0; 0 0 1; 1 1 0];

% find color distances in RGB
D = patchcolors - permute(colorrefs,[3 2 1]);
D = squeeze(sum(D.^2,2));

% find index of closest match for each patch
[~,idx] = min(D,[],2);

% look up color names
patchnames = reshape(colornames(idx),4,4)

patchnames = 4×4 cell array
    {'b'}    {'y'}    {'y'}    {'b'}
    {'w'}    {'r'}    {'y'}    {'g'}
    {'r'}    {'y'}    {'y'}    {'b'}
    {'g'}    {'y'}    {'w'}    {'r'}

[Mark]

is there a way to automatically set the thresholds, because i have multiple noisy images and I am trying to use this code on different images but it won't return all the colors right, I think it might be a problem with colour thresholding. for example in this image it miss read alot of colours.

[DGM]

I don't know what to think of this noise. It doesn't look like zero-mean noise, considering how strongly it tends to raise values and how infrequently it lowers them. It also doesn't look like a uniform noise field, considering for example how it strongly raises blue in the yellow and green regions, but not in the red regions.

Considering this seemingly asymmetric behavior, I decided to add an erosion pass (a local min() filter) to try to suppress the noise. Consequently, I adjusted the threshold lower.

A = imread('patchchart3.png');

A = imerode(A,ones(5));
A = medfilt3(A,[11 11 1]);
A = imadjust(A,stretchlim(A,0.05));

patchmask = rgb2gray(A)>20;
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
patchmask = imclearborder(patchmask); % get rid of outer white region
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)
% ... and so on

The above code works with all three images so far.

[Mohsin Ali]

can you find the colors in this image and if not so why?

[DGM]

The added noise is one thing, but without the black border, the masking approach needs to be changed anyway.

A = imread('patchchart4.png');

A = imerode(A,ones(3));
Ay = medfilt3(A,[101 1 1]); % use a long median filter
Ax = medfilt3(A,[1 101 1]); % both directions
A = max(Ax,Ay); % combine to keep patches from getting connected
A = imadjust(A,stretchlim(A,0.05));

patchmask = rgb2gray(A)<230;
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
oamask = imdilate(patchmask,ones(10)); % dilate to merge patches
oamask = bwareafilt(oamask,1); % select largest blob
patchmask = patchmask & oamask; % exclude fiducials and any border noise
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)

% segment the image
[L N] = bwlabel(patchmask);

% get average color in each mask region
patchcolors = zeros(N,3);
for p = 1:N % step through patches
    patchmk = L==p;
    Apatch = A(patchmk(:,:,[1 1 1]));
    patchcolors(p,:) = mean(reshape(Apatch,[],3),1);
end
patchcolors = patchcolors./255;

% try to snap the centers to a grid
S = regionprops(patchmask,'centroid');
C = vertcat(S.Centroid);
climits = [min(C,[],1); max(C,[],1)];
C = round((C-climits(1,:))./range(climits,1)*3 + 1);

% reorder color samples
idx = sub2ind([4 4],C(:,2),C(:,1));
patchcolors(idx,:) = patchcolors;

% specify a correlated list of colors and color names
colornames = {'w','r','g','b','y'};
colorrefs = [1 1 1; 1 0 0; 0 1 0; 0 0 1; 1 1 0];

% find color distances in RGB
D = patchcolors - permute(colorrefs,[3 2 1]);
D = squeeze(sum(D.^2,2));

% find index of closest match for each patch
[~,idx] = min(D,[],2);

% look up color names
patchnames = reshape(colornames(idx),4,4);

% alternatively:
% find index of closest match for each patch
idx = rgb2ind(permute(patchcolors,[1 3 2]),colorrefs,'nodither') + 1;

% look up color names
patchnames = reshape(colornames(idx),4,4)

patchnames = 4×4 cell array
    {'b'}    {'y'}    {'g'}    {'r'}
    {'g'}    {'r'}    {'r'}    {'b'}
    {'r'}    {'r'}    {'g'}    {'r'}
    {'b'}    {'y'}    {'y'}    {'y'}

imshow(A)

[EU]

Can you pls help me figure out why am i not getting an accurate list of colour as the image @DGM, @Image Analyst I tried to implement your code on an image, it worked but wasn't returning the right list of colour or colour names in the 4x4 array.

A = imread('reel.png');

A = imerode(A,ones(5));

A = medfilt3(A,[11 11 1]);

A = imadjust(A,stretchlim(A,0.05));

imshow(A)

Unrecognized function or variable 'A'.

[DGM]

I posted several versions of code, so I don't know which one you tried, and you didn't post your image, so I don't know what you tried it on.

[kiana]

Hello

thanks for the codes and explanation

I have a problem similar to this one

I tried this code for normal photos and it works but with photos with noise it does not work well

For Denoise I use

A = imerode(A,ones(3));

Ay = medfilt3(A,[101 1 1]); % use a long median filter

Ax = medfilt3(A,[1 101 1]); % both directions

A = max(Ax,Ay); % combine to keep patches from getting connected

A = imadjust(A,stretchlim(A,0.05));

patchmask = rgb2gray(A)<230;

patchmask = bwareaopen(patchmask,100); % remove positive specks

patchmask = ~bwareaopen(~patchmask,100); % remove negative specks

oamask = imdilate(patchmask,ones(10)); % dilate to merge patches

oamask = bwareafilt(oamask,1); % select largest blob

patchmask = patchmask & oamask; % exclude fiducials and any border noise

patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

filter as you sent but it does not solve the problem

i upload the result and my photo

can you help me to solve this issue ?

[DGM]

That example was intended for a type of image without a black border. The prior versions may work, but for this example, I'm going to stop doing everything strictly with base/IPT tools.

A = imread('noise_1.png');

% A = amedfilt(A,5,1); % more robust against JPG-compressed impulse noise
A = fmedfiltforum(A,5); % fixed median noise reduction filter (get rid of SNP noise)

A = imerode(A,ones(5)); % this may no longer be strictly necessary, but it's safer
A = medfilt3(A,[11 11 1]); % this is not ideal, but it's succinct
A = imadjust(A,stretchlim(A,0.05));

patchmask = rgb2gray(A)>20; % threshold
patchmask = bwareaopen(patchmask,100); % remove positive specks
patchmask = ~bwareaopen(~patchmask,100); % remove negative specks
patchmask = imclearborder(patchmask); % get rid of outer white region
patchmask = imerode(patchmask,ones(10)); % erode to exclude edge effects

imshow(patchmask)

% segment the image
[L N] = bwlabel(patchmask);

% get average color in each mask region
patchcolors = zeros(N,3);
for p = 1:N % step through patches
    patchmk = L==p;
    Apatch = A(patchmk(:,:,[1 1 1]));
    patchcolors(p,:) = mean(reshape(Apatch,[],3),1);
end
patchcolors = patchcolors./255;

% try to snap the centers to a grid
S = regionprops(patchmask,'centroid');
C = vertcat(S.Centroid);
climits = [min(C,[],1); max(C,[],1)];
C = round((C-climits(1,:))./range(climits,1)*3 + 1);

% reorder color samples
idx = sub2ind([4 4],C(:,2),C(:,1));
patchcolors(idx,:) = patchcolors;

% specify a correlated list of colors and color names
colornames = {'w','r','g','b','y'};
colorrefs = [1 1 1; 1 0 0; 0 1 0; 0 0 1; 1 1 0];

% find color distances in RGB
D = patchcolors - permute(colorrefs,[3 2 1]);
D = squeeze(sum(D.^2,2));

% find index of closest match for each patch
[~,idx] = min(D,[],2);

% look up color names
patchnames = reshape(colornames(idx),4,4)

patchnames = 4x4 cell array
    {'b'}    {'w'}    {'r'}    {'g'}
    {'w'}    {'b'}    {'r'}    {'b'}
    {'b'}    {'r'}    {'b'}    {'y'}
    {'g'}    {'y'}    {'g'}    {'g'}

Doing an actual noise-removal median filter instead of trying to suppress everything with just simple median filters and morphological operations should be a lot more robust against impulse noise. The attached file fmedfiltforum() is a slower, version-dependent copy of MIMT fmedfilt(). It doesn't require MIMT, but it does require IPT and R2019b or newer. MIMT amedfilt() would potentially be quite a bit more robust (especially if the image is compressed) but it's slower, and probably overkill.

We have two types of images: ones with a contiguous white background, and ones with a black frame around the tiles. Consequently, I'm using two different ways to create the mask. Is it possible to write one function/script which can handle both types of images? What if we use saturation to find the foreground instead of looking at luma alone? Maybe, but relying on chroma/saturation will be problematic for two reasons. FIrst, the existence of white tiles means that you're going to have to extrapolate the location of tiles which the segmentation will miss. Second, the use of any JPG compression will probably ruin saturation data and risk creating extra problems.

Is there a better way? Probably. I'm just adapting ad-hoc scripts as people keep giving me slightly different versions. I don't have an understanding of the full scope of the problem requirements. If the images are grossly transformed or changed in other unexpected ways, the given examples will probably break.