How can I determine if my data is part of one population or another?

2 views (last 30 days)
pop1=randn(1000,1);
>> pop2=randn(9000,1)-1;
>> pop=[pop1;pop2];
>> k=randsample(pop,500);
How do I go from here to put the elements of k that are originally found in pop1 into their own matrix?

Accepted Answer

John BG
John BG on 17 Aug 2017
Hi Mr Anzulis
since not yet sure the sampling is needed, let's start determining common elements
1.
Simplified version of your input data
pop1=randi([-10 10],5,1)
=
-6
-5
2
-1
-3
pop2=randi([-10 10],10,1)-1
=
6
1
0
8
-5
4
4
-4
0
-10
2.
Merged data
pop=[pop1;pop2]
=
-6
-5
2
-1
-3
6
1
0
8
-5
4
4
-4
0
-10
3.
what elements of pop1 are in pop, a their locations
[pop1_in_pop locs_in_pop locs_in_pop1]=intersect(pop,pop1)
pop1_in_pop =
-6
-5
-3
-1
2
locs_in_pop =
1
2
5
4
3
locs_in_pop1 =
1
2
5
4
3
checking indices match
pop(locs_in_pop)
=
-6
-5
-3
-1
2
pop1(locs_in_pop1)
=
-6
-5
-3
-1
2
Same for pop and pop2
4.
[pop2_in_pop locs_in_pop locs_in_pop2]=intersect(pop,pop2)
pop2_in_pop =
-10
-5
-4
0
1
4
6
8
locs_in_pop =
15
2
13
8
7
11
6
9
locs_in_pop2 =
10
5
8
3
2
6
1
4
Now you know exactly
  • what elements of pop1 are in pop
  • what elements of pop2 are in pop
  • what elements of pop are in pop2
  • what elements of pop are in pop2
And you also know all their respective indices.
5.
to randomly sample, you can use command randsample
length_sample=3
sample1=randsample([1:1:length(pop)],length_sample)
sample1 =
8 12 1
if you find this answer useful would you please be so kind to consider marking my answer as Accepted Answer?
To any other reader, if you find this answer useful please consider clicking on the thumbs-up vote link
thanks in advance
John BG

More Answers (3)

Jan
Jan on 14 Aug 2017
Edited: Jan on 14 Aug 2017
pop1 = randn(1000,1);
pop2 = randn(9000,1)-1;
pop = [pop1;pop2];
k = randperm(pop, 500); % or: randsample(numel(pop), 500))
pop1_k = pop(k(k <= numel(pop1)));
pop2_k = pop(k(k > numel(pop1)));
You obtain the indices of the random samples instead of the values, and then you can check easily, if they belong to the first part of [pop1; pop2] or not.
Perhaps you mean another problem: All you have is the vector of random samples. Then it is hard to predict, if e.g. a 1.0 belong to the randn or the randn-1 group. Both is possible. The identification of the 2 clusters without any further knowledge is hard, I do not have enough experiences for this. Do you know, that both are normal distributions and have a distance of 1.0?
Not that there is a chance to select only values from pop2. Then you cant recognize, if there is a 2nd cluster.

Image Analyst
Image Analyst on 14 Aug 2017
Simply call randsample() on pop1, not pop. Why did you call randsample() on pop when you knew in advance that you wanted elements strictly from pop1, and you still have the pop1 variable to use in calling randsample()???
  2 Comments
Robyn Anzulis
Robyn Anzulis on 14 Aug 2017
I want to take a random sample from all of the data, but I want to find out which of the individual elements in k were originally from pop1 and pop2. However, I'm not sure that this is a calculation that I need at this point.

Sign in to comment.


Walter Roberson
Walter Roberson on 14 Aug 2017
Edited: Walter Roberson on 14 Aug 2017
If you have two non-overlapping distributions, then you can examine the compare the individual values to the known ranges of the two distributions to determine which populations they are from.
If you have two overlapping distributions, you can never be sure which distribution any given element is from. However, if the two distributions are not identical, you can calculate zscores to figure out which distribution the item has the higher probability of belonging to. If the zscores are "close" then this would not be at all powerful.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!