Using find to check the contents of files within a struct

Hello, I am working on a project in which I have a 5122x1 struct containing files with latitude and longitude data. I access the files in the struct with a for loop. ie)

latlon_data = 'C:\Users\bbush\Desktop\Matlab Code and Plots\Microwave Files\20190407\GATMO'; %Need to change dated folder to the correct day
latlonfn = dir(fullfile(latlon_data,'*_*_*_*_*_*_*_*_*.h5'));    % return the list of all files

for i = 1:length(latlonfn)
    %lat/lon and bt variables with all of the unmatched files    
    lat = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Latitude'));
    lon = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Longitude'));
    
    emptyIndex = find(latlonfn(i).name((lat >= 20 & lat <= 60) && (lon >= -128 & lon <= -65)));
end   

These files contain satellite swaths that cover the entire planet but I need to be able to figure out which ones are only over the U.S. I am having trouble implementing the find function to to search through the struct and look at the lat and lon points. I need to return the index of files within the struct that have lat and lon points inside the given ranges.

I have tried using double && inside the comparisons as well but that didnt work, I'm just not sure how to set up the find command

[Brandon Bush]

[Edit] The lat and lon files are 96x12 matrices. I neglected some important info, but I have included it. The individual file represents a arc like band stretching from west to east. So each file has a 96x12 lat and lon file that represents the points within the band

[Guillaume]

Note: strictly speaking your structure does not contain files. It contains, among other things, filenames. The files are stored on the hard drive.

As Jon has pointed out your find syntax is completely wrong, and I'm not sure what indices you're actually wanting. The indices of the elements of the structure (i.e filenames) for which the matrices lat and lon have some (or is all?) coordinates within your band? Or the indices of lat and lon which are within the band for each element of the structure? Or both?

Also, isn't there a function in the mapping toolbox for finding if a set of coordinates is within a country. That would be better than assuming the US is rectangular (or whatever the projection of a rectangle on sphere is called).

[Brandon Bush]

I want to select the indices of the filenames that have lat and lon coordinates over the United States. ie) within the range of coordinates below. I have no idea how to do the search for this. i will try searching through the mapping toolbox

Answer 1

[Jon]

It looks like your difficulty is coming from the argument that you are providing to the find command.

If you look carefully at your line of code:

emptyIndex = find(latlonfn(i).name((lat >= 20 & lat <= 60) && (lon >= -128 & lon <= -65)))

you will see that that the argument that you are giving to the find function is:

latlonfn(i).name((lat >= 20 & lat <= 60) && (lon >= -128 & lon <= -65))

which looks to me as if it would evaluate to being the name of a file, which is a character array.

You want to provide an array of logicals to the find function.

Maybe you can rework your logic so that you produce an array of logicals where the criteria for lat and lon are met and then do the find on that.

If I am understanding what you are trying to do, then this might be a good start at how you might modify your code

latlon_data = 'C:\Users\bbush\Desktop\Matlab Code and Plots\Microwave Files\20190407\GATMO'; %Need to change dated folder to the correct day
latlonfn = dir(fullfile(latlon_data,'*_*_*_*_*_*_*_*_*.h5'));    % return the list of all files
% preallocate vectors to hold latitude and longitude
numFiles = length(latlonfn);
lat = zeros(numFiles,1);
lon = zeros(numFiles,1);
for i = 1:length(latlonfn)
    %lat/lon and bt variables with all of the unmatched files    
    lat(i) = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Latitude'));
    lon(i) = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Longitude'));
   
end 

% find the indices of the files which match the latitude and longitude
% requirements
idxMatch = find((lat >= 20 & lat <= 60) & (lon >= -128 & lon <= -65));

% now if you only wanted to look at files which met the criteria you could
% look just at these, for example
someFiles = latlonfn(idxMatch)


% or if you wanted just the files that didn't meet the criteria you could get the compliment
otherFiles = latlonfn(~idxMatch)

[Brandon Bush]

I neglected some important info, but I have included it. The code would work if there were only one lat and lon in each file. The individual file represents a arc like band stretching from west to east. So each file has a 96x12 lat and lon file that represents the points within the band

[Jon]

So just to help me understand better. When you say "So each file has a 96x12 lat and lon file" I think you mean So each file has associated with it a lat and a lon file. These files each contain a 96x12 array of lat and lon data respectively.

Is my understanding correct?

If so, I'm still not clear about the 96 x 12 arrays. So we have a an array called lon that is 96 x12 and each element of the array is a longitude value, then what do the rows and columns represent. In other words what does the element lon(i,j) represent.

In any case, do you want to know if every element of lon satisfies (lon >= -128 & lon <= -65)? and similarly every element of lan satisfies (lat >= 20 & lat <= 60)

or do you need to know which i and j within the 96x12 satisfy the criteria

Specifically, do you just want to know which of the 5122 files meet some criteria, or for each of the 5122 files do you need to have the row and column indices of the points that satisfy some criteria

[Brandon Bush]

So the files in the directory GATMO, as indicated in the path name, are H5 files retrieved from a satellite. Inside the H5 files, which are located in the latlonfn struct, are the lat and lon attributes, among other attributes. That's why the lat and lon variables have names like that because I need to select them specifically from each file.

The lat and lon attributes are each 96x12 arrays so yes, your understanding is correct. If it helps, I made an example plot where the bands that are spaced by the white lines are each from one of the individual files inside the latlonfn struct. So each band has a specific 96x12 lat and a 96x12 lon associated with it. There are 5122 of these bands that cover the entire planet and I need to find which ones occur over the United States. Or at least within the lon and lat values indicted by the x and y axis.

[Jon]

So if the band goes over the US then there is at least one pair of j,k values such that lat(j,k) lon(i,j) is within the US, where j is is between 1 and 96 and k is between 1 and 12?

In this case you could do something like:

latlon_data = 'C:\Users\bbush\Desktop\Matlab Code and Plots\Microwave Files\20190407\GATMO'; %Need to change dated folder to the correct day
latlonfn = dir(fullfile(latlon_data,'*_*_*_*_*_*_*_*_*.h5'));    % return the list of all files
% preallocate vectors to hold latitude and longitude
numFiles = length(latlonfn);
inUS = false(numFiles,1);
for i = 1:length(latlonfn)
    %lat/lon and bt variables with all of the unmatched files    
    lat = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Latitude'));
    lon = double(h5read(latlonfn(i).name, '/All_Data/ATMS-SDR-GEO_All/Longitude'));
    % check if any latitude or longitude is within the US
    inUS(i) = any((lat >= 20 & lat <= 60) & (lon >= -128 & lon <= -65),'all'); % 'all' tests over all elements of a matrix
end 

The variable inUS will be a length 5122 vector (assuming that is the number of files) whose elements are either 0 (false) or 1 (true). You can then use that to select the bands you want.

[Brandon Bush]

Ok, I only saw this after I posted my comment, this looks like it will do the same thing as the answer i posted. I will accept your answer so that you get the credit for it but I will also keep mine posted.

[Jon]

Thanks for that. I've also run into collisions on MATLAB answers where by the time I post something someone else has already posted a similar solution. I'm just glad you were able to get moving on your project.

Answer 2

[Brandon Bush]

Using this block of code, you can define a polygon, and then search for points inside the polygon. I used an if statement to tell me which files had lat and lon points that were inside the polygon.

in = zeros(size(latlonfn))
xv = [-128 -65]; %define the height
yv = [20 60]; %define the width

if any(inpolygon(lon,lat,xv,yv)) %determines if the lat and lon values are inside the polygon
    in(i) = 1; %If the coordinates are inside the polygone,, the 0 becomes a 1 
end

[Jon]

Glad you were able to anwer your question.

Note that you have some redundant lines of code. No need for the if statement. You could streamline that to

in(i) = any(inpolygon(lon,lat,xv,yv)) %determines if the lat and lon values are inside the polygon

Also since your polygon is just a rectangle you don't need all of the machinery of inpolygon. I would guess that it is simpler and more efficient to check it as shown in my final comment to you, with just simple <= range checks

[Brandon Bush]

Thanks alot for your help!