geodetic2aer elevation and range calculation

Hi,

I'd like to calculate an elevation and a range from a known point on earth to a plane flying at distanse 200Km.

Our calculation using following formulas:

% p1.alt, p2.alt - highth above earth + earth radius (6378138 m)
% assumtoin is that earth is sphere with same radius.

alpha=acos(sin(p1.lat)*sin(p2.lat)+cos(p1.lat)*cos(p2.lat)*cos(p2.long-p1.long))  % Earth Central Angle
SlantRange=sqrt(p1.alt.^2 + p2.alt.^2-2*p1.alt*p2.alt*cos(alpha))  %cosine law.
elevation=acos(p1.alt.^2 + SlantRange.^2 -p2.alt.^2)/(2*SlantRange*p1.alt)

Plotting the 2 graphs I can see that the results are same more or less, but when doing the zoom-in, while a plane is at 120Km distanse (for example), I can see that there is diffrenece between my code and result of geodetic2aer by ~500m and elevation of 1 degree!

Digging into the code of geodetic2aer, I can see that some part of the geodetic2aer is using a Pythagorean theorem, but it is also uses semimahor axis and semiminor axis valeus.

So my question is - is there some assumtions which are taken into account for the geodetic2aer?

Which is more correct? (Myabe none of the 2 is correct and needs to use other formula to get the exact value).

Answer 1

[Greg]

Hi Michael, Thank you for your answer, geodetic2aer uses a reference ellipsoid, normally WGS84, whereas your calculation assumes a spherical Earth. These models differ in Earth radius, geodetic versus geocentric latitude, and the definition of the local vertical, so differences in elevation and range are expected—especially over long distances.

There is also a parenthesis error in the spherical elevation calculation. The argument of acos should include the entire quotient:

alpha = acos(sind(lat1)*sind(lat2) + cosd(lat1)*cosd(lat2)*cosd(lon2-lon1));
r1 = R + h1;
r2 = R + h2;

slantRange = sqrt(r1^2 + r2^2 - 2*r1*r2*cos(alpha));

theta = acos((r1^2 + slantRange^2 - r2^2) / (2*r1*slantRange));
elevation = theta - pi/2;

Here, lat1, lat2, lon1, and lon2 are in degrees when using sind and cosd, and h1 and h2 must use the same units as R. For consistency with geodetic2aer, use the same ellipsoid and compare results using WGS84 rather than a spherical model.