Precision of coordinates

From OpenStreetMap Wiki
Jump to navigation Jump to search


Definition

1 arc minute along a meridian or along the Equator (over the WGS84 reference geoid) equals 1 852 m.

1′ = 1 852 m

Precision of latitudes

DD.dddddd° Decimal places DD°MM′SS.sss″ Metres
0 111 120
0.1° 1 0°06′ 11 112
0.01° 2 0°00′36″ 1 111.2
0.001° 3 0°00′03.6" 111.12
0.000 1° 4 0°00′00,36″ 11.112
0.000 01° 5 0°00′00.04″ 1.1112
0.000 001° 6 0°00′00.004″ 0.11112
0.000 000 1° 7 0°00′00.000 4″ 0.011112
MM.m′ DD.dddddd° Metres
10′ 0.016 6° 18 520
1′ 0.001 66° 1 852
0.1′ 0.001 66° 185.2
0.01′ 0.000 166° 18.52
0.001′ 0.000 016 6° 1.852
0.000 1′ 0.000 001 66° 0.185 2
0.000 01′ 0.000 000 166° 0.018 52
SS.sss″ DD.dddddd° Metres
10″ 0.002 77° 308.6
1″ 0.000 277° 30.86
0.1″ 0.000 027 7° 3.086
0.01″ 0.000 002 77° 0.308 6
0.001″ 0.000 000 277° 0.030 86

Precision of longitudes

Latitude Cosinus
0° (equator) 1.00
10° 0.98
20° 0.94
30° 0.86
40° 0.77
50° 0.64
60° 0.50
70° 0.34
80° 0.17

The precision of longitudes is dependent on the latitude : the higher the latitude, the closer the meridians are to each other. The value in meters is to be multiplied by the cosine of the latitude (except along the equator). So, for example, in Germany (50 ° N) the meridians are about 2/3 as large as on the equator and the accuracy is correspondingly higher.

Precision of given longitude Distance along a parallel depending on latitude
DD.dddddd° Decimal places DD°MM′SS.sss″ 0° (equator) 30° 45° 60° 75°
0 111 120 m 96 233 m 78 574 m 55 560 m 28 760 m
0.1° 1 0°06′ 11 112 m 9 623 m 7 857 m 5 556 m 2 876 m
0.01° 2 0°00′36″ 1 111.2 m 962 m 785 m 555 m 288 m
0.001° 3 0°00′03.6″ 111.12 m 96 m 78 m 55 m 29 m
0.000 1° 4 0°00′00.36″ 11.112 m 9.6 m 7.8 m 5.5 m 2.9 m
0.000 01° 5 0°00′00.036″ 1.111 2 m 0.96 m 0.78 m 0.55 m 0.29 m
0.000 001° 6 0°00′00.003 6″ 0.111 12 m 0.096 m 0.078 m 0.055 m 0.028 m
0.000 000 1° 7 0°00′00.000 36″ 0.0111 12 m 0.0096 m 0.0078 m 0.0055 m 0.0028 m

Distance

The loxodromic distance between two points A and B (on the WGS84 reference geoid) may be approximated by twice the quadratic average of the two distances below if the difference of latitudes is not too important (less than about 10°):

latitudinal distance along any meridian (exact)
latitudinal distance (in metres) = (decimal latitude A - decimal latitude B) * 111 120 m
longitudinal distance along an average parallel (approximation; the highest the difference of the two latitudes, the lowest is the precision)
longitudinal distance (in metres) ≈ (decimal longitude A - decimal longitude B) * cos(average latitude) * 111 120m

In practice, most objects in OSM (including the largest ones such as coastlines and land boundaries of countries) are traced with small segments whose two end points have very near latitudes whose difference is much below 1°; if this is not the case the polygons should be improved to add missing intermediate points if arcs are not traced along a parallel or meridian (this should be done for roads).

The effective distance on highways/railways/waterways cannot be computed exactly this way, because of approximation of curves by polylines, and because of additional differences of altitude (and imprecision on the terrain data model) and the impossibility to follow exactly an averaged theoretical curve.

Conversion to decimal

Accuracy DD.dddddd°
decimal places
10 m 4
1 m 5
0.1 m 6

If coordinates in [DD°MM′SS.sss″] are converted to [DD.dddddd°] or vice versa, then sometimes a higher number of decimal places are created, which simulate a greater accuracy than what actually exists.

For conversions, it makes sense to round out the results to reflect the original accuracy.

Even better we could add a separate key for accuracy: accuracy=##.# (in metres)

Converting other datums to WGS 84

When other datums such as Gauß-Krüger coordinates are converted to coordinates in the World Geodetic System 1984 (WGS 84) , or vice-versa, then an certain amount of error is introduced depending on the formula used.

This conversion may also change the number of decimal places, but this is unrelated to the accuracy.

Display and measurement accuracy

Warning: Just because a coordinate has many decimal places does not mean that it is an accurate measurement.

If a device produces a measured coordinate with 8 decimal places, this would suggest that it has an accuracy of 1 centimetre. However, if the device is manufactured to only have an accuracy of 10 metres then the last 3 decimal places should not be considered valid. See the following table.

Related: Significant Figures on Wikipedia

Reasonable Display Accuracy

Since the number of decimal places implies measurement accuracy, the appropriate number of decimal places should be used:
(drawn on the Great circle on Wikipedia)

Measurement accuracy Display accuracy
DD.dddddd°
Nautic miles Metres
0.0006 nmi 1.111 m ##.#####°
0.006 nmi 11.11 m ##.####°
0.06 nmi 111.1 m ##.###°
0.6 nmi 1 111 m ##.##°
 
Display accuracy
DD°MM′SS.sss″
Measurement accuracy
Nautic miles Metres
##°##′##.#″ 0.0016 nmi 3.1 m
##°##′##″ 0.016 nmi 31 m
##°##.#′ 0.1 nmi 185 m
##°##′ 1 nmi 1 852 m

See also

External links