Recently, members of the MalaGIS group discussed an issue encountered during POI cleaning: the same batch of data had inconsistent decimal places for coordinates. Some were recorded to 6 decimal places, while others went to 13, such as 19.0649070739746 and 73.1308670043945. Would fewer decimal places shift the point location? Would more digits make it more accurate? This is an interesting question. I'll share some insights based on my daily experience with coordinate processing.

Precision Does Not Equal Accuracy

The number of decimal places reflects numerical precision, not measurement accuracy.

Precision is about how finely a value can be expressed, while accuracy is how close the measured result is to the true location. They are related but not the same thing. For most GIS work, 6 decimal places are usually sufficient, providing a resolution of roughly 11 centimeters. If sub-centimeter results are required, you should keep 7 to 8 decimal places. However, adding more digits beyond that offers limited benefit to real accuracy because the measuring instruments themselves generally cannot achieve that level. Coordinates written to 13 decimal places are often just un-truncated floating-point outputs and will not make ordinary GPS sampling more accurate.

As an example, an ordinary smartphone GPS collecting points among buildings may have a true position that fluctuates within a ±10 meter range, yet the coordinates can be stored with 13 decimal places. They look extremely fine, but most of those extra digits are generated by computer floating-point arithmetic. They do not improve the actual positioning quality and can easily mislead downstream users into thinking this is centimeter-level or even millimeter-level data.

What Does Each Decimal Place Represent in Ground Distance?

To understand what each decimal place means in terms of ground distance, we first need to know how long one degree is. The distance from the Earth's equator to the pole is roughly 10,000 kilometers, corresponding to 90 degrees of latitude. Therefore, 1 degree of latitude equals approximately 111 kilometers. In the longitude direction, the Earth is divided into 360 degrees as well, but the distance represented by one degree of longitude narrows toward the poles. At the equator it is longest, and it becomes shorter at higher latitudes.

The following uses the latitude direction as a reference. Longitude can be understood similarly near the equator, but for higher latitudes the cosine of the latitude must be applied.

Integer part

The sign indicates north/south or east/west. The tens digit roughly corresponds to a scale of 1,000 kilometers, helping to identify the continent or ocean region. One degree in the units place equals about 111 kilometers, allowing a rough location down to the province or national level.

Decimal part

  • The 1st decimal place (0.1°) corresponds to about 11 kilometers, useful for distinguishing neighboring large cities.
  • The 2nd decimal place (0.01°) corresponds to about 1.1 kilometers, useful for distinguishing adjacent towns.
  • The 3rd decimal place corresponds to about 110 meters, covering areas like large farmlands or a campus.
  • The 4th decimal place corresponds to about 11 meters, close to the size of a small land parcel and also close to the typical error level of an ordinary GPS without differential correction under good signal conditions.
  • The 5th decimal place corresponds to about 1.1 meters, capable of distinguishing adjacent features. Commercial GPS usually needs differential correction to achieve this level stably.
  • The 6th decimal place corresponds to about 11 centimeters, suitable for detail staking, road edges, glacier monitoring, etc. It normally requires standardized measurement accompanied by differential correction.
  • The 7th decimal place corresponds to about 1.1 centimeters, sufficient for many surveying needs and approaching the upper limit of conventional GPS.
  • The 8th decimal place corresponds to about 1.1 millimeters, mostly used in scientific scenarios such as tectonic plate movement and volcanic deformation, typically requiring long-running fixed reference stations.
  • Beyond that, the 9th decimal place enters the scale of 100 micrometers, and the 13th can reach the nanometer scale. For ordinary point applications on Earth, these digits carry almost no practical measurement meaning, and existing surveying equipment cannot achieve corresponding accuracy.

How to Choose the Number of Decimal Places in Practice

Based on the relationships above, here is a rough, unofficial suggestion:

  • For mobile phone sampling, web map picking, and rough POI cleaning, keeping 4 to 5 decimal places is usually enough.
  • For general GIS projects, navigation, and LBS applications, use 6 decimal places.
  • For surveying, staking, and high-precision monitoring, keep 7 to 8 decimal places according to industry specifications, and clearly annotate the coordinate system, collection method, and accuracy description.
  • Continuing beyond 9 decimal places, unless it belongs to special earth science research, is more likely a case where the export process did not control the significant digits.

For longitude, you also need to consider the cosine of the latitude. For example, at 40° north latitude, 1 degree of longitude equals about 85 kilometers, so the 1st decimal place corresponds to 8.5 kilometers, which is finer than the 11 kilometers at the equator.

Summary

The number of decimal places determines how finely you can express a point, but not how accurately it was measured. The same 6 decimal places from a smartphone A-GPS fix and an RTK fixed solution represent completely different quality levels. Coordinate reliability depends on factors such as the collection instrument, whether differential correction was used, and the presence of obstructions or multipath effects. It is best to separately explain these aspects before storing in a database or releasing to the public.

If you have better practices for precision control, feel free to leave a comment.