GIS professionals have all done vectorization data processing, tracing buildings and roads in imagery point by point, with sore eyes and aching hands. Now that we are in the AI era, is there a faster method? I previously introduced relevant tools in "Automatic Vectorization with an AI Plugin in QGIS", but that tool was still not quite good enough; it was only icing on the cake. Recently I found that Microsoft has released a large model called MARS, which can directly generate line and polygon vector files used in GIS software from imagery. Today I will introduce it.

About MARS

MARS is a feature extraction model released by Microsoft. It can convert satellite and aerial imagery into vector GIS data suitable for map processing. The output is not an image, but GeoJSON data with classes and confidence scores, which can be loaded into QGIS or ArcGIS for further editing and analysis.

MARS extracts buildings, roads, railways, and water from London aerial imagery

Figure: MARS extracts buildings, roads, railways, and water from London aerial imagery. Source: Microsoft.

Traditional remote sensing models usually output raster masks first, then go through steps such as contour extraction, connection, and simplification to convert them into vectors, causing errors to propagate step by step. MARS directly learns coordinate sequences of points, lines, and polygons, eliminating the intermediate conversion.

Comparison of original London imagery and the map generated by MARS

Figure: Comparison of original London imagery and the map generated by MARS. Source: Microsoft.

How MARS Works

MARS generates results in a way somewhat similar to large language models. A large language model predicts text piece by piece based on the preceding context, while MARS reads image content and then continues predicting feature classes and the next coordinate based on existing results. When coordinates are connected in order, they form vector geometries such as building polygons and road centerlines.

Specifically, image tiles are first fed into GeoformerModel, and the accompanying CocoAutoregressiveProcessor is responsible for converting map information such as classes and coordinates into sequences that the model can process. After the model completes prediction, these sequences are restored to GeoJSON, which contains geometry, feature class, and confidence, and can directly enter subsequent GIS data processing workflows. Traditional methods usually first determine pixel by pixel "where is a building and where is a road," obtain a raster mask, and then extract contours or centerlines, followed by simplification, connection, and topology repair. MARS skips the step of "first generating a raster, then converting to vectors" and directly outputs coordinates of points, lines, and polygons, thus reducing contour deformation and precision loss caused by intermediate conversion.

Currently, the official deployment provides two models by feature type: the building model processes 256×256 pixel tiles and mainly generates building polygons; the road and railway model processes 512×512 pixel tiles and mainly generates linear road networks. Water bodies are extracted by an accompanying model, and can ultimately be organized into map layers together with other features. To train this set of models, Microsoft compiled the MAP-3M dataset with about 3.4 million samples.

Microsoft also demonstrated results for a high-density urban area in Manila, Philippines. Faced with rivers, dense buildings, and a complex road network, MARS can still generate building, road, and water layers that can be used for further mapping, as shown below.

Comparison of original imagery and MARS-generated map for a high-density urban area in Manila

Figure: Comparison of original imagery and MARS-generated map for a high-density urban area in Manila. Source: Microsoft.

How to Try MARS

The paper provides a Hugging Face demo page. The Foundry preview requires an Azure subscription, GPU quota, and Planetary Computer Pro. I currently do not have the RMB to try it, so I did not try it myself.

The specific workflow is as follows:

  1. Log in to the MARS model page and deploy the model in Microsoft Foundry.
  2. Check GPU quota. The official specification uses Standard_NC24ads_A100_v4. If there is no quota, you need to submit an application first. After deployment, billing is based on actual usage.
  3. Create Microsoft Planetary Computer Pro and establish an imagery collection in GeoCatalog.
  4. Import orthorectified RGB imagery. The official recommendation is a ground sampling distance of 25–75 cm, with around 60 cm being more suitable. Save the data as COG, and clearly write gsd, proj:epsg, eo:bands, bbox, and geometry in STAC.
  5. Call MARS through the GeoAI SDK. It is recommended to perform inference on 512×512 pixel tiles, then check and export vector layers.

Can It Replace Manual Vectorization?

MARS currently relies on high-resolution imagery at 25–75 cm. Its performance may degrade when switching regions or sensors, or when encountering shadows and occlusion. In addition, generated results still require manual checks of topology, offset, attribute completeness, coordinate systems, and so on. Because I have not actually tested it, I cannot draw a conclusion for now. If the accuracy stated on the official website holds, relevant vectorization work can first be run automatically with MARS and then manually processed in a focused way. Compared with traditional vectorization work, however, this can greatly reduce time and labor costs. Of course, given the current salary levels for data processing in China, it is hard to say whether human labor or AI is more expensive (self-deprecating joke).

Finally, when dealing with data in China, be sure to pay attention to confidentiality regulations. Classified data must not be sent directly to overseas cloud services. Those who have used it are also welcome to leave comments for discussion.

References

  1. AI-Enabled Map Production Using Microsoft's MARS Model: https://techcommunity.microsoft.com/blog/microsoft-planetary-computer-blog/ai-enabled-map-production-using-microsofts-mars-model/4542498
  2. MARS - Microsoft Foundry Labs: https://labs.ai.azure.com/innovations/mars/
  3. MARS model page: https://ai.azure.com/catalog/models/mars-map-autoregressive
  4. MARS - A Foundational Map Auto-Regressor: https://openreview.net/forum?id=QV4sV5cbLl
  5. Manage and increase quotas for resources: https://learn.microsoft.com/en-us/azure/foundry/how-to/quota
  6. MAP-3M dataset: https://huggingface.co/datasets/bag-lab/MAP-3M