metaverseDubai/data/OSM_DATA_RESEARCH.md

OpenStreetMap Data Research - Dubai Marina

Overview

Research and documentation for acquiring OpenStreetMap (OSM) data for Dubai Marina district.

Target Area

Location: Dubai Marina, Dubai, UAE
Coordinates: 25.0772° N, 55.1394° E
Bounding Box:

• Min: 25.07° N, 55.13° E
• Max: 25.09° N, 55.15° E
• Format: 25.07,55.13,25.09,55.15

Data Sources

1. Overpass API (Recommended)

Endpoint: https://overpass-api.de/api/interpreter

Query for Buildings:

[out:xml][bbox:25.07,55.13,25.09,55.15];
(way["building"];);
out meta;

Query for Roads:

[out:xml][bbox:25.07,55.13,25.09,55.15];
(way["highway"];);
out meta;

Query for Complete Area Data:

[out:xml][bbox:25.07,55.13,25.09,55.15];
(
  way["building"];
  way["highway"];
  relation["building"];
);
out meta;

2. Geofabrik Downloads

Regional Extract: Middle East

• URL: https://download.geofabrik.de/asia/united-arab-emirates.html
• Format: OSM PBF
• Size: ~50-100MB for UAE
• Update Frequency: Daily

3. Planet OSM

Full Planet Extract:

• URL: https://planet.openstreetmap.org/
• Format: OSM XML or PBF
• Size: Very large (entire planet)
• Use: Extract UAE region using osmium-tool

Data Acquisition Methods

Method 1: Overpass API (Quick, Small Areas)

Using curl:

curl -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "data=[out:xml][bbox:25.07,55.13,25.09,55.15];(way[\"building\"];);out meta;" \
  "https://overpass-api.de/api/interpreter" \
  > dubai_marina.osm

Using Python (overpy library):

import overpy

api = overpy.Overpass()
query = """
[bbox:25.07,55.13,25.09,55.15];
(way["building"];);
out body;
"""
result = api.query(query)

Method 2: Geofabrik Download (Larger Areas)

# Download UAE extract
wget https://download.geofabrik.de/asia/united-arab-emirates-latest.osm.pbf

# Extract Dubai Marina area using osmium-tool
osmium extract -b 55.13,25.07,55.15,25.09 \
  united-arab-emirates-latest.osm.pbf \
  -o dubai_marina.osm.pbf

Method 3: Using OSMnx (Python)

import osmnx as ox

# Define area
place = "Dubai Marina, Dubai, UAE"
bounds = (25.07, 55.13, 25.09, 55.15)

# Download buildings
buildings = ox.geometries_from_bbox(
    north=25.09, south=25.07, 
    east=55.15, west=55.13,
    tags={'building': True}
)

# Download roads
roads = ox.graph_from_bbox(
    north=25.09, south=25.07,
    east=55.15, west=55.13,
    network_type='drive'
)

Data Processing

Convert OSM to GeoJSON

Using osmtogeojson:

npm install -g osmtogeojson
osmtogeojson dubai_marina.osm > dubai_marina.geojson

Using Python (osmium):

import osmium
import json

class BuildingHandler(osmium.SimpleHandler):
    def __init__(self):
        osmium.SimpleHandler.__init__(self)
        self.buildings = []
    
    def way(self, w):
        if 'building' in w.tags:
            # Extract building geometry
            # Convert to GeoJSON
            pass

handler = BuildingHandler()
handler.apply_file("dubai_marina.osm")

Process with our script

Using our import script:

python3 scripts/data/import_osm_data.py \
  --output data/processed/dubai_marina_buildings.geojson \
  --bounds "25.07,55.13,25.09,55.15"

Expected Data

Buildings

• Estimated Count: 50-100 buildings in Dubai Marina area
• Types: Residential towers, commercial buildings, hotels
• Key Buildings:
  • Cayan Tower (Hero landmark)
  • Marina Towers
  • Various residential complexes

Roads

• Main Roads:
  • Sheikh Zayed Road (E11)
  • Dubai Marina Walk
  • Various internal roads

Additional Features

• Water features (marina)
• Parks and green spaces
• Parking areas
• Sidewalks

Data Quality Considerations

Accuracy

• OSM data is community-sourced
• Building footprints may not be 100% accurate
• Heights are often missing
• Some buildings may be missing

Validation

• Cross-reference with satellite imagery
• Verify building positions
• Check for missing major buildings
• Validate road network

Enhancement

• Add building heights from other sources
• Refine building footprints using satellite imagery
• Add missing buildings manually
• Enhance road network details

Recommended Approach

1. Initial Acquisition: Use Overpass API for quick download
2. Validation: Compare with satellite imagery
3. Enhancement: Add missing buildings and details
4. Processing: Convert to GeoJSON for Unreal import
5. Import: Use our import script to bring into Unreal

Tools Required

• curl or wget - Download data
• Python 3.8+ - Processing scripts
• overpy - Python OSM API client
• osmnx - Advanced OSM data handling (optional)
• osmtogeojson - OSM to GeoJSON converter (optional)
• osmium-tool - Advanced OSM data processing (optional)

Next Steps

1. Run data/acquire_osm_data.sh to download initial data
2. Validate data quality
3. Process with scripts/data/import_osm_data.py
4. Import to Unreal Engine after project creation

---

Last Updated: 2024-11-21
Status: Research Complete, Ready for Data Acquisition