Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue converting the generated tiles to MBtiles #12

Closed
idouiri9806 opened this issue Sep 22, 2024 · 1 comment
Closed

Issue converting the generated tiles to MBtiles #12

idouiri9806 opened this issue Sep 22, 2024 · 1 comment

Comments

@idouiri9806
Copy link

Hi! I’ve used the sectional chart shapes from your repository and successfully generated the tiles. They are structured in the Tiles directory. and they turned out great! My goal is to eventually convert them into MBTiles. However, I keep running into errors. What’s the best method to convert them to MBTiles?

Here’s the script I’m currently using:

#!/bin/bash
set -e

# Change to the directory where the script is located
cd "$(dirname "$0")"

CHARTS=("Albuquerque" "Anchorage" "Atlanta" "Bethel" "Billings" "Brownsville" "Cape_Lisburne" "Charlotte" "Cheyenne" "Chicago" "Cincinnati" "Cold_Bay" "Dallas-Ft_Worth" "Dawson" "Denver" "Detroit" "Dutch_Harbor" "El_Paso" "Fairbanks" "Great_Falls" "Green_Bay" "Halifax" "Hawaiian_Islands" "Houston" "Jacksonville" "Juneau" "Kansas_City" "Ketchikan" "Klamath_Falls" "Kodiak" "Lake_Huron" "Las_Vegas" "Los_Angeles" "McGrath" "Memphis" "Miami" "Montreal" "New_Orleans" "New_York" "Nome" "Omaha" "Phoenix" "Point_Barrow" "Salt_Lake_City" "San_Antonio" "San_Francisco" "Seattle" "Seward" "St_Louis" "Twin_Cities" "Washington" "Western_Aleutian_Islands" "Wichita")

rm -f all_charts.vrt || true
rm -rf webviewer/tiles || true

# Print the current working directory and list files for debugging
echo "Current working directory: $(pwd)"
echo "Files in the current directory:"
ls -l

process_chart() {
  local CHART=$1
  CHART_DIR="${CHART//_/ }"
  SHAPEFILE_NAME=$(echo "${CHART}" | tr '[:upper:]' '[:lower:]' | tr '-' '_')
  echo "Processing chart: $CHART"

  if [ "$CHART" == "Western_Aleutian_Islands" ]; then
    for PART in "East" "West"; do
      SOURCE_FILE="${CHART}/${CHART_DIR} ${PART} SEC.tif"
      DEST_FILE="${CHART}/${CHART_DIR} ${PART} cropped.tif"
      SHAPEFILE_PART_NAME="${SHAPEFILE_NAME}_$(echo ${PART} | tr '[:upper:]' '[:lower:]')"
      echo "Source file: $SOURCE_FILE"
      echo "Destination file: $DEST_FILE"
      echo "Shapefile: sectional_chartmaker/${SHAPEFILE_PART_NAME}.shp"

      if [ ! -f "$SOURCE_FILE" ]; then
        echo "Error: Source file $SOURCE_FILE does not exist."
        exit 1
      fi

      gdalwarp \
        -t_srs EPSG:3857 \
        -co TILED=YES \
        -dstalpha \
        -of GTiff \
        -cutline "sectional_chartmaker/${SHAPEFILE_PART_NAME}.shp" \
        -crop_to_cutline \
        -wo NUM_THREADS=$(sysctl -n hw.ncpu) \
        -multi \
        -overwrite \
        "$SOURCE_FILE" \
        "$DEST_FILE"

      gdal_translate -of vrt -expand rgba "$DEST_FILE" "${CHART}/${CHART_DIR} ${PART}.vrt"
    done
  else
    SOURCE_FILE="${CHART}/${CHART_DIR} SEC.tif"
    DEST_FILE="${CHART}/${CHART_DIR} cropped.tif"
    echo "Source file: $SOURCE_FILE"
    echo "Destination file: $DEST_FILE"
    echo "Shapefile: sectional_chartmaker/${SHAPEFILE_NAME}.shp"

    if [ ! -f "$SOURCE_FILE" ]; then
      echo "Error: Source file $SOURCE_FILE does not exist."
      exit 1
    fi

    gdalwarp \
      -t_srs EPSG:3857 \
      -co TILED=YES \
      -dstalpha \
      -of GTiff \
      -cutline "sectional_chartmaker/${SHAPEFILE_NAME}.shp" \
      -crop_to_cutline \
      -wo NUM_THREADS=$(sysctl -n hw.ncpu) \
      -multi \
      -overwrite \
      "$SOURCE_FILE" \
      "$DEST_FILE"

    gdal_translate -of vrt -expand rgba "$DEST_FILE" "${CHART}/${CHART_DIR}.vrt"
  fi
}

export -f process_chart

# Use GNU Parallel to process charts in parallel
parallel process_chart ::: "${CHARTS[@]}"

gdalbuildvrt all_charts.vrt */*.vrt

gdal2tiles.py \
  --zoom "0-11" \
  --processes=$(sysctl -n hw.ncpu) \
  --webviewer=none \
  --exclude \
  --tiledriver=WEBP \
  --webp-quality=50 \
  all_charts.vrt \
  webviewer/tiles
@N129BZ
Copy link
Owner

N129BZ commented Sep 22, 2024

There are 2 steps you need to accomplish. The Tiling operation produces redundant tiles where charts overlap at their boundaries, and this will end up being many thousands of individual images due to the multiple zoom levels. The tiles need to go through a merging operation to merge these duplicate images into a single image, and then processed into an mbtiles database using another python utility named mbutil, you can find that in this project's directories. Look at the code in make.js to see how this is being done.

@N129BZ N129BZ closed this as completed Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants