Re: Is it possible to ignore existing overview when performing decimated read?


Sean Gillies
 

HI Loïc,

I don't think we currently have any option other than to remove the overviews. In theory GDAL's OVERVIEW_LEVEL open option could help us, but the implementation isn't there. For example, if you used gdalwarp with `-ovr NONE` it will set an internal overview level to the value of -1. But that bypasses GDALOpenEx (used by rasterio). GDALOpenEx won't let us pass `OVERVIEW_LEVEL=-1. It fails with a "Cannot open overview level -1" error.

To illustrate

>>> rasterio.open("tests/data/green.tif").shape
(64, 64)
>>> rasterio.open("tests/data/green.tif", OVERVIEW_LEVEL=0).shape
(32, 32)
>>> rasterio.open("tests/data/green.tif", OVERVIEW_LEVEL="NONE").shape
(32, 32)
>>> rasterio.open("tests/data/green.tif", OVERVIEW_LEVEL=-1).shape
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 261, in rasterio._base.DatasetBase.__init__
    self._hds = open_dataset(filename, flags, driver, kwargs, None)
  File "rasterio/_shim.pyx", line 78, in rasterio._shim.open_dataset
    return exc_wrap_pointer(hds)
  File "rasterio/_err.pyx", line 213, in rasterio._err.exc_wrap_pointer
    raise exc
rasterio._err.CPLE_OpenFailedError: Cannot open overview level -1 of tests/data/green.tif

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sean/projects/rasterio/rasterio/env.py", line 433, in wrapper
    return f(*args, **kwds)
  File "/home/sean/projects/rasterio/rasterio/__init__.py", line 221, in open
    s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
  File "rasterio/_base.pyx", line 263, in rasterio._base.DatasetBase.__init__
    raise RasterioIOError(str(err))
rasterio.errors.RasterioIOError: Cannot open overview level -1 of tests/data/green.tif


On Fri, Nov 13, 2020 at 4:33 AM Loïc Dutrieux <loic.dutrieux@...> wrote:
Hi Denis,

Thanks for pointing to that issue. I may have misunderstood it but I don't think it helps answering my question.

Cheers,
Loïc
________________________________________
From: main@rasterio.groups.io <main@rasterio.groups.io> on behalf of Denis Rykov <rykovd@...>
Sent: 10 November 2020 21:36:13
To: main@rasterio.groups.io
Subject: Re: [rasterio] Is it possible to ignore existing overview when performing decimated read?

You can find an example how to do that here: https://github.com/mapbox/rasterio/issues/1929<https://urldefense.com/v3/__https://github.com/mapbox/rasterio/issues/1929__;!!DOxrgLBm!XaInc3_J0VCLXQo6gYXo0rBWHs4b0p4ZkMuimU2g9HtJy0eRb55BN-J7JDPt-MB3Dhg5K3E$>.

On Tue, Nov 10, 2020, 8:12 PM Loïc Dutrieux <loic.dutrieux@...<mailto:loic.dutrieux@...>> wrote:
Hi everyone,

I'm trying to perform a decimated read of a dataset that already contains overviews. Regardless of which value I pass to the resampling= argument of the read method, it seems that the already existing overview is used. Is there any way to ignore it? See the reproducible example below.

Cheers,
Loïc

---

import tempfile
import os

import numpy as np
import rasterio
from rasterio.enums import Resampling


filename = os.path.join(tempfile.gettempdir(), 'overview_test.tif')

# Create random int array
shape = (100, 100)
arr = np.random.randint(0, 9999, size=shape, dtype=np.uint16)

meta = {'height': 100,
        'width': 100,
        'driver': 'GTiff',
        'dtype': np.uint16,
        'count': 1}

# Write random array to file and compute first overview
with rasterio.open(filename, 'w', **meta) as dst:
    dst.write(arr, 1)
    dst.build_overviews([2], Resampling.nearest)

# Read with downsample
with rasterio.open(filename) as src:
    arr_avg = src.read(1, out_shape=(1,50,50), resampling=Resampling.average,
                                  out_dtype=np.float)
    arr_nrt = src.read(1, out_shape=(1,50,50), resampling=Resampling.nearest)

print(np.max(arr_avg - arr_nrt))
# when the source file does not contain overviews, the max of the difference array
# is > 0


--
Sean Gillies

Join main@rasterio.groups.io to automatically receive all group messages.