Re: Is it possible to ignore existing overview when performing decimated read?

Sean Gillies

HI Loïc,

I don't think we currently have any option other than to remove the overviews. In theory GDAL's OVERVIEW_LEVEL open option could help us, but the implementation isn't there. For example, if you used gdalwarp with `-ovr NONE` it will set an internal overview level to the value of -1. But that bypasses GDALOpenEx (used by rasterio). GDALOpenEx won't let us pass `OVERVIEW_LEVEL=-1. It fails with a "Cannot open overview level -1" error.

To illustrate

(64, 64)
>>>"tests/data/green.tif", OVERVIEW_LEVEL=0).shape
(32, 32)
>>>"tests/data/green.tif", OVERVIEW_LEVEL="NONE").shape
(32, 32)
>>>"tests/data/green.tif", OVERVIEW_LEVEL=-1).shape
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 261, in rasterio._base.DatasetBase.__init__
    self._hds = open_dataset(filename, flags, driver, kwargs, None)
  File "rasterio/_shim.pyx", line 78, in rasterio._shim.open_dataset
    return exc_wrap_pointer(hds)
  File "rasterio/_err.pyx", line 213, in rasterio._err.exc_wrap_pointer
    raise exc
rasterio._err.CPLE_OpenFailedError: Cannot open overview level -1 of tests/data/green.tif

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sean/projects/rasterio/rasterio/", line 433, in wrapper
    return f(*args, **kwds)
  File "/home/sean/projects/rasterio/rasterio/", line 221, in open
    s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
  File "rasterio/_base.pyx", line 263, in rasterio._base.DatasetBase.__init__
    raise RasterioIOError(str(err))
rasterio.errors.RasterioIOError: Cannot open overview level -1 of tests/data/green.tif

On Fri, Nov 13, 2020 at 4:33 AM Loïc Dutrieux <loic.dutrieux@...> wrote:
Hi Denis,

Thanks for pointing to that issue. I may have misunderstood it but I don't think it helps answering my question.

From: <> on behalf of Denis Rykov <rykovd@...>
Sent: 10 November 2020 21:36:13
Subject: Re: [rasterio] Is it possible to ignore existing overview when performing decimated read?

You can find an example how to do that here:<;!!DOxrgLBm!XaInc3_J0VCLXQo6gYXo0rBWHs4b0p4ZkMuimU2g9HtJy0eRb55BN-J7JDPt-MB3Dhg5K3E$>.

On Tue, Nov 10, 2020, 8:12 PM Loïc Dutrieux <loic.dutrieux@...<mailto:loic.dutrieux@...>> wrote:
Hi everyone,

I'm trying to perform a decimated read of a dataset that already contains overviews. Regardless of which value I pass to the resampling= argument of the read method, it seems that the already existing overview is used. Is there any way to ignore it? See the reproducible example below.



import tempfile
import os

import numpy as np
import rasterio
from rasterio.enums import Resampling

filename = os.path.join(tempfile.gettempdir(), 'overview_test.tif')

# Create random int array
shape = (100, 100)
arr = np.random.randint(0, 9999, size=shape, dtype=np.uint16)

meta = {'height': 100,
        'width': 100,
        'driver': 'GTiff',
        'dtype': np.uint16,
        'count': 1}

# Write random array to file and compute first overview
with, 'w', **meta) as dst:
    dst.write(arr, 1)
    dst.build_overviews([2], Resampling.nearest)

# Read with downsample
with as src:
    arr_avg =, out_shape=(1,50,50), resampling=Resampling.average,
    arr_nrt =, out_shape=(1,50,50), resampling=Resampling.nearest)

print(np.max(arr_avg - arr_nrt))
# when the source file does not contain overviews, the max of the difference array
# is > 0

Sean Gillies

Join to automatically receive all group messages.