Hi everyone,

I'm trying to perform a decimated read of a dataset that already contains overviews. Regardless of which value I pass to the resampling= argument of the read method, it seems that the already existing overview is used. Is there any way to ignore it? See the reproducible example below.

Cheers,
Loïc

---

import tempfile
import os

import numpy as np
import rasterio
from rasterio.enums import Resampling


filename = os.path.join(tempfile.gettempdir(), 'overview_test.tif')

# Create random int array
shape = (100, 100)
arr = np.random.randint(0, 9999, size=shape, dtype=np.uint16)

meta = {'height': 100,
        'width': 100,
        'driver': 'GTiff',
        'dtype': np.uint16,
        'count': 1}

# Write random array to file and compute first overview
with rasterio.open(filename, 'w', **meta) as dst:
    dst.write(arr, 1)
    dst.build_overviews([2], Resampling.nearest)

# Read with downsample
with rasterio.open(filename) as src:
    arr_avg = src.read(1, out_shape=(1,50,50), resampling=Resampling.average,
                                  out_dtype=np.float)
    arr_nrt = src.read(1, out_shape=(1,50,50), resampling=Resampling.nearest)

print(np.max(arr_avg - arr_nrt))
# when the source file does not contain overviews, the max of the difference array
# is > 0

Hi Denis,

Thanks for pointing to that issue. I may have misunderstood it but I don't think it helps answering my question.

Cheers,
Loïc
________________________________________
From: main@rasterio.groups.io <main@rasterio.groups.io> on behalf of Denis Rykov <rykovd@...>
Sent: 10 November 2020 21:36:13
To: main@rasterio.groups.io
Subject: Re: [rasterio] Is it possible to ignore existing overview when performing decimated read?

You can find an example how to do that here: https://github.com/mapbox/rasterio/issues/1929<https://urldefense.com/v3/__https://github.com/mapbox/rasterio/issues/1929__;!!DOxrgLBm!XaInc3_J0VCLXQo6gYXo0rBWHs4b0p4ZkMuimU2g9HtJy0eRb55BN-J7JDPt-MB3Dhg5K3E$>.

On Tue, Nov 10, 2020, 8:12 PM Loïc Dutrieux <loic.dutrieux@...<mailto:loic.dutrieux@...>> wrote:
Hi everyone,

I'm trying to perform a decimated read of a dataset that already contains overviews. Regardless of which value I pass to the resampling= argument of the read method, it seems that the already existing overview is used. Is there any way to ignore it? See the reproducible example below.

Cheers,
Loïc

---

import tempfile
import os

import numpy as np
import rasterio
from rasterio.enums import Resampling


filename = os.path.join(tempfile.gettempdir(), 'overview_test.tif')

# Create random int array
shape = (100, 100)
arr = np.random.randint(0, 9999, size=shape, dtype=np.uint16)

meta = {'height': 100,
        'width': 100,
        'driver': 'GTiff',
        'dtype': np.uint16,
        'count': 1}

# Write random array to file and compute first overview
with rasterio.open(filename, 'w', **meta) as dst:
    dst.write(arr, 1)
    dst.build_overviews([2], Resampling.nearest)

# Read with downsample
with rasterio.open(filename) as src:
    arr_avg = src.read(1, out_shape=(1,50,50), resampling=Resampling.average,
                                  out_dtype=np.float)
    arr_nrt = src.read(1, out_shape=(1,50,50), resampling=Resampling.nearest)

print(np.max(arr_avg - arr_nrt))
# when the source file does not contain overviews, the max of the difference array
# is > 0

Many thanks Sean and Even,

So with rasterio compiled against gdal 3.3 I'll be able to force ignore overviews?
If that deserves a new rasterio unit test I'm happy to send a pull request in the coming days.

Cheers,
Loïc
________________________________________
From: main@rasterio.groups.io <main@rasterio.groups.io> on behalf of Sean Gillies via groups.io <sean=mapbox.com@groups.io>
Sent: 13 November 2020 20:57
To: main@rasterio.groups.io
Subject: Re: [rasterio] Is it possible to ignore existing overview when performing decimated read?

Hi Even,

On Fri, Nov 13, 2020 at 10:38 AM Even Rouault <even.rouault@...<mailto:even.rouault@...>> wrote:
On vendredi 13 novembre 2020 09:05:41 CET Sean Gillies via groups.io<https://urldefense.com/v3/__http://groups.io__;!!DOxrgLBm!U8N393pmVlI170WWlMCwvmrIV7A8JSl9WHWOD7E3re7ET8O3tNpqP7DhM2SV30dTeRRzVSk$> wrote:
> HI Loïc,
>
> I don't think we currently have any option other than to remove the
> overviews. In theory GDAL's OVERVIEW_LEVEL open option could help us, but
> the implementation isn't there. For example, if you used gdalwarp with
> `-ovr NONE` it will set an internal overview level to the value of -1. But
> that bypasses GDALOpenEx (used by rasterio). GDALOpenEx won't let us pass
> `OVERVIEW_LEVEL=-1. It fails with a "Cannot open overview level -1" error.

Was left as an exercice for a contributor:
https://github.com/OSGeo/gdal/pull/3181<https://urldefense.com/v3/__https://github.com/OSGeo/gdal/pull/3181__;!!DOxrgLBm!U8N393pmVlI170WWlMCwvmrIV7A8JSl9WHWOD7E3re7ET8O3tNpqP7DhM2SV30dTK8LUNCg$>

Even

Thanks