Re: Does rasterio load data into memory while reprojecting?
On Wed, Jul 4, 2018 at 4:09 AM, Denis Rykov <rykovd@...> wrote:
Yes, you can warp arbitrarily large rasters when you use rasterio.band(). The warper chunks the work so that its working arrays are no more than 64 MB in size. You can increase this limit with reproject's warp_mem_limit keyword argument.The source dataset's bands are incrementally loaded into memory as the warper works, but only up to the limit of GDAL's block cache (Rasterio uses the GDAL C library). If you profile your program and see the memory allocation increase, this is due to GDAL's caching. You can see also the notes at http://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#WarpandCacheMemory:TechnicalDetails, which apply to Rasterio.
Please note that you can warp multiple imagery bands in one reproject() call by passing sequences of band indexes.
reproject(source=rasterio.band(src, src.indexes), destination=rasterio.band(src, dst.indexes), ...)
This is faster than looping over the bands.