Topics

debugging build_overviews

Guy Doulberg
 

Hi guys,

I wonder if you can give me some ideas about something,
I am working on a raster (with mask) that I am creating overviews on it.

It looks like that in some of the overviews (not all) I am getting artifacts, these artifacts looks like the product of using masked values in the resampling algo. 

How would you debug such a problem?

If I understand correctly some of the code that handles the build overviews is in cython code (rasterio) and some is part of GDAL c code.  Can I run a debug tool that I can traverse these two code bases and add breakpoints in them  

What else can I do?

artifact example:
first plot is of original data, the white spaces are masks:



and now in and overview for X8 resolution I get the black line



Any advice can help,

Thanks

Sean Gillies
 

Hi Guy,

Let's try to answer your questions on the main discussion group. The dev group is for discussing development of the library itself: new API methods, infrastructure, governance, &c. I've cross-posted to main@rasterio.groups.io.

Yes, if you configure and build a debug-enabled version of GDAL (see the GDAL wiki for instructions) and build Rasterio using that build you can run your program under gdb or lldb and set breakpoints in Rasterio and GDAL. However, there is very little logic in the Rasterio code around building overviews:


We do some pre-overview error handling and then call GDALBuildOverviews:


I have seen various reports of overview bugs in the old GDAL tracker, mostly fixed, but maybe there is a lingering one in your case. It may be worth a check in the tracker for open issues or ones that sound familiar.

Simpler than using a debugger: add some logging statements to rasterio/_io.pyx in a custom build of Rasterio to report the overview factors and resampling method passed to GDALBuildOverviews.

On Wed, Jan 2, 2019 at 1:59 AM Guy Doulberg <guyd@...> wrote:
Hi guys,

I wonder if you can give me some ideas about something,
I am working on a raster (with mask) that I am creating overviews on it.

It looks like that in some of the overviews (not all) I am getting artifacts, these artifacts looks like the product of using masked values in the resampling algo. 

How would you debug such a problem?

If I understand correctly some of the code that handles the build overviews is in cython code (rasterio) and some is part of GDAL c code.  Can I run a debug tool that I can traverse these two code bases and add breakpoints in them  

What else can I do?

artifact example:
first plot is of original data, the white spaces are masks:



and now in and overview for X8 resolution I get the black line



Any advice can help,

Thanks



--
Sean Gillies