Re: MODIS (HDF4) data with MemoryFile


Sean Gillies
 

Hi Hussain,

Thanks for posting here. I don't participate in that Slack site and so I don't see any rasterio questions there.

Can you show the exact code you are using?

I see in GDAL's tests and in an announcement https://lists.osgeo.org/pipermail/gdal-dev/2018-August/048934.html that in-memory HDF5 is supported. It may be possible that in-memory HDF4 is not supported by GDAL. I am not a regular HDF user and so I am not certain whether that should be expected to work or not.

On Fri, Jul 9, 2021 at 4:29 PM Hussain Hassam via groups.io <hhassam=hatfieldgroup.com@groups.io> wrote:

Hello,

 

I posted the below message in the Spatial Community #mapbox slack channel, but I have yet to receive a response. I am hoping to get some information about this before I consider it a bug and post on github issues.

 

We have been trying to open HDF4 files (MODIS data) directly from a source (e.g. https://e4ftl01.cr.usgs.gov//DP131/MOLT/MOD09A1.061/2021.06.26/MOD09A1.A2021177.h14v00.061.2021186035450.hdf) with rasterio (in Python).The first issue is that the source requires authentication. We solved by using requests to download the HDF file into a buffer (python io.BytesIO object. We also seeked it back to 0,0 after writing to it). Our thought was to then load this into rasterio using rasterio.io.MemoryFile. When we try to open the memoryfile, we get an error:

 

CPLE_OpenFailedError: '/vsimem/c604e9d5-cc4b-42bc-a34a-de4e541355cc/c604e9d5-cc4b-42bc-a34a-de4e541355cc.hdf' not recognized as a supported file format.

 

The odd part about this is if we write the io.BytesIO object into a file on disk first (example.hdf) then do a rasterio.open('example.hdf'), everything works as expected, however, we are trying to avoid any sort of IO for a streamlined process.

 

Thank you for your time,

Hussain Hassam

_._,_._,_

--
Sean Gillies

Join main@rasterio.groups.io to automatically receive all group messages.