I am having a strange rasterio issue where if I build rasterio docker image from osgeo/gdal:ubuntu-small-latest as the base image and run it as docker-compose up, the rio fails to open any http based COG files. It works fine when the same TIF file is copied locally and accessed from a mounted drive. However, gdalinfo succeeds for COG file on S3 via HTTP url. When I run the docker image without docker-compose and use the docker commands (docker build && docker run), rio works for COG file on S3 via HTTP url. It seems like some environment is not being set or initialized when run via docker-compose and I am missing something obvious. Anyone ran into this issue before and any debugging tips to debug why the FileOpen is failing via Docker Compose? I set gdal CPL_DEBUG and CPL_LOG_ERRORS env to ON and logs are still minimal.
$ rio --version
OGRCT: PROJ >= 4.8.0 features enabled
OGRCT: Using locale-safe proj version
1.1.1
$ rio info -v http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif
OGRCT: PROJ >= 4.8.0 features enabled
OGRCT: Using locale-safe proj version
Traceback (most recent call last):
File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.__init__
File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset
File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer
rasterio._err.CPLE_OpenFailedError: '/vsicurl/http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif' does not exist in the file system, and is not recognized as a supported dataset name.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/rio", line 11, in <module>
sys.exit(main_group())
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/rasterio/rio/info.py", line 66, in info
with ctx.obj['env'], rasterio.open(input) as src:
File "/usr/local/lib/python3.6/dist-packages/rasterio/env.py", line 445, in wrapper
return f(*args, **kwds)
File "/usr/local/lib/python3.6/dist-packages/rasterio/__init__.py", line 219, in open
s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.__init__
rasterio.errors.RasterioIOError: '/vsicurl/http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif' does not exist in the file system, and is not recognized as a supported dataset name.
$ gdalinfo http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif
HTTP: Fetch(http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif)
HTTP: libcurl/7.58.0 GnuTLS/3.5.18 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
GDAL: GDALOpen(/vsimem/http_1/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif, this=0x55e917f3afa0) succeeds as GTiff.
GDAL: GDALOpen(http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif, this=0x55e917f3afa0) succeeds as HTTP.
Driver: GTiff/GeoTIFF
GDAL: GDALDefaultOverviews::OverviewScan()
MDReaderPleiades: Not a Pleiades product
MDReaderPleiades: Not a Pleiades product
Files: none associated
Size is 49402, 28398
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 20N",
BASEGEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]],
CONVERSION["UTM zone 20N",
METHOD["Transverse Mercator",
ID["EPSG",9807]],
PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",-63,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["Scale factor at natural origin",0.9996,
SCALEUNIT["unity",1],
ID["EPSG",8805]],
PARAMETER["False easting",500000,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],
PARAMETER["False northing",0,
LENGTHUNIT["metre",1],
ID["EPSG",8807]]],
CS[Cartesian,2],
AXIS["(E)",east,
ORDER[1],
LENGTHUNIT["metre",1]],
AXIS["(N)",north,
ORDER[2],
LENGTHUNIT["metre",1]],
USAGE[
SCOPE["unknown"],
AREA["World - N hemisphere - 66°W to 60°W - by country"],
BBOX[0,-66,84,-60]],
ID["EPSG",32620]]
Data axis to CRS axis mapping: 1,2
Origin = (494088.931940000038594,1993386.902620000066236)
Pixel Size = (0.027070000000000,-0.027070000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=pix4dmapper
Image Structure Metadata:
COMPRESSION=YCbCr JPEG
INTERLEAVE=PIXEL
SOURCE_COLOR_SPACE=YCbCr
GTiff: ScanDirectories()
GTiff: Opened 24701x14199 overview.
GTiff: Opened 12351x7100 overview.
GTiff: Opened 6176x3550 overview.
GTiff: Opened 3088x1775 overview.
GTiff: Opened 1544x888 overview.
GTiff: Opened 772x444 overview.
GTiff: Opened 386x222 overview.
Corner Coordinates:
Upper Left ( 494088.932, 1993386.903) ( 63d 3'21.05"W, 18d 1'44.14"N)
Lower Left ( 494088.932, 1992618.169) ( 63d 3'21.04"W, 18d 1'19.13"N)
Upper Right ( 495426.244, 1993386.903) ( 63d 2'35.56"W, 18d 1'44.15"N)
Lower Right ( 495426.244, 1992618.169) ( 63d 2'35.56"W, 18d 1'19.14"N)
Center ( 494757.588, 1993002.536) ( 63d 2'58.30"W, 18d 1'31.64"N)
Band 1 Block=512x512 Type=Byte, ColorInterp=Red
NoData Value=-10000
Overviews: 24701x14199, 12351x7100, 6176x3550, 3088x1775, 1544x888, 772x444, 386x222
Band 2 Block=512x512 Type=Byte, ColorInterp=Green
NoData Value=-10000
Overviews: 24701x14199, 12351x7100, 6176x3550, 3088x1775, 1544x888, 772x444, 386x222
Band 3 Block=512x512 Type=Byte, ColorInterp=Blue
NoData Value=-10000
Overviews: 24701x14199, 12351x7100, 6176x3550, 3088x1775, 1544x888, 772x444, 386x222
GDAL: GDALClose(http://oin-hotosm.s3.amazonaws.com/59c66c5223c8440011d7b1e4/0/7ad397c0-bba2-4f98-a08a-931ec3a6e943.tif, this=0x55e917f3afa0)
GDAL: In GDALDestroy - unloading GDAL shared library.
