This could be similar to https://github.com/mapbox/rasterio/issues/1686

In an AWS-Lambda python 3.7 runtime, there is an error like:

 

    CPLReleaseMutex: Error = 1 (Operation not permitted)

    [WARNING] 2021-05-07T17:38:33.315Z 3e8b4b55-9066-426c-9319-540e39e17709

    CPLE_AppDefined in HTTP response code on https://bucket-xxx-us-west-2.s3.amazonaws.com/prefix/geotiff.tif: 0

    2021-05-07 17:38:33,366 | ERROR | '/vsis3/bucket-xxx-us-west-2/prefix/geotiff.tif' not recognized as a supported file format.

The python concurrency that wraps the rasterio/gdal reader is a common pattern like:

    dataset = []

    with concurrent.futures.ThreadPoolExecutor() as executor:

        future_to_task = {}

        for task in peril_queries:

            future = executor.submit(extract_geotiff_data, task)

            future_to_task[future] = task

    for future in concurrent.futures.as_completed(future_to_task):

        try:

            data = future.result()

            dataset.extend(data)

        except Exception as exc:

            task = future_to_task[future]

            LOGGER.error("Failed Task: %s", task)

            raise

 

Where the 'extract_geotiff_data' function is basically given a task to read an s3-COG,
and that function encapsulates everything to setup a rasterio.Env with gdal env-var
settings, opening the s3 object and reading data from it.

What is causing the `CPLReleaseMutex: Error` and how is it debugged and avoided?

TIA,
Darren