Hi all,

Rasterio 1.2.5 distributions were uploaded to PyPI yesterday evening. Here are the changes: https://github.com/mapbox/rasterio/blob/master/CHANGES.txt#L17-L21.

Thank you for the bug reports and contributions, everyone!

--

Sean Gillies

[Edited Message Follows]
[Reason: typos]

I'm not sure if this is a bug or a subtle unexpected behaviour of the function rasterio.merge.merge.

I'm creating a mosaic with several images for the whole globe. The resolution is such that the image should be 16000x8001 (width x height). The calculation of the resolution is done through these variables:

pixels = 16000

res = 360 / pixels

xmin = -180 - res / 2

xmax = 180 - res / 2

ymin = -90 - res / 2

ymax = 90 + res / 2

Which result in xmin, xmax, ymin, ymax = (-180.01125, 179.98875, -90.01125, 90.01125). OK, the problem is that when I use merge as:  
mosaic, out_trans = merge(files, bounds=(xmin, ymin, xmax, ymax), res=res,  method='max')

The shape os mosaic is NOT (4, 16000, 8001), but (4, 16000, 8002) (note the extra row).

I think the problem comes from a rounding issue. In my python terminal (ymax-ymin)/res = 8001.000000000001 (whereas (xmax-xmin)/res = 16000.0). This rounding error of 1E-12 seems to be forcing the number of rows to be 8002 instead of 8001. This might seem like a tiny approximation error, but the problem is that controlling the shape of the image is very important for further calculations downstream of the algorithm.

Is this what it is supposed to happen? Is there a way to "force" merge to produce an image of the expected shape 16000x8001?

Thanks.

Hi Everyone,

I understand that AWS S3 is by far the most common cloud object store provider, especially in the geospatial community. Unfortunately some users don't get to choose which object store provider they're given to use.

Rasterio already has Session support for five different Cloud sessions, including GSSession, OSSSession, AzureSession, SwiftSession and AWSSession. (In this case, a sessions are "classes that configure access to secured resources".)

However it seems that only AWS is a first-class citizen in rasterIO for the following reasons:

• In the docs, on the topic of the use of objects on the cloud as virtual files, only support for AWS S3 is shown: https://rasterio.readthedocs.io/en/latest/topics/vsi.html
  • Other cloud providers are not even mentioned. I didn't know other sessions existed until I looked into the code.
• When installing rasterio, you can add support for credentailizing AWS S3 (with boto3) using the package "extras" syntax: `pip install rasterio[s3]`.
  • No other cloud providers have their own addon available at install-time.
• When configuring a rasterio context-manger with `rasterio.Env()` you have the ability to pass in an AWSSession or aws credentials to credentialize your context.
  • If you don't pass in credentails, `Env` will make a session using `Session.aws_or_dummy()`, but doesn't attempt to check the if other provider sessions should be used.

I propose the change to move the current object-store feature implementation to be less AWS focused, with

• Ability to install requirement packages for other cloud providers, using additional "extras" options:
  • eg, `rasterio[gs]`, `rasterio[azure]`, `rasterio[swift]`
• Modify the `session.Env()` context manger to be cloud-platform agnostic, ie, no "aws-or-dummy" behaviour, based on which supporting-packages are installed and which environment variables are present.
• Document all of the different cloud providers that Rasterio supports, and how to configure and use them.

I understand this is a large piece of work, and an tall proposal for an opensource project. I'm usually one to scratch my own itch, my specific requirement out of this is to be able to use rasterio with my existing OpenStack Swift object store. I want to be able to use swift by passing pre-configured application credentials and using openstack `keystonev3` and `swiftclient` libraries to configure and credentialize the context used for GDAL. Of course I could simply open a PR with some code to patch in that support in the existing SwiftSession, but I think a cleaner solution would involve looking at the issues highlighted above, to make a more generalized fix overall, on top of the extra Swift features that I require.

Thanks for reading.

Greetings everyone,

I am trying to do reprojection in chunks and have been using rasterio.warp.calculate_default_transform for getting the projected extent and transform of the chunk. However, I noticed rasterio.warp.aligned_target also returns a transform and bounds, but the transform is a bit different from the one returned by calculate_default_transform. The documentation is a little vague – calculate_default_transform: Output dimensions and transform for a reprojection; aligned_target: Aligns target to specified resolution.

Can anyone illuminate this for me? And do you have any insight for which might be better for reprojecting a huge raster in chunks that will eventually be merged later? Thank you.

Hello everyone,

I was wondering if there are any good examples/reasons for having rasters that contain bands with mixed dtypes e.g. ('uint8', 'float32').

This seem to be mostly supported in rasterio, but functions like rasterio.open accept only one dtype. Furthermore, mixed dtypes are not supported in GTiff apparently: https://lists.osgeo.org/pipermail/gdal-dev/2010-August/025657.html

I'm just wondering if I should make the effort to accept mixed dtypes in a library I'm developing, which is based upon rasterio.

Thanks!

Amaury

Looks like your data is Single Look Complex (SLC), you'll need to convert it to float. You can do this yourself with numpy, or with https://gdal.org/drivers/raster/derived.html GDAL (which rasterio uses under the hood) will compute the intensity for you. 

e.g.

src = rasterio.open('DERIVED_SUBDATASET:INTENSITY:../x.SAFE/measurement/y-001.tif')

Sebastien

   

On Wed, May 26, 2021 at 12:38 PM <earthdata@...> wrote:

I'm a first time user. I downloaded a Sentinel-1 granule. I'm now trying to visualize it via the following script:

import rasterio
from rasterio.plot import show
path = '.../x.SAFE/measurement/y-001.tiff'
src = rasterio.open(path)

show(src)

However this gives me the following error:

TypeError: Image data of dtype complex128 cannot be converted to float

What else do I need to do?

._,

I'm a first time user. I downloaded a Sentinel-1 granule. I'm now trying to visualize it via the following script:

import rasterio
from rasterio.plot import show
path = '.../x.SAFE/measurement/y-001.tiff'
src = rasterio.open(path)

show(src)

However this gives me the following error:

TypeError: Image data of dtype complex128 cannot be converted to float

What else do I need to do?

This could be similar to https://github.com/mapbox/rasterio/issues/1686

In an AWS-Lambda python 3.7 runtime, there is an error like:

 

    CPLReleaseMutex: Error = 1 (Operation not permitted)

    [WARNING] 2021-05-07T17:38:33.315Z 3e8b4b55-9066-426c-9319-540e39e17709

    CPLE_AppDefined in HTTP response code on https://bucket-xxx-us-west-2.s3.amazonaws.com/prefix/geotiff.tif: 0

    2021-05-07 17:38:33,366 | ERROR | '/vsis3/bucket-xxx-us-west-2/prefix/geotiff.tif' not recognized as a supported file format.

The python concurrency that wraps the rasterio/gdal reader is a common pattern like:

    dataset = []

    with concurrent.futures.ThreadPoolExecutor() as executor:

        future_to_task = {}

        for task in peril_queries:

            future = executor.submit(extract_geotiff_data, task)

            future_to_task[future] = task

    for future in concurrent.futures.as_completed(future_to_task):

        try:

            data = future.result()

            dataset.extend(data)

        except Exception as exc:

            task = future_to_task[future]

            LOGGER.error("Failed Task: %s", task)

            raise

 

Where the 'extract_geotiff_data' function is basically given a task to read an s3-COG,
and that function encapsulates everything to setup a rasterio.Env with gdal env-var
settings, opening the s3 object and reading data from it.

What is causing the `CPLReleaseMutex: Error` and how is it debugged and avoided?

TIA,
Darren