Date   

Clarification on usage with QGIS

Ari Meyer
 

Note: I originally submitted this as a github issue, as I'd like to see an update to the documentation regarding using Rasterio with QGIS (and perhaps for integrating with any apps that use GDAL).

We're looking to simplify our code for a QGIS plugin to use Rasterio/Shapely in place of GDAL/OGR. Shapely is included in the standard QGIS distribution, but Rasterio is not (not sure why). In any case, I found on
https://rasterio.readthedocs.io/en/latest/topics/switch.html :
GDAL’s bindings (gdal for the rest of this document) and Rasterio are not entirely compatible and should not, without a great deal of care, be imported and used in a single Python program. The reason is that the dynamic library they each load (these are C extension modules, remember), libgdal.so on Linux, gdal.dll on Windows, has a number of global objects and the two modules take different approaches to managing these objects.

Does this imply that we may have problems if we try to port the GDAL Python code of our QGIS plugin to Rasterio that we may run into problems, as QGIS is built on GDAL? Can you add a section to rasterio.readthedocs.io clarifying interoperability with QGIS?


GDAL Error 4 triggers when working with geo-referenced jpeg

ashnair0007@...
 
Edited

I used gdal_translate to convert a geotiff to a jpg with jpg.aux.xml file.  The following is part of the script I used to get and write the tiles to disk

import rasterio as rio

im = rio.open('./rgb.tif')
meta = im.meta.copy()

for window, transform in get_tiles(w,h,im):
   tile_name = f"{window.col_off}_{window.row_off}"
   meta["transform"] = transform
   meta["width"], meta["height"] = window.width, window.height
   outpath = "./tiles/" + tile_name
   with rio.open(outpath, "w", **meta) as outds:
       outds.write(self.im.read(window=window))

Now when the input is the original geotiff, writing the tiles to disk works perfectly. However, if my input is the jpeg that I created via gdal_translate, writing to tiles still works however it keeps printing out 
ERROR 4: tiles/0_0.jpg: No such file or directory for each tile. So it'll go something like this:

ERROR 4: tiles/0_0.jpg: No such file or directory
ERROR 4: tiles/0_1000.jpg: No such file or directory
ERROR 4: tiles/0_2000.jpg: No such file or directory
.........................

and so on despite successfully writing the jpeg tiles. I'm confused as to why this error gets printed out and would appreciate any clarification regarding this behaviour.


Re: TypeError: open() takes at most 8 arguments (14 given) when writing

ashnair0007@...
 

Yes. That was exactly it. My bad. Thanks.


Re: TypeError: open() takes at most 8 arguments (14 given) when writing

Alan Snow
 

I think the issue is that you need to use `rasterio.open`.


TypeError: open() takes at most 8 arguments (14 given) when writing

ashnair0007@...
 

Hi guys,

I'm trying to write to an array to a tiff as follows:

profile.update(driver="GTiff", count=1)
with open('res.tiff', 'w', **profile) as dst:
dst.write(thresh.astype(rio.uint8), 1)

However, I'm encountering the following error:
TypeError: open() takes at most 8 arguments (14 given)

This is my profile:

{'driver': 'JPEG', 'dtype': 'uint8', 'nodata': None, 'width': 1000, 'height': 1000, 'count': 3, 'crs': CRS.from_wkt('PROJCS["unknown",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Lambert_Azimuthal_Equal_Area"],PARAMETER["latitude_of_center",-90],PARAMETER["longitude_of_center",0],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1],AXIS["Easting",NORTH],AXIS["Northing",NORTH]]'), 'transform': Affine(25.610955512797485, 0.0, -2357528.4841969036,
       0.0, -25.610955512797485, 1593925.6448274788), 'tiled': False, 'compress': 'jpeg', 'interleave': 'pixel', 'photometric': 'ycbcr'}

I'm unsure what fields must be removed for open() to work.


16 bit to 8 bit conversion using rasterio

ashnair0007@...
 

How can I convert a 16 bit tiff to an 8 bit tiff using rasterio. I can do this via gdal_translate so I was wondering if there's a rasterio equivalent?


Re: Merge bug?

Marco
 


Yes, all rasters have nodata set to match the correct nodata value. QGIS correctly mask the nodata zone(transparent).


Re: Merge bug?

Alan Snow
 

One thing to check would be if the nodata values are properly set on the rasters. If they are not, then merging could cause the nodata values to fill in those areas instead of the correct values.


Merge bug?

Marco
 

I want to make a mosaic of satellite images.
I have 7 images clipped each one to fill my BBOX exactly. Loading that images in QGIS i can see that is totally fulfilled and without gaps.

Now, reading these images with rasterio and than merging(files_to_join it's a list of opened images with rasterio.open)

mosaic, out_trans = merge(files_to_join, precision=50)

and saving it with out_trans and sizes updated, I get:

that has gaps and missing pieces.

Why this is happening? (Same CRS and same ZONE)


Rasterio 1.2.0

Sean Gillies
 

Hi all,

A source distribution and binary wheels for manylinux1 and macosx are on PyPI now: https://pypi.org/project/rasterio/1.2.0/#files. These wheels include PROJ 7.2.1 and GDAL 3.2.1 and no longer include any PROJ datum grids. See https://proj.org/usage/network.html#how-to-enable-network-capabilities for instructions on how to get grid data from the PROJ CDN. There are no wheels for Python versions 2.7 and 3.5 because Rasterio 1.2 only supports Python versions 3.6 and up. Wheels for Windows are not yet available on PyPI.

The change log https://github.com/mapbox/rasterio/blob/master/CHANGES.txt has a full list of new features and bugs fixed since 1.1.8. Thanks to all of you who contributed to this version!

--
Sean Gillies


rasterio 1.2b4

Sean Gillies
 

Hi all,

I think this may be it, the last pre-1.2.0 release. Would love to know if you find any issues with it.

What a day!

--
Sean Gillies


rio-color 1.0.2

Sean Gillies
 

Hi all,

Version 1.0.2 of rio-color is on PyPI today. It has one bug fix, a change to the rasterio requirement's version specifier which filters out pre-releases. Installing rio-color 1.0.1 could entail one of the rasterio pre-releases. Thank you, Vincent Sarago and Kyle Barron, for pointing out the problem.

--
Sean Gillies


Re: reading with a window doesn't honour window shape

Loïc Dutrieux
 

Note that setting out_shape to (1200, 1200) will likely read a (1200, 1199) shaped chunk of data and resample it to (1200, 1200); which is not exactly the same as reading a 1200*1200 window.

Kind regards,
Loïc
________________________________________
From: main@rasterio.groups.io <main@rasterio.groups.io> on behalf of Paolo Corti <pcorti@gmail.com>
Sent: 15 January 2021 20:43:21
To: main@rasterio.groups.io
Subject: Re: [rasterio] reading with a window doesn't honour window shape

Thanks Guillaume, that did the trick :)
Would also love to know the reason why that is needed.
best
Paolo

On Fri, Jan 15, 2021 at 1:53 PM Guillaume Lostis <guillaume@lostis.com<mailto:guillaume@lostis.com>> wrote:
Hi,

If you add `out_shape=(1200, 1200)` to your `read()` call it returns an array with the right shape. I don't know exactly why you need to specify this though.

Best,

Guillaume

Le ven. 15 janv. 2021 à 17:48, Paolo Corti <pcorti@gmail.com<mailto:pcorti@gmail.com>> a écrit :
Hi all

I have the following problem: I want to read a dataset using a window with a specific shape, but the window shape isn't always honored from the returned array.

You can replicate this problem using this dataset: https://www.worldpop.org/geodata/summary?id=6334<https://urldefense.com/v3/__https://www.worldpop.org/geodata/summary?id=6334__;!!DOxrgLBm!R8DE-M0LoCdGJkxHEE7d81JPP8UIGkAXjLZ_GJL5tBBF3QRGWwWHOBaSpab9fGiBA4Bpvcs$>

with rasterio.open(raster_pop_path) as src_pop:
window = Window(col_off=1470.4999917338748, row_off=2902.4961942222235, width=1200, height=1200)
pop_arr = src_pop.read(1, window=window)
print(pop_arr.shape)

print returns (1200, 1199) while it should be (1200, 1200)

Any idea why this could happen?
Thanks in advance

Paolo

--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net<https://urldefense.com/v3/__http://www.paolocorti.net__;!!DOxrgLBm!R8DE-M0LoCdGJkxHEE7d81JPP8UIGkAXjLZ_GJL5tBBF3QRGWwWHOBaSpab9fGiBh8FrJuw$>
twitter: @capooti
skype: capooti
#drt3jc1


--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net<https://urldefense.com/v3/__http://www.paolocorti.net__;!!DOxrgLBm!R8DE-M0LoCdGJkxHEE7d81JPP8UIGkAXjLZ_GJL5tBBF3QRGWwWHOBaSpab9fGiBh8FrJuw$>
twitter: @capooti
skype: capooti
#drt3jc1


Re: reading with a window doesn't honour window shape

Paolo Corti
 

Thanks Guillaume, that did the trick :)
Would also love to know the reason why that is needed.
best
Paolo

On Fri, Jan 15, 2021 at 1:53 PM Guillaume Lostis <guillaume@...> wrote:
Hi,

If you add `out_shape=(1200, 1200)` to your `read()` call it returns an array with the right shape. I don't know exactly why you need to specify this though.

Best,

Guillaume

Le ven. 15 janv. 2021 à 17:48, Paolo Corti <pcorti@...> a écrit :
Hi all

I have the following problem: I want to read a dataset using a window with a specific shape, but the window shape isn't always honored from the returned array.

You can replicate this problem using this dataset: https://www.worldpop.org/geodata/summary?id=6334

with rasterio.open(raster_pop_path) as src_pop:
window = Window(col_off=1470.4999917338748, row_off=2902.4961942222235, width=1200, height=1200)
pop_arr = src_pop.read(1, window=window)
print(pop_arr.shape)

print returns (1200, 1199) while it should be (1200, 1200)

Any idea why this could happen?
Thanks in advance

Paolo

--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
#drt3jc1



--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
#drt3jc1


Re: reading with a window doesn't honour window shape

Guillaume Lostis
 

Hi,

If you add `out_shape=(1200, 1200)` to your `read()` call it returns an array with the right shape. I don't know exactly why you need to specify this though.

Best,

Guillaume

Le ven. 15 janv. 2021 à 17:48, Paolo Corti <pcorti@...> a écrit :
Hi all

I have the following problem: I want to read a dataset using a window with a specific shape, but the window shape isn't always honored from the returned array.

You can replicate this problem using this dataset: https://www.worldpop.org/geodata/summary?id=6334

with rasterio.open(raster_pop_path) as src_pop:
window = Window(col_off=1470.4999917338748, row_off=2902.4961942222235, width=1200, height=1200)
pop_arr = src_pop.read(1, window=window)
print(pop_arr.shape)

print returns (1200, 1199) while it should be (1200, 1200)

Any idea why this could happen?
Thanks in advance

Paolo

--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
#drt3jc1


reading with a window doesn't honour window shape

Paolo Corti
 

Hi all

I have the following problem: I want to read a dataset using a window with a specific shape, but the window shape isn't always honored from the returned array.

You can replicate this problem using this dataset: https://www.worldpop.org/geodata/summary?id=6334

with rasterio.open(raster_pop_path) as src_pop:
window = Window(col_off=1470.4999917338748, row_off=2902.4961942222235, width=1200, height=1200)
pop_arr = src_pop.read(1, window=window)
print(pop_arr.shape)

print returns (1200, 1199) while it should be (1200, 1200)

Any idea why this could happen?
Thanks in advance

Paolo

--
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
#drt3jc1


Re: gdal_proximity

Spencer Gardner
 

Thanks. I'll give that a go.


On Wed, Jan 13, 2021 at 7:45 AM Sean Gillies <sean.gillies@...> wrote:
Hi Spencer,

On Mon, Jan 4, 2021 at 1:21 PM Spencer Gardner <spencergardner@...> wrote:
Hi all. I'm using Rasterio for a raster analysis but I need to calculate proximity as is done in gdal_proximity.

Is this implemented somewhere in Rasterio? I don't see anything like it in the docs. Or if not, is there a way to run the GDAL operation on the array? Or even better, I could run a numpy operation on it, but I'm not finding anything in numpy that is suitable.

As an example, I am rasterizing a line feature that represents a segment of road. My analysis needs to account for proximity to the road such that cells touched by the road or immediately adjacent are weighted higher than roads that are 500 ft away. My intent with the proximity analysis was to use results to generate these weights.

Any pointers are much appreciated. Thanks!

Spencer

You're not overlooking anything. The rasterio project doesn't surface GDALComputeProximity. There are pointers to equivalent functionality in scipy at https://github.com/mapbox/rasterio/issues/531.

Hope this helps,

--
Sean Gillies


Re: Rasterio installation on Amazon/AWS/EC2/notebook instance

Sean Gillies
 

Hi,

On Wed, Jan 13, 2021 at 7:33 AM <robmarkcole@...> wrote:
I am attempting to install rasterio on an AWS notebook instance (essentially managed EC2 with Jupyter installed).
This should be straightforward but I have already sunk several hours without success. 

The instance has a bunch of conda environments available so I wish to install into one named python3.
The following results:

sh-4.2$ conda install -c conda-forge rasterio --name python3
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: - 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
 
  - defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  - defaults/linux-64::python-language-server==0.31.7=py36_0
  - defaults/linux-64::nb_conda==2.2.1=py36_0
  - defaults/noarch::numpydoc==0.9.2=py_0
  - defaults/noarch::anaconda-project==0.8.4=py_0
  - defaults/noarch::boto3==1.9.162=py_0
  - defaults/linux-64::s3transfer==0.2.1=py36_0
  - defaults/linux-64::nbconvert==5.6.1=py36_0
  - defaults/linux-64::bokeh==1.4.0=py36_0
  - defaults/noarch::jupyterlab_server==1.0.6=py_0
  - defaults/noarch::botocore==1.12.189=py_0
  - defaults/linux-64::jupyter==1.0.0=py36_7
  - defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  - defaults/linux-64::imageio==2.6.1=py36_0
  - defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  - defaults/linux-64::spyder==4.0.1=py36_0
  - defaults/linux-64::requests==2.22.0=py36_1
  - defaults/noarch::dask==2.11.0=py_0
  - defaults/noarch::ipywidgets==7.5.1=py_0
  - defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  - defaults/noarch::s3fs==0.4.2=py_0
  - defaults/linux-64::notebook==6.0.3=py36_0
  - defaults/linux-64::anaconda-client==1.7.2=py36_0

At this point conda hangs. I also tried running:

$ conda update all

And repeating the process, but with the same result.
Any guidance greatly appreciated

I don't know anything about AWS notebook instances and am not a regular conda user, so my help might be a little sketchy. In my experience, one cannot incrementally upgrade or install new packages into a conda environment without breaking it. Instead, create a completely new environment, and expect that to be resolved properly.

--
Sean Gillies


Re: gdal_proximity

Sean Gillies
 

Hi Spencer,

On Mon, Jan 4, 2021 at 1:21 PM Spencer Gardner <spencergardner@...> wrote:
Hi all. I'm using Rasterio for a raster analysis but I need to calculate proximity as is done in gdal_proximity.

Is this implemented somewhere in Rasterio? I don't see anything like it in the docs. Or if not, is there a way to run the GDAL operation on the array? Or even better, I could run a numpy operation on it, but I'm not finding anything in numpy that is suitable.

As an example, I am rasterizing a line feature that represents a segment of road. My analysis needs to account for proximity to the road such that cells touched by the road or immediately adjacent are weighted higher than roads that are 500 ft away. My intent with the proximity analysis was to use results to generate these weights.

Any pointers are much appreciated. Thanks!

Spencer

You're not overlooking anything. The rasterio project doesn't surface GDALComputeProximity. There are pointers to equivalent functionality in scipy at https://github.com/mapbox/rasterio/issues/531.

Hope this helps,

--
Sean Gillies


Rasterio installation on Amazon/AWS/EC2/notebook instance

robmarkcole@...
 

I am attempting to install rasterio on an AWS notebook instance (essentially managed EC2 with Jupyter installed).
This should be straightforward but I have already sunk several hours without success. 

The instance has a bunch of conda environments available so I wish to install into one named python3.
The following results:

sh-4.2$ conda install -c conda-forge rasterio --name python3
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: - 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
 
  - defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  - defaults/linux-64::python-language-server==0.31.7=py36_0
  - defaults/linux-64::nb_conda==2.2.1=py36_0
  - defaults/noarch::numpydoc==0.9.2=py_0
  - defaults/noarch::anaconda-project==0.8.4=py_0
  - defaults/noarch::boto3==1.9.162=py_0
  - defaults/linux-64::s3transfer==0.2.1=py36_0
  - defaults/linux-64::nbconvert==5.6.1=py36_0
  - defaults/linux-64::bokeh==1.4.0=py36_0
  - defaults/noarch::jupyterlab_server==1.0.6=py_0
  - defaults/noarch::botocore==1.12.189=py_0
  - defaults/linux-64::jupyter==1.0.0=py36_7
  - defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  - defaults/linux-64::imageio==2.6.1=py36_0
  - defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  - defaults/linux-64::spyder==4.0.1=py36_0
  - defaults/linux-64::requests==2.22.0=py36_1
  - defaults/noarch::dask==2.11.0=py_0
  - defaults/noarch::ipywidgets==7.5.1=py_0
  - defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  - defaults/noarch::s3fs==0.4.2=py_0
  - defaults/linux-64::notebook==6.0.3=py36_0
  - defaults/linux-64::anaconda-client==1.7.2=py36_0

At this point conda hangs. I also tried running:

$ conda update all

And repeating the process, but with the same result.
Any guidance greatly appreciated

121 - 140 of 828