|
Rasterio's general discussion group is open for business
Sean Gillies
Greetings, Questions about installation,
distribution, and usage of Rasterio are welcome here. Issues opened in
Rasterio's GitHub repo that fall into one these categories may be
perfunctorily closed. Note well: the Rasterio project's code of conduct, https://github.com/mapbox/ Yours, -- Sean Gillies
|
|
|
|
Rasterio 1.0rc3
Sean Gillies
Hi all, I almost forgot to announce here before the holiday! The Rasterio 1.0rc3 source distribution and wheels for macos 10.9 (or greater) and manylinux1 platforms are on PyPI now: https://pypi.org/project/rasterio/1.0rc3/. Please remember that pip install -U rasterio will install Rasterio 0.36, the last "stable" release, and that you need to execute either pip install rasterio==1.0rc3 or pip install -U --pre rasterio to get the latest pre-release. Packages for the Anaconda platform (Linux, OS X, and Windows) should be available in the conda-forge channel (under the "dev" label) very soon. The final 1.0.0 release is tentatively scheduled for Tuesday, July 10. -- Sean Gillies
|
|
|
|
Does rasterio load data into memory while reprojecting?
Denis Rykov
Hi there!
I have a general question about reprojection. Consider the following example: with rasterio.open(fname_src) as src: with rasterio.open(fname_dst, 'w', **kwargs) as dst:
for i in range(1, src.count + 1):
rasterio.warp.reproject(
source=rasterio.band(src, i),
destination=rasterio.band(dst, i),
src_transform=src.transform,
src_crs=src.crs,
dst_transform=dest_affine,
dst_crs=dst_crs,
resampling=resampling) As I can see from source code rasterio reads each band as an N-d array. Does it mean that rasterio loads each band into memory? My main question: can I use this code for warping rasters of arbitrary size (or I'm limited by memory size)? Thanks.
|
|
|
|
Re: Does rasterio load data into memory while reprojecting?
Sean Gillies
Hi Denis, On Wed, Jul 4, 2018 at 4:09 AM, Denis Rykov <rykovd@...> wrote: Hi there! Yes, you can warp arbitrarily large rasters when you use rasterio.band(). The warper chunks the work so that its working arrays are no more than 64 MB in size. You can increase this limit with reproject's warp_mem_limit keyword argument. The source dataset's bands are incrementally loaded into memory as the warper works, but only up to the limit of GDAL's block cache (Rasterio uses the GDAL C library). If you profile your program and see the memory allocation increase, this is due to GDAL's caching. You can see also the notes at http://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp#WarpandCacheMemory:TechnicalDetails, which apply to Rasterio.Please note that you can warp multiple imagery bands in one reproject() call by passing sequences of band indexes. reproject(source=rasterio.band(src, src.indexes), destination=rasterio.band(src, dst.indexes), ...) This is faster than looping over the bands. -- Sean Gillies
|
|
|
|
Re: Does rasterio load data into memory while reprojecting?
Denis Rykov
Thank you very much, Sean, for such detailed reply!
|
|
|
|
Rasterio 1.0rc4
Sean Gillies
Hi all, The Rasterio 1.0rc4 source distribution and wheels for macos 10.9 (or greater) and manylinux1 platforms are on PyPI now: https://pypi.org/project/rasterio/1.0rc4/. Please remember that pip install -U rasterio will install Rasterio 0.36, the last "stable" release, and that you need to execute either pip install rasterio==1.0rc4 or pip install -U --pre rasterio to
get the latest pre-release. Packages for the Anaconda platform (Linux,
OS X, and Windows) should be available in the conda-forge channel (under
the "dev" label) very soon. The final 1.0.0 release is now tentatively scheduled for Tuesday, July 12. -- Sean Gillies
|
|
|
|
Rasterio 1.0rc5
Sean Gillies
Hi all, The Rasterio 1.0rc5 source distribution and wheels for macos 10.9 (or greater) and manylinux1 platforms are on PyPI now: https://pypi.org/project/ Please remember that pip install -U rasterio will install Rasterio 0.36, the last "stable" release, and that you need to execute either pip install rasterio==1.0rc5 or pip install -U --pre rasterio to
get the latest pre-release. Packages for the Anaconda platform (Linux,
OS X, and Windows) should be available in the conda-forge channel (under
the "dev" label) very soon. The final 1.0.0 release is still scheduled for Thursday, July 12. I accidentally wrote Tuesday in the 1.0rc4 announcement. -- Sean Gillies
|
|
|
|
New Member
Armstrong Manuvakola Ezequias Ngolo
Hello everyone,
I’m a new member of rasterio group willing to know more about rasterio and satellite image processing. I’m also new in programming, so, don’t be surprised if I make weird questions!!!
P.S: non native English speaker!!!!
Pleasure to be in this group and willing to hear from you.
Armstrong Ngolo
Sent from Mail for Windows 10
|
|
|
|
Plugin for common color maps?
me@...
Hey there,
Thanks for a great library - I've just been getting into it via the rio-tiler library where I'm building an app for showing imagery in combination with stream gauge data (see below). What I'd like to do is apply a suitable colormap to my tile to match the Water Index that I'm displaying which I'm sure this is doable with rasterio (he says without having tried it). Would it make sense to perhaps package this sort of thing as a rio plugin, eg a plugin containing standard color ramps for commonly used indices? Have I missed anything existing anywhere and would this make sense and be useful? Cheers Rowan
|
|
|
|
Rasterio 1.0.0
Sean Gillies
Hi, all. What is the significance of 1.0.0? Stability. After more than a year of changes, there is at last a stable base for applications. You can pin `rasterio ~= 1.0` in your project’s requirements and enjoy nothing but bug fixes for as long as the project supplies them. Highlights Many new features have been added since the last stable release (0.36). Especially notable are the following. - All new documentation at https://rasterio.readthedocs.io/en/stable/. - Binary wheels with GDAL 2.x included for the macOS and manylinux1 platforms are available on pypi.org. - A new Window class with floating point origin and offsets has been added to help with windowed dataset operations. - BytesIO-like MemoryFile and ZipMemoryFile classes that support access to in-memory datasets. - A WarpedVRT class exposes GDAL’s warp-on-demand VRT features. - Support for georeferencing by ground control points has been added. - The rasterio.shutil module provides many of the same features as Python’s shutil, but also knows about sidecar files (masks, overviews, metadata). Upgrading and compatibility Rasterio is compatible with GDAL versions 1.11-2.3 and Python versions 2.7, and 3.4-3.7. We have deprecated a number of features since 0.36. Features have been removed, after some warning, at 1.0a1, 1.0a10, 1.0b1, and 1.0rc2. We recommend migrating stepwise through those tags to get from 0.36 to 1.0.0 if you're feeling extra cautious. Support The primary channel for installation and usage support is the Rasterio user discussion group at https://rasterio.groups.io/g/main. Please see https://github.com/mapbox/rasterio/blob/master/CONTRIBUTING.rst for guidance on reporting bugs or proposing new features. Acknowledgements Rasterio 1.0.0 is the work of 68 authors. The individuals are listed in https://github.com/mapbox/rasterio/blob/master/AUTHORS.txt. This number does not include people who have created and commented on issues in the project’s issue tracker, but their contributions are also very important. We’ve received some of the best bug reports ever written. Rasterio benefits from a userbase in many different areas of work and study. Advocates across companies, classrooms, and projects have helped make Rasterio what it is. We particularly thank people who help distribute it, and people who help teach others to use it. Among those are: Christoph Gohlke, the conda-forge and Debian GIS teams, Howard Butler, Sara Safavi, Dana Bauer, Leah Wasser, Chris Holdgraf, and everyone else who has taught Rasterio in a workshop or course, or presented Rasterio to a conference or meetup audience. Rasterio has benefited from helpful folks on the GIS StackExchange. Luke Pinner, Loïc Dutrieux, Martin Laloux, and Kersten Clauss are foremost among them. Everything we know about building and distributing binary wheels comes from the SciPy community. Thank you, Matthew Brett, Nathaniel Smith, et al., for writing the delocate and auditwheel tools and supporting them. Even Rouault, GDAL’s maintainer, has been our patient guru and guide during many difficult passages. Early adoption by engineering teams in the satellite imagery business has also been key to Rasterio’s success. The project is indebted to developers who tried it and managers who supported them. Thank you all! -- Sean Gillies
|
|
|
|
Calculate transformation using output dimensions
Denis Rykov
In gdalwarp we have at least two options for managing of output size of a raster:
But how I can calculate transformation (destination affine first of all) if I know output dimensions (width and height)? In other words I'm looking for how I can achieve the same result with rasterio as with gdalwarp ts option. I hope what I have requested is possible.
|
|
|
|
Re: Calculate transformation using output dimensions
Alan Snow
Looks like you are most of the way there.
If you have a dataset open with `rasterio.open`, then you should be able to get the information you need for the input: with rasterio.open('/path/do/dataset.tif') as rds: left, bottom, right, top = rds.bounds src_crs = rds.crs width = rds.width height = rds.height
|
|
|
|
Re: Calculate transformation using output dimensions
Alan Snow
This may be what you are looking for as well:
rasterio.transform.from_bounds
|
|
|
|
Re: Calculate transformation using output dimensions
Denis Rykov
Thanks for reply.
I could not find what I need in rasterio and have created PR: https://github.com/mapbox/rasterio/pull/1409
|
|
|
|
LERC Compression Error
calebrob6@...
Hello all,
I have a raster that uses LERC compression. Opening/reading from the raster works fine, however I get the following error when trying to access the `.profile` property: ValueError: 'LERC' is not a valid Compression I've found that simply adding `lerc = 'LERC'` to the Compression enum in rasterio/enums.py solves this problem. Is this the correct fix and if so, would you like me to open a pull request for it? Thanks, Caleb
|
|
|
|
Copying Rasterio crs from input to output dataset
Luke
Hi all. Congrats on the 1.0.0 release! Simplified example demonstrating: import rasterio as rio inpath = r"test.tif" outpath = r"testout.tif" with rio.open(inpath) as inds: print(repr(inds.crs), inds.meta['crs']) with rio.open(outpath, 'w', **inds.meta) as outds: outds.write(inds.read()) with rio.open(outpath) as outds: print(repr(outds.crs), outds.meta['crs']) The output of the above is: CRS({'init': 'epsg:4326'}) +init=epsg:4326 None None Dataset info: rio info test.tif {"blockxsize": 128, "blockysize": 128, "bounds": [143.22468138068513, -34.72000041269073, 146.20968138068514, -31.952500412690732], "colorinterp": ["grey", "undefined", "undefined", "undefined", "undefined", "undefined", "undefined", "undefined", "undefined", "undefined", "undefined"], "count": 11, "crs": "EPSG:4326", "descriptions": [null, null, null, null, null, null, null, null, null, null, null], "driver": "GTiff", "dtype": "float32", "height": 1107, "indexes": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "interleave": "pixel", "lnglat": [144.71718138068513, -33.33625041269073], "mask_flags": [["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"], ["nodata"]], "nodata": -9999.0, "res": [0.0025000000000000113, 0.0024999999999999983], "shape": [1107, 1194], "tiled": true, "transform": [0.0025000000000000113, 0.0, 143.22468138068513, 0.0, -0.0024999999999999983, -31.952500412690732, 0.0, 0.0, 1.0], "units": [null, null, null, null, null, null, null, null, null, null, null], "width": 1194} Is this user error (do I need to pass a crs string in?) or perhaps a bug? Regards Luke
|
|
|
|
Re: LERC Compression Error
Sean Gillies
Hi Caleb, Sure, a PR would be welcome. This compression algorithm isn't available in a GDAL release yet, but it doesn't do any harm to have it enumerated in Rasterio. Behavior for Rasterio 1.0.0 and GDAL 2.2.4 is this: $ rio convert tests/data/RGB.byte.tif /tmp/lerc.tif --co compress=LERC WARNING:rasterio._gdal:CPLE_NotSupported in 'LERC' is an unexpected value for COMPRESS creation option of type string-select. WARNING:rasterio._gdal:CPLE_IllegalArg in COMPRESS=LERC value not recognised, ignoring.
On Wed, Jul 18, 2018 at 7:45 PM, <calebrob6@...> wrote: Hello all, --
Sean Gillies
|
|
|
|
Re: Copying Rasterio crs from input to output dataset
Sean Gillies
Hi Luke, Unfortunately, I cannot reproduce your results on my computer. $ python Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import rasterio >>> with rasterio.open('tests/data/RGB.byte.tif') as src: ... with rasterio.open('/tmp/foo.tif', 'w', **src.meta) as dst: ... dst.write(src.read()) ... >>> with rasterio.open('/tmp/foo.tif') as src: ... print(src.meta) ... print(src.profile) ... {'driver': 'GTiff', 'dtype': 'uint8', 'nodata': 0.0, 'width': 791, 'height': 718, 'count': 3, 'crs': CRS({'init': 'epsg:32618'}), 'transform': Affine(300.0379266750948, 0.0, 101985.0, 0.0, -300.041782729805, 2826915.0)} {'driver': 'GTiff', 'dtype': 'uint8', 'nodata': 0.0, 'width': 791, 'height': 718, 'count': 3, 'crs': CRS({'init': 'epsg:32618'}), 'transform': Affine(300.0379266750948, 0.0, 101985.0, 0.0, -300.041782729805, 2826915.0), 'tiled': False, 'interleave': 'pixel'} I wonder if you may have a problem with unresolved Proj data. To debug this, turn on Python's logging facility and set the level to DEBUG at the top of your code like so: import logging logging.basicConfig(level=logging.DEBUG)
On Thu, Jul 19, 2018 at 11:24 PM, Luke Pinner <lukepinnerau@...> wrote:
--
Sean Gillies
|
|
|
|
Reproject with mask preserving
Denis Rykov
Hi there!
I have GeoTIFF files with internal mask and I need to reproject them and not lose mask. I've found that direct generating warped mask band is not supported by GDAL: https://github.com/OSGeo/gdal/issues/689 rasterio.warp.reproject() also doesn't preserve mask. How I can use rasterio for achieving what I need? I'm looking for something similar described in this issue but with rasterio.
|
|
|
|
Re: Copying Rasterio crs from input to output dataset
Luke
My GDAL_DATA env var wasn't set.
Explanation: I was running the code within the PyCharm IDE that supposedly supports conda envs but doesn't actually *activate* them properly. There are init scripts in CONDA_PREFIX/etc/conda/activate.d that get run when you activate/source activate a conda env that do some additional environment setup and there's a gdal one that should've been run but wasn't. Apologies for the noise. Luke
|
|
|
