Re: metadata and profile: What's the difference?
Amine Aboufirass <amine.aboufirass@...>
Thanks for your reply. I guess the profile is more comprehensive then. Is that a good argument for dropping metadata then?
On Fri, Nov 29, 2019 at 4:11 PM Sean Gillies <sean.gillies@...> wrote:
|
|
Re: metadata and profile: What's the difference?
Sean Gillies
Indeed they are largely redundant. The meta property came first. Then we introduced a profile property to better support the pattern of using one dataset's profile as keyword arguments for creating new similar datasets. Note that the profile may contain items that don't appear in the meta property.
On Fri, Nov 29, 2019, 9:57 AM Amine Aboufirass <amine.aboufirass@...> wrote:
|
|
metadata and profile: What's the difference?
Amine Aboufirass <amine.aboufirass@...>
Also, what is the difference between a dataset's metadata and profile? I have the impression one of these values is redundant. Please see example below: >>> dataset.profile {'driver': 'AAIGrid', 'dtype': 'int32', 'nodata': -9999.0, 'width': 8, 'height': 8, 'count': 1, 'crs': None, 'transform': Affine(1.0, 0.0, 0.0, 0.0, -1.0, 8.0), 'tiled': False} >>> dataset.meta {'driver': 'AAIGrid', 'dtype': 'int32', 'nodata': -9999.0, 'width': 8, 'height': 8, 'count': 1, 'crs': None, 'transform': Affine(1.0, 0.0, 0.0, 0.0, -1.0, 8.0)}
|
|
Re: Geodatabase, FileGeodatabase, and Esri Geodatabase
Sean Gillies
Hi Armstrong, Rasterio relies on a library named GDAL for data access and GDAL doesn't support Esri raster databases. There are people in the open source community who store raster data in a PostGIS database and manipulate it using SQL. I don't have any experience with that. My own applications use GeoTIFF and numpy arrays for storage and all the manipulation is done using the numpy and rasterio APIs, in python. Open Data Cube is one project that aims to provide a more comprehensive raster data management system. It may be worth your time to look at it. I'm sure subscribers on this list can recommend other systems. Yours,
On Thu, Nov 28, 2019, 10:14 PM Armstrong Manuvakola Ezequias Ngolo <arngolo@...> wrote: Hi everyone,
|
|
Problem with AAIGrid overwrite
Amine Aboufirass <amine.aboufirass@...>
Dear All, For one project I am reading and writing these test raster datasets as ASCII grids. I have found that overwriting these grids is impossible and I get the following error: ERROR 1: Deleting test_data/out.asc failed: Permission denied This is quite annoying because for my testing I need to overwrite and delete files repeatedly. I have to keep deleting them again before I run my application each time and this is quite time consuming. I have the impression this is related to the GDAL creation options but I am not sure. Could someone please enlighten me? Thanks. Regards, Amine
|
|
Geodatabase, FileGeodatabase, and Esri Geodatabase
Armstrong Manuvakola Ezequias Ngolo
Hi everyone,
After relying on GIS softwares for a long time with a bit of python help with rasterio, I decided to move on using rasterio (and geopandas) for AI applications. I would like to have access to some common tools that I have been using in GIS softwares!!!. One important thing is the ability to store and manipulate the datasets using only one file such as the Esri FileGeodatabase (FileGDB). My question is: Are there any FileGDBs that we can use in Python to store and manipulate spatial data? I found that Fiona can create, read and write FileGDB but only for storing shapefile kind of datasets. thank you in advance, Armstrong Ngolo
|
|
Re: reading raster with masked=True leads to error
Eyal Saiet
This is quite embarrassing, I cannot believe I did not catch that myself. The link is very useful thanks!
On Fri, Nov 15, 2019 at 12:32 AM Guillaume Lostis <g.lostis@...> wrote:
-- Eyal Saiet The mind is not a vessel to be filled, but a fire to be kindled. Plutarch
|
|
Re: reading raster with masked=True leads to error
Guillaume Lostis
Hi, I think the reason you get an error is that you have misspelled the argument (you wrote FYI, the list of arguments of the Best, Guillaume Lostis
|
|
reading raster with masked=True leads to error
Eyal Saiet
Hello, I wanted to try using read(1,masked=True) on an imported DEM, but I get the following error below: Since I could not verify what arguments the read() method accepts, I am asking here. thanks -- Eyal Saiet
|
|
Rasterio 1.1.1
Sean Gillies
Hi all, Rasterio 1.1.1 source distribution and wheels for macosx and manylinux1 are on PyPI now. Here are the changes from the change log: Bug fixes: - Calling a dataset's sample method with coordinates outside the extent of a dataset with a nodata value of None has raised a TypeError (#1822). Now, it gives the values we would get from a boundless read of the dataset's values. - Use new set_proj_search_path() function to set the PROJ data search path. For GDAL versions before 3.0 this sets the PROJ_LIB environment variable. For GDAL version 3.0 this calls OSRSetPROJSearchPaths(), which overrides PROJ_LIB (#1823). - Check for header.dxf file instead of pcs.csv when looking for installed GDAL data. The latter is gone with GDAL 3.0 but the former remains (#1823). - RGB rasters are now properly displayed by rasterio.plot.show (#1650). Notes: - The wheels on PyPI include GDAL 2.4.3 with a patch that addresses the multithreading issue reported in #1828. Some of you have been reporting trouble with concurrent reads of GeoTIFFs on S3 and I'm optimistic that we have a fix for at least one class of these problems. Please give the 1.1.1 wheels a try (or patch your own installations of GDAL) and let us know how it goes. Sean Gillies
|
|
Re: reproject from a tiff to another tiff gives "destination band 1 appears to be read only"
Sean Gillies
Hi, It's a good thing that you opened the destination (population) file in "r" mode. Otherwise, it could have been overwritten. You must create an output array for the reprojected data and pass it as the second argument, along with dst_crs and dst_transform keyword arguments. with rasterio.open("population") as population, rasterio.open("lights") as lights: reprojected = numpy.zeros(population.shape, dtype=lights.dtypes[0]) reprojected, transform = rasterio.warp.reproject(rasterio.band(lights, 1), reprojected, dst_crs=population.crs, dst_transform=population.transform) The reprojected array will be aligned with and have the same shape as lights.read(1).
On Mon, Nov 11, 2019 at 6:28 AM <simonm3@...> wrote: Please can someone explain how I align a tiff file of lights with another of population to create two numpy arrays the same size and aligned. I tried: -- Sean Gillies
|
|
reproject from a tiff to another tiff gives "destination band 1 appears to be read only"
simonm3@...
Please can someone explain how I align a tiff file of lights with another of population to create two numpy arrays the same size and aligned. I tried:
population = rio.open(population) lights = rio.open(lights) res = rio.warp.reproject(rio.band(lights,1), rio.band(population,1)) This gives "CPLE_IllegalArgError: Destination band 1 appears to be read-only." I was expecting it to read the meta data (crs, height, width and affine transform) from source and destination then return something with the data of source but the shape of destination.
|
|
Re: DataReader.crs is empty with new environment?
Denise Draper
Yup, that is it: the environment variables are not set. I may have set them manually before. I'll have to investigate how conda is supposed to handle this (since the variables should only be set when you are in the right environment), but in the meantime it seems to work to set them manually. -- Denise Draper draperd@...
|
|
Re: DataReader.crs is empty with new environment?
Is your GDAL_DATA and/or PROJ_LIB environment variables are set in your new windows 10 environment? Typically these are set for you when you create a new conda environment and install GDAL. Is it possible that they are set but are somehow not readable? Have you tried opening another image and reading the crs? does it happen only with this one specific image?
|
|
DataReader.crs is empty with new environment?
Denise Draper
Hi ---
I just got a new (Windows 10) laptop, and recreated my software environment from my old laptop. Code that had previously worked didn't. In particular, on opening a file, I find the crs field is empty: import rasterio foo = rasterio.open('path_to_file.tif') foo.crs # returns nothing I assume it is some sort of version difference at fault. However, I have tried upgrading on an old machine and downgrading on the new one, and neither helped. So far I've only looked at the rasterio version and the gdal version. What else should I be looking at? I use miniconda to create the software environment if that matters. Here's a summary of version combinations that I tried Old Windows 10 machine rasterio version 1.0.22 gdal version 2040100 ==> crs is CRS.from_epsg(32629) Cloud Linux VM rasterio version 1.0.28 gdal version 3000100 ==> crs is CRS.from_epsg(32629) rasterio version 1.1.00 gdal version 3000100 ==> crs is CRS.from_epsg(32629) New Windows 10 machine rasterio version 1.1.00 gdal version 3000100 ==> crs is NULL rasterio version 1.0.28 gdal version 3000100 ==> crs is NULL Also: I've independently verified (with gdalinfo) that the data file does have a proper CRS, which is the same from all machines.
|
|
Re: setting BIGTIFF='IF_SAFER' not solving BigTiff problems
Luke
How did you install rasterio?
If you installed via Anaconda / conda using the default channel, your gdal may lack bigtiff support. https://github.com/ContinuumIO/anaconda-issues/issues/9887
|
|
setting BIGTIFF='IF_SAFER' not solving BigTiff problems
Amine Aboufirass <amine.aboufirass@...>
Dear All, I would like to point to this issue in which I brought up the BIGTIFF limitation with regards to merging rasters using rasterio: It turns out that the suggested solution of adding BIGTIFF="IF_SAFER" to the rasterio.open arguments does not remove the problem. I still get an error: rasterio._err.CPLE_NotSupportedError: A 140401 pixels x 36001 lines x 1 bands Int16 image would be larger than 4GB but this is the largest size a TIFF can be, and BigTIFF is unavailable. Creation failed. Is there a way to circumvent or workaround this? Why is BIGTIFF="IF_SAFER" not working? Thanks, Amine
|
|
Re: Clarification on rio bounds --sequence/--collection
Sean Gillies
Hi, I agree, it is confusing. And there is a bug here. The --sequence/--collection switch was intended to allow printing a sequence of JSON texts (GeoJSON feature or bbox) or a single GeoJSON feature collection. But the latter feature is disabled by the reuse of --collection to determined the type of JSON output.
On Thu, Oct 17, 2019 at 10:44 PM Yann-Sebastien Tremblay-Johnston <yanns.tremblay@...> wrote:
-- Sean Gillies
|
|
Re: masking using another raster
Sean Gillies
Hi Matt, Yes, I think that's the only builtin command that will do it. Enhancing rio-stack to more clearly provide this feature might be feasible. Note that few formats support addition of a mask and that adding a mask generally requires creation of a new file.
On Fri, Oct 11, 2019 at 3:12 PM Gregory, Matthew <matt.gregory@...> wrote: Hi all, -- Sean Gillies
|
|
Re: reading hdf4 files with rasterio
Sean Gillies
Rasterio doesn't improve the situation with subdatasets very much. You would still need to discover them via a rasterio dataset's subdatasets property. Something like with rasterio.open("file.hdf4") as dataset: for name in dataset.subdatasets: with rasterio.open(name) as subdataset: print(subdataset.profile) All the caveats of https://gdal.org/drivers/raster/hdf4.html apply.
On Mon, Oct 21, 2019 at 2:39 AM Amine Aboufirass <amine.aboufirass@...> wrote:
-- Sean Gillies
|
|