|
Re: problem with window sizes when parallelizing a funciton
You are indeed correct Sean. Thanks for catching that and providing the correct answer! It seems I skimmed through the original message too fast.
You are indeed correct Sean. Thanks for catching that and providing the correct answer! It seems I skimmed through the original message too fast.
|
By
Alan Snow
·
#222
·
|
|
Re: problem with window sizes when parallelizing a funciton
Hi Sean, thanks a lot. This was very helpfull indeed, and the performance improved a bit when changing the size to 512.
Grate job on the library, congrats!
Cheers,
Javier
--
Javier Lopatin
Institute
Hi Sean, thanks a lot. This was very helpfull indeed, and the performance improved a bit when changing the size to 512.
Grate job on the library, congrats!
Cheers,
Javier
--
Javier Lopatin
Institute
|
By
javier lopatin
·
#223
·
|
|
Clipping a raster (grib2), end up with zeros on left edge when cropping with mask
Hi everyone,
I am currently clipping a grib2 file, and am ending up with zeros on the left edge of the raster. I'm not an expert with GIS so maybe this is a common problem but was curious if anyone
Hi everyone,
I am currently clipping a grib2 file, and am ending up with zeros on the left edge of the raster. I'm not an expert with GIS so maybe this is a common problem but was curious if anyone
|
By
Shane Mill - NOAA Affiliate
·
#224
·
|
|
Occasional "not recognized as a supported file format" errors when reading from S3
Dear rasterio group,
(I initially posted this at https://github.com/mapbox/rasterio/issues/1686)
Lately, we have encountered a strange bug in Terracotta. It basically always leads to errors like
Dear rasterio group,
(I initially posted this at https://github.com/mapbox/rasterio/issues/1686)
Lately, we have encountered a strange bug in Terracotta. It basically always leads to errors like
|
By
Dion Häfner <dion.haefner@...>
·
#225
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
This is not something I've been seen for now. does this happens with Rasterio wheels and rasterio build on gdal source ?
This is not something I've been seen for now. does this happens with Rasterio wheels and rasterio build on gdal source ?
|
By
vincent.sarago@...
·
#226
·
|
|
Re: Clipping a raster (grib2), end up with zeros on left edge when cropping with mask
Hi Shane,
It looks like your feature geometries extend beyond the extent of your raster, yes? In that case, crop can create an extra row or column of pixels, which will be empty. This seems like a bug
Hi Shane,
It looks like your feature geometries extend beyond the extent of your raster, yes? In that case, crop can create an extra row or column of pixels, which will be empty. This seems like a bug
|
By
Sean Gillies
·
#227
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
We've only tried rasterio wheels.
We've only tried rasterio wheels.
|
By
Dion Häfner <dion.haefner@...>
·
#228
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
Hi Dion,
That error comes from these lines in GDALOpenEx when the function fails to return successfully:
Hi Dion,
That error comes from these lines in GDALOpenEx when the function fails to return successfully:
|
By
Sean Gillies
·
#229
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
Hey Sean,
thank you very much for the detailed assessment. I think this already sheds some light on some of the problems.
To answer your questions:
- The /vsis3/ identifiers are
Hey Sean,
thank you very much for the detailed assessment. I think this already sheds some light on some of the problems.
To answer your questions:
- The /vsis3/ identifiers are
|
By
Dion Häfner <dion.haefner@...>
·
#230
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
I found the VRT multi-threading guidance at https://gdal.org/drivers/raster/vrt.html#multi-threading-issues. It moved during the site migration.
This smells like a GDAL bug to me, perhaps a
I found the VRT multi-threading guidance at https://gdal.org/drivers/raster/vrt.html#multi-threading-issues. It moved during the site migration.
This smells like a GDAL bug to me, perhaps a
|
By
Sean Gillies
·
#231
·
|
|
Re: Clipping a raster (grib2), end up with zeros on left edge when cropping with mask
Hey Sean,
As always, thanks for the feedback. So to follow up, it turns out that I had to add "filled=False". In my particular situation, the user drags a bounding box over the original raster and is
Hey Sean,
As always, thanks for the feedback. So to follow up, it turns out that I had to add "filled=False". In my particular situation, the user drags a bounding box over the original raster and is
|
By
Shane Mill - NOAA Affiliate
·
#232
·
|
|
Re: gdalinfo's and rasterio's reading problem in LUSTRE FS with NetCDF file (ubuntu:bionic)
Hi Erick,
I'm not familiar with Lustre and only slightly familiar with the details of GDAL's netCDF driver. I think, since the problem manifests with gdalinfo as well as rasterio programs, that the
Hi Erick,
I'm not familiar with Lustre and only slightly familiar with the details of GDAL's netCDF driver. I think, since the problem manifests with gdalinfo as well as rasterio programs, that the
|
By
Sean Gillies
·
#235
·
|
|
Re: gdalinfo's and rasterio's reading problem in LUSTRE FS with NetCDF file (ubuntu:bionic)
Hi Sean,
Thank you for your answer.
Even answered that LUSTRE is causing this behaviour and my best option is building GDAL and libnetcdf against a libhdf5 version that plays well with LUSTRE.
In
Hi Sean,
Thank you for your answer.
Even answered that LUSTRE is causing this behaviour and my best option is building GDAL and libnetcdf against a libhdf5 version that plays well with LUSTRE.
In
|
By
Erick Palacios Moreno
·
#236
·
|
|
Rasterio 1.0.24
Hi all,
Rasterio 1.0.24 is on PyPI now: https://pypi.org/project/rasterio/1.0.24/#files. This release fixes a pretty major bug that potentially affected multi-threaded programs or programs that
Hi all,
Rasterio 1.0.24 is on PyPI now: https://pypi.org/project/rasterio/1.0.24/#files. This release fixes a pretty major bug that potentially affected multi-threaded programs or programs that
|
By
Sean Gillies
·
#237
·
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
Here is some more insight on the issue. Remember, we are seeing this when reading small chunks from a cloud-optimized GeoTIFF on S3 with our Terracotta tile server.
On our latest encounters of the
Here is some more insight on the issue. Remember, we are seeing this when reading small chunks from a cloud-optimized GeoTIFF on S3 with our Terracotta tile server.
On our latest encounters of the
|
By
Jonas
·
#238
·
Edited
|
|
Re: Occasional "not recognized as a supported file format" errors when reading from S3
OK, sorry, forget what I said - the same map tile that earlier was failing is now served fine. :/
OK, sorry, forget what I said - the same map tile that earlier was failing is now served fine. :/
|
By
Jonas
·
#239
·
|
|
Re: problem with window sizes when parallelizing a funciton
Hi Guys!
I'm still working with the parallelization of raster processes with rasterio, and I came across a problem that you might know about. I've some raster stacks of time series data from which I
Hi Guys!
I'm still working with the parallelization of raster processes with rasterio, and I came across a problem that you might know about. I've some raster stacks of time series data from which I
|
By
javier lopatin
·
#240
·
|
|
Re: problem with window sizes when parallelizing a funciton
Hey Javier,
this is not really a rasterio problem, mostly a Python peculiarity.
In general, you cannot pass functions defined in a local closure to a different process. Just
Hey Javier,
this is not really a rasterio problem, mostly a Python peculiarity.
In general, you cannot pass functions defined in a local closure to a different process. Just
|
By
Dion Häfner <dion.haefner@...>
·
#241
·
|
|
Re: problem with window sizes when parallelizing a funciton
Hi Dion, Thanks for the tip. I thought that that might be the problem. Any ideas of how to pass a function to executor.map that need several arguments? so far I'm only able to pass functions with a
Hi Dion, Thanks for the tip. I thought that that might be the problem. Any ideas of how to pass a function to executor.map that need several arguments? so far I'm only able to pass functions with a
|
By
javier lopatin
·
#242
·
|
|
Re: problem with window sizes when parallelizing a funciton
As I mentioned, the canonical way to do this is to leverage functools.partial.
For example: from functools import partial from concurrent.futures import ProcessPoolExecutor
As I mentioned, the canonical way to do this is to leverage functools.partial.
For example: from functools import partial from concurrent.futures import ProcessPoolExecutor
|
By
Dion Häfner <dion.haefner@...>
·
#243
·
|