Tracking down a segmentation fault


Angus Dickey
 

We have a Chalice app that uses Rasterio to read COGs from an AWS S3 bucket, do some processing, and deliver the result as XYZ tiles.

Recently we upgraded the rasterio version and started seeing segmentation faults killing the chalice development server. Rasterio 1.2.3 and lower work fine, but 1.2.4 and up segfaults after a small number of requests.

The app uses a python 3.6 virtual environment and the rasterio manylinux wheels (which are awesome BTW). The backtrace (from gdb) seems to show the problem is in libgdal but doesn't provide much to go on:

Thread 13 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff9e5d8700 (LWP 3887)]
0x00007fffad92e0b3 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
(gdb) bt
#0  0x00007fffad92e0b3 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#1  0x00007fffad92e552 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#2  0x00007fffad92e5b1 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#3  0x00007fffad92e679 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#4  0x00007fffad8d0fcf in GDALDataset::ReleaseRef() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#5  0x00007fffad86448c in VRTSimpleSource::~VRTSimpleSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#6  0x00007fffad864569 in VRTComplexSource::~VRTComplexSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#7  0x00007fffad863e8a in VRTSourcedRasterBand::CloseDependentDatasets() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#8  0x00007fffad863eeb in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#9  0x00007fffad863f59 in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#10 0x00007fffad8d67ef in GDALDataset::~GDALDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#11 0x00007fffad8409f2 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#12 0x00007fffad840a99 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#13 0x00007fffad92dd11 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#14 0x00007fffad92e311 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#15 0x00007fffad92e62c in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#16 0x00007fffad92e679 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#17 0x00007fffad8d0fcf in GDALDataset::ReleaseRef() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#18 0x00007fffad86448c in VRTSimpleSource::~VRTSimpleSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#19 0x00007fffad864569 in VRTComplexSource::~VRTComplexSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#20 0x00007fffad863e8a in VRTSourcedRasterBand::CloseDependentDatasets() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#21 0x00007fffad863eeb in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#22 0x00007fffad863f59 in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#23 0x00007fffad8d67ef in GDALDataset::~GDALDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#24 0x00007fffad8409f2 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#25 0x00007fffad840a99 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#26 0x00007fffaec0ac99 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#27 0x00007fffaec21216 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#28 0x00007fffaec2d5d0 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#29 0x0000000000566bbc in _PyCFunction_FastCallDict ()
#30 0x00000000005a4cd1 in _PyObject_FastCallDict ()
#31 0x00000000005a4fb8 in PyObject_CallFunctionObjArgs ()
#32 0x000000000050d698 in _PyEval_EvalFrameDefault ()
#33 0x00000000005095c8 in ?? ()
#34 0x000000000050a2fd in ?? ()
#35 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#36 0x00000000005095c8 in ?? ()
#37 0x000000000050a2fd in ?? ()
#38 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#39 0x0000000000507be4 in ?? ()
#40 0x0000000000588e5c in ?? ()
#41 0x000000000059fd0e in PyObject_Call ()
#42 0x000000000050d256 in _PyEval_EvalFrameDefault ()
#43 0x00000000005095c8 in ?? ()
#44 0x000000000050a2fd in ?? ()
#45 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#46 0x0000000000507be4 in ?? ()
#47 0x0000000000509900 in ?? ()
#48 0x000000000050a2fd in ?? ()
#49 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#50 0x0000000000507be4 in ?? ()
#51 0x0000000000509900 in ?? ()
#52 0x000000000050a2fd in ?? ()
#53 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#54 0x00000000005095c8 in ?? ()
#55 0x000000000050a2fd in ?? ()
#56 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#57 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#58 0x0000000000594a01 in ?? ()
#59 0x000000000054a971 in ?? ()
#60 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#61 0x000000000050a433 in ?? ()
#62 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#63 0x0000000000507be4 in ?? ()
#64 0x0000000000508ec2 in _PyFunction_FastCallDict ()
#65 0x0000000000594a01 in ?? ()
#66 0x000000000054a971 in ?? ()
#67 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#68 0x000000000050a433 in ?? ()
#69 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#70 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#71 0x0000000000594a01 in ?? ()
#72 0x000000000054a971 in ?? ()
#73 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#74 0x000000000050a433 in ?? ()
#75 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#76 0x0000000000507be4 in ?? ()
#77 0x0000000000509900 in ?? ()
#78 0x000000000050a2fd in ?? ()
#79 0x000000000050cc96 in _PyEval_EvalFrameDefault ()
#80 0x00000000005095c8 in ?? ()
#81 0x000000000050a2fd in ?? ()
#82 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#83 0x00000000005095c8 in ?? ()
#84 0x000000000050a2fd in ?? ()
#85 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#86 0x00000000005095c8 in ?? ()
#87 0x000000000050a2fd in ?? ()
#88 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#89 0x00000000005095c8 in ?? ()
#90 0x000000000050a2fd in ?? ()
#91 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#92 0x0000000000507be4 in ?? ()
#93 0x0000000000508ec2 in _PyFunction_FastCallDict ()
#94 0x0000000000594a01 in ?? ()
#95 0x0000000000549e8f in ?? ()
#96 0x00000000005515c1 in ?? ()
#97 0x00000000005a48ec in _PyObject_FastCallDict ()
#98 0x00000000005f03bc in ?? ()
#99 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#100 0x000000000050a433 in ?? ()
#101 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#102 0x00000000005095c8 in ?? ()
#103 0x000000000050a2fd in ?? ()
#104 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#105 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#106 0x0000000000594a01 in ?? ()
#107 0x000000000059fd0e in PyObject_Call ()
#108 0x000000000050d256 in _PyEval_EvalFrameDefault ()
#109 0x00000000005095c8 in ?? ()
#110 0x000000000050a2fd in ?? ()
#111 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#112 0x00000000005095c8 in ?? ()
#113 0x000000000050a2fd in ?? ()
#114 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#115 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#116 0x0000000000594a01 in ?? ()
#117 0x000000000059fd0e in PyObject_Call ()
#118 0x00000000005e12f2 in ?? ()
#119 0x0000000000631b94 in ?? ()
#120 0x00007ffff77ca6db in start_thread (arg=0x7fff9e5d8700) at pthread_create.c:463
#121 0x00007ffff7b0371f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95


Is there a location where the libgdal version for each resterio release is documented? I presume 1.2.4 uses a newer GDAl version.

Any tips for tracking down what the problem might be in rasterio/GDAL?

Thanks for any help,

Angus


Sean Gillies
 

Hi Angus,

On Wed, Aug 11, 2021 at 9:41 PM Angus Dickey <angus@...> wrote:
We have a Chalice app that uses Rasterio to read COGs from an AWS S3 bucket, do some processing, and deliver the result as XYZ tiles.

Recently we upgraded the rasterio version and started seeing segmentation faults killing the chalice development server. Rasterio 1.2.3 and lower work fine, but 1.2.4 and up segfaults after a small number of requests.

The app uses a python 3.6 virtual environment and the rasterio manylinux wheels (which are awesome BTW). The backtrace (from gdb) seems to show the problem is in libgdal but doesn't provide much to go on:

Thread 13 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff9e5d8700 (LWP 3887)]
0x00007fffad92e0b3 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
(gdb) bt
#0  0x00007fffad92e0b3 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#1  0x00007fffad92e552 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#2  0x00007fffad92e5b1 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#3  0x00007fffad92e679 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#4  0x00007fffad8d0fcf in GDALDataset::ReleaseRef() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#5  0x00007fffad86448c in VRTSimpleSource::~VRTSimpleSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#6  0x00007fffad864569 in VRTComplexSource::~VRTComplexSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#7  0x00007fffad863e8a in VRTSourcedRasterBand::CloseDependentDatasets() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#8  0x00007fffad863eeb in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#9  0x00007fffad863f59 in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#10 0x00007fffad8d67ef in GDALDataset::~GDALDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#11 0x00007fffad8409f2 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#12 0x00007fffad840a99 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#13 0x00007fffad92dd11 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#14 0x00007fffad92e311 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#15 0x00007fffad92e62c in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#16 0x00007fffad92e679 in GDALProxyPoolDataset::~GDALProxyPoolDataset() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#17 0x00007fffad8d0fcf in GDALDataset::ReleaseRef() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#18 0x00007fffad86448c in VRTSimpleSource::~VRTSimpleSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#19 0x00007fffad864569 in VRTComplexSource::~VRTComplexSource() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#20 0x00007fffad863e8a in VRTSourcedRasterBand::CloseDependentDatasets() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#21 0x00007fffad863eeb in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#22 0x00007fffad863f59 in VRTSourcedRasterBand::~VRTSourcedRasterBand() ()
   from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#23 0x00007fffad8d67ef in GDALDataset::~GDALDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#24 0x00007fffad8409f2 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#25 0x00007fffad840a99 in VRTDataset::~VRTDataset() () from /home/user/project/env/lib/python3.6/site-packages/rasterio/../rasterio.libs/libgdal-41ff680c.so.29.0.0
#26 0x00007fffaec0ac99 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#27 0x00007fffaec21216 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#28 0x00007fffaec2d5d0 in ?? () from /home/user/project/env/lib/python3.6/site-packages/rasterio/_base.cpython-36m-x86_64-linux-gnu.so
#29 0x0000000000566bbc in _PyCFunction_FastCallDict ()
#30 0x00000000005a4cd1 in _PyObject_FastCallDict ()
#31 0x00000000005a4fb8 in PyObject_CallFunctionObjArgs ()
#32 0x000000000050d698 in _PyEval_EvalFrameDefault ()
#33 0x00000000005095c8 in ?? ()
#34 0x000000000050a2fd in ?? ()
#35 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#36 0x00000000005095c8 in ?? ()
#37 0x000000000050a2fd in ?? ()
#38 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#39 0x0000000000507be4 in ?? ()
#40 0x0000000000588e5c in ?? ()
#41 0x000000000059fd0e in PyObject_Call ()
#42 0x000000000050d256 in _PyEval_EvalFrameDefault ()
#43 0x00000000005095c8 in ?? ()
#44 0x000000000050a2fd in ?? ()
#45 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#46 0x0000000000507be4 in ?? ()
#47 0x0000000000509900 in ?? ()
#48 0x000000000050a2fd in ?? ()
#49 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#50 0x0000000000507be4 in ?? ()
#51 0x0000000000509900 in ?? ()
#52 0x000000000050a2fd in ?? ()
#53 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#54 0x00000000005095c8 in ?? ()
#55 0x000000000050a2fd in ?? ()
#56 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#57 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#58 0x0000000000594a01 in ?? ()
#59 0x000000000054a971 in ?? ()
#60 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#61 0x000000000050a433 in ?? ()
#62 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#63 0x0000000000507be4 in ?? ()
#64 0x0000000000508ec2 in _PyFunction_FastCallDict ()
#65 0x0000000000594a01 in ?? ()
#66 0x000000000054a971 in ?? ()
#67 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#68 0x000000000050a433 in ?? ()
#69 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#70 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#71 0x0000000000594a01 in ?? ()
#72 0x000000000054a971 in ?? ()
#73 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#74 0x000000000050a433 in ?? ()
#75 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#76 0x0000000000507be4 in ?? ()
#77 0x0000000000509900 in ?? ()
#78 0x000000000050a2fd in ?? ()
#79 0x000000000050cc96 in _PyEval_EvalFrameDefault ()
#80 0x00000000005095c8 in ?? ()
#81 0x000000000050a2fd in ?? ()
#82 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#83 0x00000000005095c8 in ?? ()
#84 0x000000000050a2fd in ?? ()
#85 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#86 0x00000000005095c8 in ?? ()
#87 0x000000000050a2fd in ?? ()
#88 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#89 0x00000000005095c8 in ?? ()
#90 0x000000000050a2fd in ?? ()
#91 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#92 0x0000000000507be4 in ?? ()
#93 0x0000000000508ec2 in _PyFunction_FastCallDict ()
#94 0x0000000000594a01 in ?? ()
#95 0x0000000000549e8f in ?? ()
#96 0x00000000005515c1 in ?? ()
#97 0x00000000005a48ec in _PyObject_FastCallDict ()
#98 0x00000000005f03bc in ?? ()
#99 0x00000000005a9dac in _PyObject_FastCallKeywords ()
#100 0x000000000050a433 in ?? ()
#101 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#102 0x00000000005095c8 in ?? ()
#103 0x000000000050a2fd in ?? ()
#104 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#105 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#106 0x0000000000594a01 in ?? ()
#107 0x000000000059fd0e in PyObject_Call ()
#108 0x000000000050d256 in _PyEval_EvalFrameDefault ()
#109 0x00000000005095c8 in ?? ()
#110 0x000000000050a2fd in ?? ()
#111 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#112 0x00000000005095c8 in ?? ()
#113 0x000000000050a2fd in ?? ()
#114 0x000000000050beb4 in _PyEval_EvalFrameDefault ()
#115 0x0000000000508cd5 in _PyFunction_FastCallDict ()
#116 0x0000000000594a01 in ?? ()
#117 0x000000000059fd0e in PyObject_Call ()
#118 0x00000000005e12f2 in ?? ()
#119 0x0000000000631b94 in ?? ()
#120 0x00007ffff77ca6db in start_thread (arg=0x7fff9e5d8700) at pthread_create.c:463
#121 0x00007ffff7b0371f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95


Is there a location where the libgdal version for each resterio release is documented? I presume 1.2.4 uses a newer GDAl version.

Any tips for tracking down what the problem might be in rasterio/GDAL?

Thanks for any help,

Angus


Rasterio's 1.2.3 wheels include GDAL 3.2.2. The 1.2.4 wheels include GDAL 3.3.0. We really should write these down in the readme.

I did a shallow search of the GDAL issue tracker and found https://github.com/OSGeo/gdal/issues/3189. There may be a lead in there or in another issue I didn't find.

--
Sean Gillies


Even Rouault
 



I did a shallow search of the GDAL issue tracker and found https://github.com/OSGeo/gdal/issues/3189.

Likely unrelated: the above issue was specific to VRT pansharpening.

-- 
http://www.spatialys.com
My software is free, but my time generally not.


Angus Dickey
 

Thanks to both Sean and Even for their responses.

I ended up compiling GDAL 3.3.0 (to include the debug symbols), then building rasterio on top of that. The backtrace (full bt is attached) now provides some more information:

#0  GDALDatasetPool::_CloseDatasetIfZeroRefCount (this=0x7fffb03c4940, pszFileName=pszFileName@entry=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0") at gdalproxypool.cpp:378
#1  0x00007fffcb5034a2 in GDALDatasetPool::CloseDatasetIfZeroRefCount (pszFileName=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", eAccess=eAccess@entry=GA_ReadOnly, pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0")at gdalproxypool.cpp:520

Seems the issue is here in GDAL. I am not sure if this is a bug or am am doing something "off label". For context I am reading a VRT stored in S3 that is made up of a series of COGS.

Should I be moving this out of the rasterio list to GDAL? It might not be easy to replicate with GDAL alone though.

Thanks again for any thoughts,

Angus

 


Sean Gillies
 

Hi Angus,

Would you be willing to make an issue at https://github.com/OSGeo/gdal/issues. It's possible that we're using GDAL incorrectly, but it should guard against a crash like this.

On Wed, Aug 18, 2021 at 12:12 PM Angus Dickey <angus@...> wrote:

Thanks to both Sean and Even for their responses.

I ended up compiling GDAL 3.3.0 (to include the debug symbols), then building rasterio on top of that. The backtrace (full bt is attached) now provides some more information:

#0  GDALDatasetPool::_CloseDatasetIfZeroRefCount (this=0x7fffb03c4940, pszFileName=pszFileName@entry=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0") at gdalproxypool.cpp:378
#1  0x00007fffcb5034a2 in GDALDatasetPool::CloseDatasetIfZeroRefCount (pszFileName=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", eAccess=eAccess@entry=GA_ReadOnly, pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0")at gdalproxypool.cpp:520

Seems the issue is here in GDAL. I am not sure if this is a bug or am am doing something "off label". For context I am reading a VRT stored in S3 that is made up of a series of COGS.

Should I be moving this out of the rasterio list to GDAL? It might not be easy to replicate with GDAL alone though.

Thanks again for any thoughts,

Angus


--
Sean Gillies


Angus Dickey
 

Sean,

I created an issue as suggested.

https://github.com/OSGeo/gdal/issues/4318

Thanks,

Angus

Angus Dickey

Technical Lead

angus@...  |  LinkedIn


MyHEAT.ca  HEAT Maps  |  SOLAR Maps



On Fri, Aug 20, 2021 at 8:05 AM Sean Gillies via groups.io <sean=mapbox.com@groups.io> wrote:
Hi Angus,

Would you be willing to make an issue at https://github.com/OSGeo/gdal/issues. It's possible that we're using GDAL incorrectly, but it should guard against a crash like this.

On Wed, Aug 18, 2021 at 12:12 PM Angus Dickey <angus@...> wrote:

Thanks to both Sean and Even for their responses.

I ended up compiling GDAL 3.3.0 (to include the debug symbols), then building rasterio on top of that. The backtrace (full bt is attached) now provides some more information:

#0  GDALDatasetPool::_CloseDatasetIfZeroRefCount (this=0x7fffb03c4940, pszFileName=pszFileName@entry=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0") at gdalproxypool.cpp:378
#1  0x00007fffcb5034a2 in GDALDatasetPool::CloseDatasetIfZeroRefCount (pszFileName=0x7fffa85946f0 "/vsis3/bucket/path/01.cog.tiff", eAccess=eAccess@entry=GA_ReadOnly, pszOwner=pszOwner@entry=0x7fffa83c6eb0 "0x7fffa84f38c0")at gdalproxypool.cpp:520

Seems the issue is here in GDAL. I am not sure if this is a bug or am am doing something "off label". For context I am reading a VRT stored in S3 that is made up of a series of COGS.

Should I be moving this out of the rasterio list to GDAL? It might not be easy to replicate with GDAL alone though.

Thanks again for any thoughts,

Angus


--
Sean Gillies