Topics

Python 2/3 compatibility management in rasterio


adrienoyono@...
 

Hello everyone,

First of all, this is my occasion to say thanks for the rasterio project. It has been very helpful simplifying my life last year. I write a sofware that depends on rasterio, should ensure compatibility with Python 2 and Python 3, and should be deployed behind a web server (Apache with mod_wsgi). This last requirement triggers a problem caused by the way rasterio ensures Python 2/3 compatibility as I understand it after reading the "compat" module and the "setup.py".

Below is the stacktrace I have from Apache when trying to load my app:

Traceback (most recent call last):
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/rest/server.wsgi", line 3, in <module>
  from eodag.rest.server import app as application
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/__init__.py", line 18, in <module>
  from .api.core import EODataAccessGateway      # noqa
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/core.py", line 27, in <module>
  from eodag.api.search_result import SearchResult
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/search_result.py", line 26, in <module>
  from eodag.api.product import EOProduct
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/product/__init__.py", line 18, in <module>
  from ._product import EOProduct  # noqa
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/product/_product.py", line 25, in <module>
  import rasterio
File "/opt/eodag/venv/lib/python2.7/site-packages/rasterio/__init__.py", line 22, in <module>
  from rasterio._base import gdal_version
File "rasterio/_base.pyx", line 20, in init rasterio._base
File "/opt/eodag/venv/lib/python2.7/site-packages/rasterio/compat.py", line 26, in <module>
  import mock
ImportError: No module named mock
I guess "mock" is used in rasterio for testing purpose, I do use it for that myself. If that's the case, it is weird to have rasterio brake if we are in Python 2 without the "mock" module being installed, which is clearly the case here. To have mock installed at the same time as rasterio, one must do: "pip install rasterio[dev]". However, we always just do "pip install rasterio" (which fails to install "mock").

I think compatibility for testing purposes should be decoupled from compatibility for the core functionality. Is it possible to have it done in rasterio ? That would help avoid this kind of problem.

Thank you in advance for your time on this issue.


Sean Gillies
 

Hi,

We've always intended to keeping testing dependencies separate and the recent inclusion of mock in runtime dependencies was a mistake. It's been corrected, but it will be a few more days until we have a 1.0.22 release. I'm sorry about the inconvenience!


On Tue, Mar 19, 2019 at 8:21 AM <adrienoyono@...> wrote:
Hello everyone,

First of all, this is my occasion to say thanks for the rasterio project. It has been very helpful simplifying my life last year. I write a sofware that depends on rasterio, should ensure compatibility with Python 2 and Python 3, and should be deployed behind a web server (Apache with mod_wsgi). This last requirement triggers a problem caused by the way rasterio ensures Python 2/3 compatibility as I understand it after reading the "compat" module and the "setup.py".

Below is the stacktrace I have from Apache when trying to load my app:

Traceback (most recent call last):
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/rest/server.wsgi", line 3, in <module>
  from eodag.rest.server import app as application
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/__init__.py", line 18, in <module>
  from .api.core import EODataAccessGateway      # noqa
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/core.py", line 27, in <module>
  from eodag.api.search_result import SearchResult
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/search_result.py", line 26, in <module>
  from eodag.api.product import EOProduct
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/product/__init__.py", line 18, in <module>
  from ._product import EOProduct  # noqa
File "/opt/eodag/venv/lib/python2.7/site-packages/eodag/api/product/_product.py", line 25, in <module>
  import rasterio
File "/opt/eodag/venv/lib/python2.7/site-packages/rasterio/__init__.py", line 22, in <module>
  from rasterio._base import gdal_version
File "rasterio/_base.pyx", line 20, in init rasterio._base
File "/opt/eodag/venv/lib/python2.7/site-packages/rasterio/compat.py", line 26, in <module>
  import mock
ImportError: No module named mock
I guess "mock" is used in rasterio for testing purpose, I do use it for that myself. If that's the case, it is weird to have rasterio brake if we are in Python 2 without the "mock" module being installed, which is clearly the case here. To have mock installed at the same time as rasterio, one must do: "pip install rasterio[dev]". However, we always just do "pip install rasterio" (which fails to install "mock").

I think compatibility for testing purposes should be decoupled from compatibility for the core functionality. Is it possible to have it done in rasterio ? That would help avoid this kind of problem.

Thank you in advance for your time on this issue.



--
Sean Gillies


adrienoyono@...
 

Awesome ! Many thanks !! I will stay tuned to the release of 1.0.22 then