Re: Proposal: Allow other cloud object store providers


ashley.sommer@...
 

Thanks for your reply, Sean.

> are you proposing that potentially any cloud platform would be made first class in rasterio, concretely, as in GDAL and as with rasterio/S3 today?

Yes, thats right.

> What would you think if rasterio were to take the opposite approach and require users to write ~5 lines of code themselves to adapt output of, say, keystonev3 and swiftclient to a standard interface in rasterio

Yeah, I actually had the same thought too, but wasn't sure if it would be received well.

That is actually kind-of how you can do it in rasterio already. You can manually create a `keystonev3` or `swiftclient` auth session, and populate it with your credentials. You can then manually create a rasterio `SwiftClient`, and give it that auth session. Then pass the pre-configured `SwiftClient` into `session.Env()` and rasterio will use that as the Cloud Session. Its a bit of boilerplate code, but it works across all of the existing `Session` subclasses for the different cloud platforms already.

Should that be the "standard" way of doing it? Could it be cleaner? Would AWS S3 still be a special case?

My end goal is to be able to use OpenStack Swift ObjectStore as a storage backend for an opendatacube project. Opendatacube only supports AWS S3 for now, because it relies on rasterio's "first-class" interface to S3. I was told, if I want to get other cloud providers working natively in opendatacube, we need them to be fully supported by rasterio first (as you mentioned, they're already fully supported by GDAL).

Ashley Sommer

Join main@rasterio.groups.io to automatically receive all group messages.