Re: Rewriting uint16 headers with rasterio / applying rio color makes them unreadable by Preview, Photoshop


Sean Gillies
 

Hi Ed,

Can you try the following variation on your first command?

$ rio color -j 1 uint16_image.tif uint16_brightened.tif gamma RGB 1.5 --co photometric=RGB

Note the addition of "--co photometric=RGB". GDAL automatically sets the photometric tag (which other apps need) to RGB when the created image data type is uint8 (see https://www.gdal.org/frmt_gtiff.html) but does not do the same for other data types including uint16.


On Thu, Mar 28, 2019 at 5:48 PM Edward Boyda <boyda@...> wrote:

Hi all, I'm new at this, more of a computer vision person than a developer, so please bear with me....

I see the behavior I'm about to describe running pip-installed rasterio (1.0.22) on my Mac (OSX Mojave; homebrewed python and gdal) and also running rasterio (1.0.22) in a dockerized Ubuntu platform, on images from a variety of sources (DigitalGlobe, Planet, Landsat). 

Example 1: 
$ rio color -j 1 uint16_image.tif uint16_brightened.tif gamma RGB 1.5

When I try to open the output file, uint16_brightened.tif, in Preview or Photoshop, I get a message like "Could not complete your request because of a problem parsing the TIFF file." (That's from Photoshop; Preview is equivalent.) 

Example 2:  
$ rio color -j -1 uint16_image.tif uint16_brightened_v2.tif gamma RGB 1.5

The number of cores has changed from the first example. Now the output, uint16_brightened_v2.tif, is readable by Photoshop but has had its color interpretation changed to (reading from rasterio):

(<ColorInterp.gray: 1>, <ColorInterp.undefined: 0>, <ColorInterp.undefined: 0>)

When I open the file with Preview or view the thumbnail with Mac Finder, there are dark vertical lines interspersed with the actual pixels, and about a third of the original pixels have been pushed out of the frame. See screenshot attached.

Example 3:
I take a file that has a (gray, undefined, undefined) color interpretation and try to change that to RGB, now in the interpreter:
>>> with rasterio.open('uint16_noCI.tif', 'r+') as f:
    f.colorinterp = (ColorInterp.red, ColorInterp.green, ColorInterp.blue)

Again the edited file is unreadable by Preview and Photoshop. 

A couple of caveats:

1) I can read the data from files output from any of the above examples with rasterio or skimage, and the resulting numpy array is uncorrputed. I can resave it with skimage, show it with matplolib, etc., and the image looks as expected.
2) With any of the above outputs, I can read and then rewrite a new_image.tif, using rasterio, and the resulting files open as expected with Photoshop and Preview. This is my current (obviously inefficient) workaround:

with rasterio.open('uint16_brightened.tif') as f:
    prof = f.profile
    img = f.read()

with rasterio.open('new_image.tif', 'w', photometric='rgb', **prof) as f:
    f.write(img)

As far as I know these failures happen only with uint16 images (at least not with uint8), and it would seem to have to do with the way color interpretation is written into the headers via the different write mechanisms.  Has anyone come across similar behavior? I've been reproducing and beating my head against this for months and would really appreciate a sanity check.

Since the Docker container is likely cleaner than what I've installed on my Mac, I've run the tests for the attached output there. Here is the Dockerfile and some possibly relevant release details: 

FROM ubuntu:latest

RUN apt-get update && apt-get install -y software-properties-common

RUN apt-get install -y python3-pip python3-dev build-essential

RUN pip3 install --upgrade pip

RUN apt-get install -y gdal-bin libgdal-dev python3-gdal

RUN apt-get install -y libssl-dev libffi-dev libcurl4-openssl-dev

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get install -y python3-tk


ADD ./requirements.txt /tmp/requirements.txt

RUN pip install -r /tmp/requirements.txt


---

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=18.04

python 3.6.6

pip 18.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)


affine==2.2.2

gbdx-auth==0.4.0

gbdxtools==0.16.0

GDAL==2.2.2

rasterio==1.0.22

rio-color==1.0.0

rio-mucho==1.0.0

Shapely==1.6.4

tifffile==2018.11.28



Thanks everyone!


Ed



--
Sean Gillies

Join main@rasterio.groups.io to automatically receive all group messages.