Thanks for the reply Luke!
A context manager would work, but it's ergonomically annoying, no? I have a several functions I would like to pipe, each returning a raster. In that case I will have a lot of nested with statements:
with do_something1(ds1) as ds2:
with do_something2(ds2) as ds3:
Currently my solution was to change MemoryFile to NamedTemporaryFile, which solves the memory leak issue at the expense of some speed (but it's not terrible when you have an SSD).