'OSError: [Errno 9] Bad file descriptor' raised with Pillow image framework

[jvasi]

Python's Pillow image framework doesn't work properly with pyfakefs. When trying to save an image on the fake filesystem, the following exception is raised:

Traceback (most recent call last):
  File "/home/krasi/dev/code/tests/test.py", line 160, in test_pyfakefs_and_PIL
    img.save('/some_target_folder/target_image.jpg')
  File "/home/krasi/programs/anaconda3/envs/tensorflow3/lib/python3.6/site-packages/PIL/Image.py", line 2102, in save
    save_handler(self, fp, filename)
  File "/home/krasi/programs/anaconda3/envs/tensorflow3/lib/python3.6/site-packages/PIL/JpegImagePlugin.py", line 766, in _save
    ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
  File "/home/krasi/programs/anaconda3/envs/tensorflow3/lib/python3.6/site-packages/PIL/ImageFile.py", line 528, in _save
    s = e.encode_to_file(fh, bufsize)
OSError: [Errno 9] Bad file descriptor

Here's a sample PyTest test to reproduce:

import os
from PIL import Image

def test_pyfakefs_and_PIL(fs):
    # Create some image
    img = Image.new('RGB', (60, 30), color = 'red')
    
    # Create target folder
    os.makedirs('/some_target_folder', exist_ok=True)
    assert os.path.exists('/some_target_folder')

    # Try to save it
    img.save('/some_target_folder/target_image.jpg')
    assert os.path.exists('/some_target_folder/target_image.jpg')

Note that the issue reproduces only when saving an image as JPEG (i.e. with '.jpg' extension) and doesn't reproduce when saving the image as PNG.

Environment:
Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid (18.04.2 LTS (Bionic Beaver) )
Python 3.6.6 |Anaconda custom (64-bit)| (default, Jun 28 2018, 17:14:51) [GCC 7.2.0]
pyfakefs 4.0.2
Pillow 7.0.0

To install Pillow:

pip install Pillow

Potentially related StackOverflow issues:

• OSError: [Errno 9] Bad file descriptor while saving image with PIL

[mrbean-bremen]

Thanks for the report - looks like another instance where we are out of luck because of C modules accessing the file system.

Pillow uses an ImagingEncoder object (this is a C module, though I haven't found yet where it comes from) to write the JPEG file, and passes it a file descriptor. This file descriptor has been patched by pyfakefs, it points to a fake file and is therefore not valid in the real filesystem. The C module tries to write to this file descriptor in the real fs (C module file system access cannot be patched), and fails.

I'm not very familiar with Pillow, so I'm not sure if there is a possibility to work around this, for example by registering another imaging encoder, or patching the whole encoder (this also depends on the test - if you want to write valid JPG files in the fake fs, this will be more difficult). I may have a closer look into this some time later, but cannot promise anything at the moment.

[mrbean-bremen]

I had another look, and this doesn't look promising. You could in principle register your own JPEG encoder, but that does not help much. If you just register a mock like this:

def test_pyfakefs_and_PIL(fs):
    # create a mock encoder and setup it 
    mock_encoder = Mock()
    mock_encoder.return_value.pushes_fd = False
    mock_encoder.return_value.encode_to_file.return_value = 10

   # register the mock encoder instead of the real one
    Image.register_encoder('jpeg', mock_encoder)

    # Create some image
    img = Image.new('RGB', (60, 30), color='red')

    # Create target folder
    os.makedirs('/some_target_folder', exist_ok=True)
    assert os.path.exists('/some_target_folder')

    # Try to save it
    img.save('/some_target_folder/target_image.jpg')
    assert os.path.exists('/some_target_folder/target_image.jpg')

the test passes, but the written file is (unsurprisingly) empty. I have tried to write a simple fake encoder that just writes the image back, but that fails because the image is passed to the encoder as an opaque C object (PyCapsule), so that is not possible.

So, at least for JPEG files Pillow will not work with pyfakefs, as long as you need real JPEG files written in the fake filesystem.

[jvasi]

I see. Well, I appreciate you looking into this!

[mrbean-bremen]

I'm closing this then - I added a comment in the documentation about pyfakefs not working with Pillow, but can't do anything else here, unfortunately.