Uploading images to a dynamic path with Django

31 July 2007
Update: There’s a new method you should try first. See: Dynamic upload paths in Django

Django makes it easy to upload images by adding an ImageField to your model. The images are uploaded to your media path in a subdirectory specified with the upload_to parameter which can contain a date/time pattern like %Y/%m/%d.

class Photo(models.Model):
    caption = models.CharField(blank=True, maxlength=100)
    image = models.ImageField(upload_to='photos/%Y/%m/%d')

In this example, images will be uploaded to a path like:

/path/to/media/photos/2007/07/31/flowers.jpg.

Sometimes you want to keep related images together, rather than spreading them over multiple date directories. But if you have a lot of images, you won’t want them all stored in a single directory.

It would be nice if there was a way to upload to a directory specific to the model, perhaps a path incorporating the model object’s id or a unique slug. Something like:

/path/to/media/photos/1234/flowers.jpg
or
/path/to/media/photos/scotland-trip/castle.jpg
/path/to/media/photos/scotland-trip/bonnie-purple-heather.jpg

Django doesn’t have a standard way to do this at the moment (it’s pending a design decision according to ticket #4113).

I needed to do this in a project recently and tried various different approaches. Here’s what I tried – skip to the working solution if you’re not interested in the failed attempts.

Attempt 1: Specify upload_to dynamically

Why not make upload_to include the id of the model?

class Photo(models.Model):
    caption = models.CharField(blank=True, maxlength=100)
    image = models.ImageField(upload_to='photos/%d' % self.id)

Because there is no self; that’s why. Django builds the model with the fields we specify, but at that point, there is no instance of the model, so self is meaningless. Whatever we put in upload_to here will apply to all instances of the model.

Attempt 2: Set upload_to on save

How about overriding the save method of the model and setting upload_to on the image field at that point. Something like:

class Photo(models.Model):
    caption = models.CharField(blank=True, maxlength=100)
    image = models.ImageField(upload_to='photos')

    def save(self):
        for field in self._meta.fields:
            if field.name == 'image':
                field.upload_to = 'photos/%d' % self.id
        super(Photo, self).save()

It’s a bit icky having to iterate through self._meta.fields to find the right one. But a bigger problem is the image file may well be written to the path before save is called.

Usually you would call photo.save_image_file(filename, content) to save the file content then photo.save() to save the model’s fields, including the path in the image field. By the time we set upload_to, it’s too late.

Attempt 3: Override ImageField and pass a callable for upload_to

Taking a different approach, how about making a new class that derives from ImageField and takes either a new parameter or a callable for the upload_to parameter. Something like:

class SpecialImageField(ImageField):

    def get_directory_name(self):
        if callable(self.upload_to):
            return self.upload_to()
        else:
            return super(SpecialImageField, self).get_directory_name()

We override the get_directory_name method which is actually defined in FileField (from which ImageField inherits).

The problem is, we’re still having to pass something (a callable) for the upload_to parameter. Again, at the time the ImageField is created, we are not in an instance of the model, so we can’t pass a it a model method. We could pass a module-level function, but that’s not enough information; we want to set the path using something in the model instance.

Attempt 4 – the one that worked: Override ImageField get model instance and ask it

After going round in circles and learning a few things on the way, I came across this page in the Django wiki (mental note: check wiki first in future).

It shows how a custom field can get hold of the model instance using dispatcher. I wrote CustomImageField to get the model instance when the model instance is initialised and ask it to supply a new upload_to path.

The field looks like this:

from django.db.models import ImageField, signals
from django.dispatch import dispatcher

class CustomImageField(ImageField):
    """Allows model instance to specify upload_to dynamically.
    
    Model class should have a method like:

        def get_upload_to(self, attname):
            return 'path/to/%d' % self.id
    
    Based on: http://code.djangoproject.com/wiki/CustomUploadAndFilters    
    """
    def contribute_to_class(self, cls, name):
        """Hook up events so we can access the instance."""
        super(CustomImageField, self).contribute_to_class(cls, name)
        dispatcher.connect(self._post_init, signals.post_init, sender=cls)
    
    def _post_init(self, instance=None):
        """Get dynamic upload_to value from the model instance."""
        if hasattr(instance, 'get_upload_to'):
            self.upload_to = instance.get_upload_to(self.attname)

    def db_type(self):
        """Required by Django for ORM."""
        return 'varchar(100)'

Note the db_type method which replaces the get_internal_type method in Django trunk. It is used when you run manage.py syncdb to know what field type to create in the database.

The field is used in a model like this:

class Photo(models.Model):
    caption = models.CharField(blank=True, maxlength=100)
    image = CustomImageField(upload_to='photos')

    def get_upload_to(self, field_attname):
        """Get upload_to path specific to this photo."""
        return 'photos/%d' % self.id

get_upload_to is passed the attname of the field in that model (in this case “image”). This is so the model can distinguish between multiple custom image fields.

Ok, so the bit I’ve glossed over is that the model may not have an id at the time get_upload_to is called. If the model is new and hasn’t been saved you’ll need to save it or work something else out before you can return the dynamic upload_to path. But that was always the case, so I’m not taking the blame.

In my case, the Photo model was related to some other model (e.g. Room) which I called to get the path. It didn’t matter that Photo didn’t have an id because it was related to something that did.

So now I get to save images to paths like:

/path/to/media/photos/12345/front.jpg
/path/to/media/photos/12345/rooms/kitchen.jpg
etc

Update – 20 May 2008:

Here’s a small update to the CustomImageField class. The version above listens for the post_init signal and use it to get the dynamic upload path. This works fine when you use it like:

photo = Photo.objects.create(...)

Calling create saves the object and loads it so that post_init gets called. However, if you create the model object and upload a file before saving, it will not know about the dynamic upload path.

The version below listens for the pre_save signal instead and gets the dynamic upload path at that point. You can use it like:

photo = Photo()
photo.save_image_file(filename, content)

Note that you may still need to save the model before uploading if your dynamic path includes the model id (which is not set until it is saved).

Here is the new version of the field:

class CustomImageField(ImageField):
    """Allows model instance to specify upload_to dynamically.
    
    Model class should have a method like:

        def get_upload_to(self, attname):
            return 'path/to/%d' % self.id
    
    Based on: http://code.djangoproject.com/wiki/CustomUploadAndFilters    
    """
    def __init__(self, *args, **kwargs):
        if not 'upload_to' in kwargs:
            kwargs['upload_to'] = 'dummy'
        self.prime_upload = kwargs.get('prime_upload', False)
        if 'prime_upload' in kwargs:
            del(kwargs['prime_upload'])
        super(CustomImageField, self).__init__(*args, **kwargs)
    
    def contribute_to_class(self, cls, name):
        """Hook up events so we can access the instance."""
        super(CustomImageField, self).contribute_to_class(cls, name)
        if self.prime_upload:
            dispatcher.connect(self._get_upload_to, signals.post_init, sender=cls)
        dispatcher.connect(self._get_upload_to, signals.pre_save, sender=cls)
        
    def _get_upload_to(self, instance=None):
        """Get dynamic upload_to value from the model instance."""
        if hasattr(instance, 'get_upload_to'):
            self.upload_to = instance.get_upload_to(self.attname)

    def db_type(self):
        """Required by Django for ORM."""
        return 'varchar(100)'

In some cases you will still want the path to be specified when the model is initialised rather than saved. If you are editing a model and want to be able to save a new image without saving the model first, it needs to get the dynamic upload path when the post_init signal is raised.

This new CustomImageField takes an optional prime_upload argument. If true, it will also listen for the post_init event and get the dynamic upload path. You can use it like:

class Photo(models.Model):
    image = CustomImageField(prime_upload=True)
photo = Photo.objects.get(pk=photo_id)
photo.save_image_file(filename, content)

This is all a bit fiddly still, but it does the job until Django has a native way to specify the upload path per-instance.

Filed under: Django — Scott @ 9:25 pm

21 Comments »

  1. That sure was helpful. Thankyou.

    Comment by rtelep — 5 October 2007 @ 4:37 pm

  2. Thank you! Nice article.
    How do you think, what’s the beter way to store images in gallery?
    path/to/image_id.jpg or
    path/to/image_id/image_name.jpg

    Comment by krylatij — 6 November 2007 @ 5:56 pm

  3. Hi krylatij.

    Thanks for your comment.

    If you expect the image name to be something useful, I suppose it gives slightly more info to have a path like:

    path/to/16/edinburgh_castle.jpg

    Google image search could make use of it, for example. It’s also more useful to someone who saves a copy of the image. Django automatically sanitises the image file name, removing spaces and undesirably characters (apostrophe, angle brackets, etc), so there’s no worries there.

    Django deals with filename clashes by appending underscores to the filename, but given that you can separate photos in to directories, clashes are less likely.

    Comment by Scott — 6 November 2007 @ 6:42 pm

  4. Thank you, this article saved a lot of time

    Comment by Satheesh Chandrasekhar — 9 November 2007 @ 5:28 am

  5. Thank you for reply.

    Comment by krylatij — 9 November 2007 @ 9:50 am

  6. combine this with this and we have a winner… you guys should chat!!

    http://scompt.com/archives/2007/11/03/multiple-file-uploads-in-django

    Comment by Mogga — 30 November 2007 @ 5:34 pm

  7. [...] часть ТЗ и неработающее в современной Джанге. :) Второе – умеет заливать файлики по динамически изменяющемуся [...]

    Pingback by Amazon byteflow: ImprovedImageField — 2 March 2008 @ 11:40 pm

  8. Hi, it’s very clear and useful what you have published ! thanks!
    But …i’m having a little trouble, when i try to use CustomImageField as a field on a class and i try to run “manage.py syncdb” i get this message on console:

    …>python manage.py syncdb
    Traceback (most recent call last):
    File “manage.py”, line 11, in
    execute_manager(settings)
    File “C:\python25\Lib\site-packages\django\core\management.py”, line 1672, in
    execute_manager
    execute_from_command_line(action_mapping, argv)
    File “C:\python25\Lib\site-packages\django\core\management.py”, line 1571, in
    execute_from_command_line
    action_mapping[action](int(options.verbosity), options.interactive)
    File “C:\python25\Lib\site-packages\django\core\management.py”, line 525, in s
    yncdb
    sql, references = _get_sql_model_create(model, seen_models)
    File “C:\python25\Lib\site-packages\django\core\management.py”, line 175, in _
    get_sql_model_create
    col_type = data_types[data_type]
    KeyError: ‘CustomImageField’

    Can you please help me ?? What i’m doing wrong (i have another class that uses this ImageField extention and when i comment that field = CustomImageField declaration, syncdb runs fine).

    Class that uses CustomImageField (both classes were defined on models.py)

    ….
    class Commentarist(models.Model):
    name = models.CharField(maxlength=50, core=True)
    email = models.EmailField(maxlength=50, core=True)
    subscribed = models.BooleanField(default=False)
    avatar = CustomImageField(upload_to=’photos’)
    created_date = models.DateTimeField(default=datetime.now(), core=True)
    modified_date = models.DateTimeField(default=datetime.now())
    ….

    Comment by Jose — 31 March 2008 @ 11:00 pm

  9. Hi José.

    It sounds like you have an older version of Django than the one I used. Since r5725, the ORM looks for a db_type() method.

    Before that, it used a get_internal_type() method.

    I haven’t tested this, but try adding a get_internal_type() method to the CustomImageField class:

    class CustomImageField(ImageField):
        ...
        def get_internal_type(self):
            return 'ImageField'
    

    This method tells the ORM to treat a CustomImageField the same as a normal ImageField in the database.

    Comment by Scott — 1 April 2008 @ 9:24 am

  10. Scott !…you are the man ;-). Thank you very much !
    This is my first experience with Python and i’m really enjoying this stuff (it’s really interesting, i’m java/php developer, and sometimes i do some other stuff, but never with Python).

    To find people that helps – like you – it’s really a good sign when i have to decide to continue with a project on python or not. :-).

    Thanks again

    Comment by Jose — 1 April 2008 @ 9:05 pm

  11. This is great.. but I have a question. I want to save all uploads for a user in a folder just for them: Example – /path/to/media/Ted/head.jpg or with user id – /path/to/media/1/head.jpg

    Is there anyway to do this?

    Comment by chris rockwell — 1 May 2008 @ 11:16 pm

  12. Hi Chris.

    Yes, you can. The model that contains the CustomImageField gets to build up the path. Assuming you have a Photo model which has a CustomImageField and is related to a user, you could do something like:

    class Photo(models.Model):
        user = models.ForeignKey(User, related_name='photos')
        image = CustomImageField(upload_to='photos')
        creation_date = models.DateTimeField()
        ...
        
        def get_upload_to(self, field_attname):
            return self.user.username
    

    The upload path will be made up of the media path, username and filename. Assuming the username is “ted”, you will get something like: /path/to/media/ted/head.jpg

    Comment by Scott — 2 May 2008 @ 8:40 am

  13. The CustomImageField class doesn’t properly save the image to the specified dynamic path via the following code:

    if form.is_valid():
    l = ListingImage()

    uploadedImage = form.cleaned_data['image'] # an UploadedFile object
    l.save_image_file(uploadedImage.filename, uploadedImage.content)

    l.caption = form.cleaned_data['caption']
    l.sort = form.cleaned_data['sort']
    l.save()

    ListingImage.image is a CustomImageField()

    Anyone know why calling save_image_file() doesn’t invoke the get_upload_to() to save the image to the dynamic path?

    Comment by Michael — 20 May 2008 @ 5:55 am

  14. @Michael

    I’ve just posted an updated version (see above) which should work in this situation.

    Comment by Scott — 20 May 2008 @ 1:02 pm

  15. Great Post! I’ve used this for a while now.
    However, this seems to have broken with the inclusion of the Newforms Admin now. I’m getting a DoesNotExist error on the instance. Have you tried the newest revision?

    Comment by Ben — 5 August 2008 @ 3:38 am

  16. In an effort to get this going with newforms-admin I tried working with the post_save signal. I’ve posted it on my blog:
    http://pandemoniumillusion.wordpress.com/2008/08/06/django-imagefield-and-filefield-dynamic-upload-path-in-newforms-admin/

    Comment by Ben — 6 August 2008 @ 5:02 am

  17. Because of http://code.djangoproject.com/wiki/BackwardsIncompatibleChanges#Signalrefactoring I guess this is gonna look like this:

    def contribute_to_class(self, cls, name):
    “”"
    Hook up events so we can access the instance.
    “”"
    super(CustomImageField, self).contribute_to_class(cls, name)
    if self.prime_upload:
    signals.post_init.connect(self._get_upload_to, sender=cls)
    signals.pre_save.connect(self._get_upload_to, sender=cls)

    def _get_upload_to(self, **kwargs):
    “”"
    Get dynamic upload_to value from the model instance.
    “”"
    instance = kwargs['instance']
    if hasattr(instance, ‘get_upload_to’):
    self.upload_to = instance.get_upload_to(self.attname)

    Comment by mat — 8 August 2008 @ 11:04 am

  18. This solution was working fine until the File Storage refactoring (http://code.djangoproject.com/wiki/FileStorageRefactor), and I’m afraid the changes to make it work again may be extensive. Anyone knows a File Storage-compatible substitute?

    Comment by Fidel Ramos — 21 August 2008 @ 12:23 pm

  19. @Fidel Ramos

    I have a new short post with an example of how to get dynamic upload paths since the FileStorageRefactor merge. It uses a callable in the upload_to parameter and is much simpler than the method in this post.

    See:
    Dynamic upload paths in Django

    Comment by Scott — 25 August 2008 @ 10:48 pm

  20. Here is the updated version that works with latest Django:

    from django.db.models import ImageField, signals
    
    
    class CustomImageField(ImageField):
        """Allows model instance to specify upload_to dynamically.
    
        Model class should have a method like:
    
            def get_upload_to(self, attname):
                return 'path/to/%d' % self.id
    
        Based on: http://code.djangoproject.com/wiki/CustomUploadAndFilters
        """
        def __init__(self, *args, **kwargs):
            if not 'upload_to' in kwargs:
                kwargs['upload_to'] = 'dummy'
            self.prime_upload = kwargs.get('prime_upload', False)
            if 'prime_upload' in kwargs:
                del(kwargs['prime_upload'])
            super(CustomImageField, self).__init__(*args, **kwargs)
    
        def contribute_to_class(self, cls, name, **kwargs): 
            """Hook up events so we can access the instance."""
            super(CustomImageField, self).contribute_to_class(cls, name)
            if self.prime_upload:
                signals.post_init.connect(self._get_upload_to, sender=cls)
            signals.pre_save.connect(self._get_upload_to, sender=cls)
    
        def _get_upload_to(self, instance=None, **kwargs):
            """Get dynamic upload_to value from the model instance."""
            if hasattr(instance, 'get_upload_to'):
                self.upload_to = instance.get_upload_to(self.attname)
    
        def db_type(self):
            """Required by Django for ORM."""
            return 'varchar(100)'
    

    Comment by Mishu — 3 March 2010 @ 1:25 pm

  21. Thanks Mishu, but there’s a newer official way to do this in Django.

    See:
    http://scottbarnham.com/blog/2008/08/25/dynamic-upload-paths-in-django/

    Comment by Scott — 4 March 2010 @ 5:35 pm

RSS feed for comments on this post.

Leave a comment