vendredi 27 mars 2015

Best RoR uploader for my use-case ( requires id in watermark, s3 storage, resque, and a separate worker-server for processing images)

I have a project that uses Ruby on Rails and currently uses Carrierwave to process file uploads. There are a couple things about the file uploads which are edge-cases and I haven't found a solution that works perfectly.



  1. the id of the image goes in the watermark. Using Carrierwave's process hook, the image has not yet been saved in active record and thus has no ID to watermark. I have gotten around this by saving the image first (which then creates the thumbs and previews) and then overwriting it again with the watermark. This does not seem efficient

  2. offloading file upload to a job on a separate server: we have a file processing server that creates zips, processes IPTC data, creates the watermarks, etc. Currently, our web server is attached to a 10 terrabyte LVM pool and our "worker" server mounts the LVM pool to process the jobs. However I would like ditch our LVM pool in favor of s3.


My (not-necessarily) ideal workflow would be:



  1. Upload a file from the frontend

  2. have backend create a photo record with no photo attached (flag the photo as not yet processed and have the frontend show a gif loader)

  3. Have backend rename the file and upload the raw file to amazon S3

  4. Have backend create a job with the photo-id and with a pointer to the raw file on S3.

  5. "worker server" picks up job, downloads raw file from S3

  6. worker server creates thumb, preview, pulls IPTC data from the raw file

  7. worker server uploads thumb, preview, and deletes the locally stored files

  8. worker server updates the photo record to indicate that the photos have been processed

  9. this is an admin-only feature and I actually don't care that the user would have to refresh the page in order to see the processed thumbnails ( i can cross the bridge when I get there )


Any ideas on this? Would carrierwave, paperclip, or dragonfly support this particular set of requirements?


Aucun commentaire:

Enregistrer un commentaire