Creating Folders Programmatically with Amazon S3’s API: putting babies in buckets

blythe — Fri, 06 Feb 2009 06:02:47 +0000

Be careful of folder babies in buckets! photo by Patrish McLean

Back in the ‘burgh at Carnegie Mellon’s radio station WRCT, a really awesome poster with a baby crawling to its death in a bucket reminded you of how dangerous buckets can really be. Especially when DJing techno. I can’t work with Amazon S3 without wishing all my baby resources and folders and files would crawl in to an S3 death bucket and vanish. OMG AMAZON!

So anyhoo, to create folders instead of just files (or what appears to be folders in the awesome S3 firefox organizer), do a PUT with:

header content_type to binary/octet-stream
append _$folder$ to the end of the path name (as the extension)
set the data to blank

This bitty rails code demonstrates this using the S3 plugin:

  S3_FOLDER_EXT = '_$folder$'
  def exists_in_S3?(file_name, bucket_name, options ={})
    connect_to_S3 

    file_search_name = file_name.dup
    file_search_name << S3_FOLDER_EXT if options[:folder]

    AWS::S3::Bucket.objects(bucket_name, :prefix => file_search_name).any?

  end

  def create_S3_folder(file_name, bucket_name, options={})
    connect_to_S3
    folder = AWS::S3::S3Object.store(file_name + S3_FOLDER_EXT, '', bucket_name,
      {:access => :public_read,
       :content_type => "binary/octet-stream",
       :cache_control => 'max-age=3600,post-check=900,pre-check=3600' }.update(options))
  end

  def connect_to_S3
    unless AWS::S3::Base.connected?
      AWS::S3::Base.establish_connection!(:access_key_id => 'blah', :secret_access_key => 'giraffe!')
    end
  end

So your request path would look like:
/baby_bucket?prefix=folder1/folder2_$folder$
And the output:

baby_bucket
folder1/folder2_$folder$

1000
false

At this point, you should ask yourself why you want to create folders in the first place. Because I want to stick my head in a bucket right now.

Snow Giraffe Tech » bucket

Creating Folders Programmatically with Amazon S3’s API: putting babies in buckets