Creating Folders Programmatically with Amazon S3’s API: putting babies in buckets

February 5, 2009
Tags: , , , , , , , ,
Be careful of folder babies in buckets! photo by Patrish McLean

Be careful of folder babies in buckets! photo by Patrish McLean

Back in the ‘burgh at Carnegie Mellon’s radio station WRCT, a really awesome poster with a baby crawling to its death in a bucket reminded you of how dangerous buckets can really be. Especially when DJing techno.  I can’t work with Amazon S3 without wishing all my baby resources and folders and files would crawl in to an S3 death bucket and vanish. OMG AMAZON!

So anyhoo, to create folders instead of just files (or what appears to be folders in the awesome S3 firefox organizer), do a PUT with:

  • header content_type to binary/octet-stream
  • append _$folder$ to the end of the path name (as the extension)
  • set the data to blank

This bitty rails code demonstrates this using the S3 plugin:

  S3_FOLDER_EXT = '_$folder$'
  def exists_in_S3?(file_name, bucket_name, options ={})
    connect_to_S3 

    file_search_name = file_name.dup
    file_search_name << S3_FOLDER_EXT if options[:folder]

    AWS::S3::Bucket.objects(bucket_name, :prefix => file_search_name).any?

  end

  def create_S3_folder(file_name, bucket_name, options={})
    connect_to_S3
    folder = AWS::S3::S3Object.store(file_name + S3_FOLDER_EXT, '', bucket_name,
      {:access => :public_read,
       :content_type => "binary/octet-stream",
       :cache_control => 'max-age=3600,post-check=900,pre-check=3600' }.update(options))
  end

  def connect_to_S3
    unless AWS::S3::Base.connected?
      AWS::S3::Base.establish_connection!(:access_key_id => 'blah', :secret_access_key => 'giraffe!')
    end
  end

So your request path would look like:
/baby_bucket?prefix=folder1/folder2_$folder$
And the output:

<ListBucketResult xmlns=”http://s3.amazonaws.com/doc/2006-03-01/”>
<Name>baby_bucket</Name>
<Prefix>folder1/folder2_$folder$</Prefix>
<Marker></Marker>
<MaxKeys>1000</MaxKeys>
<IsTruncated>false</IsTruncated>
</ListBucketResult>

At this point, you should ask yourself why you want to create folders in the first place. Because I want to stick my head in a bucket right now.

Filed under:
Amazon S3, Ruby on Rails by blythe

Leave a Reply