Delicatable treats for the rubies.

Been a long time. shouldn’t of left ya without a dope plugin to step to. meh. But for reals, I’ve been working way too much. Here’s a tasty sampler of some new and revived github projects:

• ActiveRecord Dumper
• Asset Tag Extensions
• Ruby Prof Request Filters
• Google Base4r extensions

ActiveRecord Dumper

script/plugin install http://github.com/blythedunham/ar_dumper

Dumper. I barely know her! I wrote this ages ago back in the Rails 1 dot 0h(mg) daze, but still works like a charm.

Easily export records to csv, yaml, or xml using the :find options, or (new) specify an array of activerecords with the :records option. AR Dumper dumps to strings or files with a paginated option, and it supports full customization of displayed headers and content with options such as  nly, :export, :procs, :methods

Book.dumper :yml, :find => {:conditions => ['author_name like ?', 'snowgiraffe'] }

Book.dumper :csv,   nly => [:author_name, :title], :records => @my_books

Asset Tag Extensions

script/plugin install git://github.com/blythedunham/javascript_tag_extensions

Your high performance web site client loading slowly? In addition to combining asset files, one of the best ways to improve client load speed is to move the javascript files and inline script tags to the bottom of the page. Even better, execute inline javascript after the document loads.  To load files at the bottom of the page use the

include_javascript_tag 'myjsfile', :defer => true

Similarly, replacing calls to javascript_tag with Asset Tag Extension’s inline_javascript renders the script at the bottom of the page. Use the n_load => true functionality use JQuery OnLoad functionality.

  <%  inline_javascript   n_load => true do %>
    alert("I love grapes");
  <% end %>

Ruby Prof Request Filters

script/plugin install http://github.com/blythedunham/ruby_prof_request_filters

Need to instantly profile a page? Output ruby prof results to your browser by adding the following params to the url: ruby_prof=true Flat graphs, html graph, whatever you need, this little tool comes in handy when your solving big problems.

Google Base: Base4r

I did some work on Dan Dukeson’s Google Base API For Ruby. Add, modify and delete items from Google Base (formerly Froogle) with Ruby.

Some ppt slides for you. Slide share soon to come.

SMS On Rails rdoc

script/plugin install git://github.com/blythedunham/smsonrails

At the end of the snowgiraffe SMS show, I demoed the sms on rails engine plugin. I’ll build it into a gem soon, but by using the generator there shouldn’t be any collisions, and the plugin should be version safe as everything is namespaced and modularized. Anyhow, here’s the script:

> rails dog --database=mysql
> mysqladmin -uroot create dog_development
> cd dog
> script/plugin install git://github.com/blythedunham/smsonrails
> script/generate sms_on_rails setup --default-service-provider=clickatell
> rake db:migrate
> mate config/environment.rb

Sign up for clickatell and update the environment.rb with your credentials. Alternatively, use the email gateway (specify no default service provider) and setup your mail settings.

The Generator does the following:

1. copy images and stylesheets to your public directory
2. generates 3 database migrations (the 2nd and 3rd have a 2 and 3 appended)
3. installs clickatell gem and static_record_cache plugin (for email carriers caching)
4. insert configuration into environment.rb

Join the club! Rails Devs for Data Integrity

I am a Rails Developer and I believe in data integrity. There I said it! And look, there are lots of people in the club. Jer on Rails had an awesome post in support for foreign keys a while back.

After my talk on supercharging ActiveRecord to behave in an enterprise environment at the MySQL conference, a lot of folks were interested in how to get around the Rails ActiveRecord errors that appear when one starts deifying the Rails Way and uses foreign keys and unique indexes on the database. Well I thought about it and came up with not only a list but a yet another new plugin rails_devs_for_data_integrity to help deal with some of the errors. So when someone tells you not to use foreign keys, don’t listen and join the Rails Devs for Data Integrity! Here is how to have em, and still make pretty Rails apps.

Super list….

Unique Keys

Use validates_uniqueness_of

validates_uniqueness_of introduces a query to check unique records before insert or update, but alone doesn’t guaranty uniqueness in a multi server/multi mongrel environment. By using it conjunction with a unique key on the database, you have proper rails error messages and your safety belt to retain data integrity. In most cases when dealing with a single instance (a single user’s user name for example), I can afford to eat the overhead of the rails validates_uniqueness_of query.

Use on duplicate key update and ignore

ar-extensions 0.9.1 supports ON DUPLICATE KEY UPDATE and IGNORE for MySQL on import, save, and create (all inserts and updates). This is especially practical for import (bulk insert) where validating each record’s uniqueness would produce a lot of overhead.

Catch and Handle the Duplicate Violation Exception

I wrote a lot of custom code in models and controllers to catch MySQL duplicate key error (ActiveRecord::StatementInvalid) and handle them appropriately. Typically, I added a new error to ActiveRecord::Base.errors, which is then displayed nicely with error_messages_for in the view. Sometimes this was done often with a save_safe model method possibly aliased to save or a rescue on the controller action.

After MySQLConf I decided to write a little plugin rails_devs_for_data_integrity to convert exceptions into ActiveRecord errors (like validations do) for tables with unique and foreign keys . Its still infantile (and without tests, gasp!) but if anyone likes to find bugs, hit me up on githubs new issue tracker. If there is more than one unique key per table, you might want write some custom error handling methods.

script/plugin install git://github.com/blythedunham/rails_devs_for_data_integrity.git

 class User < ActiveRecord::Base
   handle_unique_key_violation  :user_name, :message => 'is taken"
   handle_foreign_key_violation :primary_email_id, :message => 'is not available'
 end

Will write ActiveRecord errors instead of nasty MySQL errors:

 >> user.errors.on(:user_name)
 => "association does not exist."

 >> user.errors.on(:primary_email_id)
 => "is a duplicate."

Overwrite rescue_action_in_public in ApplicationController

Its a good idea to either send the user to a pretty 404 static generic error page (fast) or write some custom code in rescue_action_in_public. One idea is to create DisplayableException and subclass any exceptions where the text can be displayed to the user. If this is thrown, show exeception.to_s, if not show a generic error.

def rescues_action_in_public(exception)
  @message = if exception.is_a?(DisplayableException)
    exception.to_s
  else
    "Sorry but an error occurred. Please contact your mommy."
  end
  render :action => 'error_page'
end

Foreign Keys

I have been using the Redhills foreign key migration plugin for a long time and haven’t had too much trouble with foreign key violations.

Trouble Deleting Records

At one point ON DELETE was not specified to SET NULL or CASCADE on many dependent columns and there was trouble deleting rows. Reindexing the database with this option was the solution. Similarly,  one could use ON UPDATE on foreign key indexes.  However, due to the way ActiveRecord works and doesn’t really update id columns, I haven’t had any issues.

Foreign Key Validation Errors

This has happened so seldom that I never worried about it. That’s what the refresh button is for! Perhaps its because most users have their own data and aren’t really modifying and deleting simultaneously. However, sure this can happen and it throws an ActiveRecord::StatementInvalid exception as with unique key violations. I would recommend the same approach as for unique keys: catch and handle the exception. I added foreign key support to the rails_devs_for_data_integrity plugin experiment. Yay!

Today I present Taking Active Record To the Next Level at the MySQL Conference and Expo! There is a lot of excitement in the Rails world due to Rails 3 merge with Merb which will provide framework agnosticism for the people. However, good, old, (sometimes not playing nice with MySQL) ActiveRecord is still the defacto despite the cool newcomers. Don’t worry. I bashed my head against the wall with it for 3 years so you don’t have to. There are lots of tips and tricks for unleashing the power of MySQL and whipping ActiveRecord into enterprise ready shape. Oh, and also included are some really cool pictures like the giraffe on the left.

Customize your primary keys. Pic by Kerri on flickr.

Although the standard MySQL type for id columns is an unsigned int, Rails has no support for them and uses regular signed int(11) as the default for primary keys. In addition, most of my tables are less than 16 million rows (max rows for an unsigned mediumint) and so it saves space to use smaller sized numeric types such as smallint or mediumint to save space.

Inspired by Rob Anderton’s patch which was dismissed as an uncommon requirement, I built yet another plugin to allow customization of primary keys, support unsigned integers and allow users to specify column width for numeric types.

script/plugin install http://github.com/blythedunham/mysql_migration_optimizer/tree/master

Rails ignores signs! pic by alex suarez flikr

Specifying Column Width and Unsigned Integers

The MySQL connection adapter was ignoring the :precision and :scale options and so I just loaded em up to allow unsigned numeric types and allow column width to be specified which was supported on postgresql but not MySQL.

add_column :giraffe, :beer_count, :integer, :limit => 2, :precision => :unsigned, :scale => 4
#SQL: ALTER TABLE `giraffe` ADD `beer_count` smallint(4) UNSIGNED

• :limit determines the numeric type. Rails introduced smart integer columns in 2.1 which more or less translates to the number of bytes needed for storage. Here’s a little map taken from the adapter code. Do be careful, however, as some versions of Rails (2.1) break foreign keys.
  

  :limit	Numeric Type	Column Size
  1	tinyint	1 byte
  2	smallint	2 bytes
  3	mediumint	3 bytes
  nil, 4, 11	int(11)	4 bytes
  5 to 8	bigint	8 bytes

• :scale: column width explained here by Matt Thommes number in parenthesis (4) .
• :precision : specify :signed or :unsigned. Default will be :unsigned unless you specify set MySqlMigrationOptimizer.default_sign = :signed
  in your enviroment.rb.

Customizing Primary Keys

create_table "animal", :force => true,  :primary_column => {:type=>:integer, :limit => 3, :scale=>5}  do |t| end
#SQL: CREATE TABLE `animal` ( `id` mediumint(5) UNSIGNED NOT NULL auto_increment PRIMARY KEY) ENGINE=InnoDB

If you really want to get crazy you can specify some other type. Here’s the rope, I’ll leave the room if you want to hang yourself.

# Specify a string column as the primary key
create_table "blah", :force => true,
:primary_column => {:type=>:string, :limit=>25, :auto_increment=>false} do |t|#
end

#SQL: CREATE TABLE `blah` (`id` varchar(25) NOT NULL PRIMARY KEY) ENGINE=InnoDB

Do be careful to use the same numeric type for the foreign key columns as the referenced id column.

add_column   ther_table, :blah_id, :integer, :limit => 3, :scale=>5

If you have already generated a lot of tables and do not want to change all the old column types, specify the default to be :signed in config/environment.rb. Any new columns should then specify :precision => :unsigned

MySqlMigrationOptimizer.default_sign = :signed

The schemadumper has been amended to work correctly.

Install

script/plugin install git://github.com/blythedunham/static_record_cache.git

Documentation

class SomeStaticClass < ActiveRecord::Base
  acts_as_static_record
end

#cache hits
SomeStaticClass.find_by_name('blah')
SomeStaticClass.find_by_id(5)
SomeStaticClass.find :all
assoc_class.some_static_class

One can specify the cache key for faster lookups and also customize the finder SQL for the class. More documentation can be found in the rdocs.

Another, simpler method of caching can be used by folks using the active record context plugin by technoweenie. Extending StaticRecordCache will permanently cache data for the class regardless of context. This however, only caches by id.

class TelephoneCarriers < ActiveRecord::Base
  extend StaticActiveRecordContext
end

Be sure to install active_record_cache plugin:

script/plugin install http://svn.techno-weenie.net/projects/plugins/active_record_context/

I however like enumerated columns because they save space and are pretty, and so I wrote a short and sweet migration_enum plugin to help me out. However, so be the fate of the web dev that someone else has already written a better one.

The solution is the Enumerated Columns by William Sobel. He even added support for radio buttons in your views!

t.column :severity, :enum, :limit => [:low, :medium, :high, :critical]

The one thing I dislike is that my “legacy” system which has been using enumerated columns interprets them as strings all over the place in the rails code. Enumerated Columns plugins treats them as symbols and breaks me. To fix, I wrote a little patch.

So to get it all:

script/plugin install svn://rubyforge.org/var/svn/enum-column/plugins/enum-column
cd vendor/plugins/enum-columm
curl http://snowgiraffe.com/downloads/rails/enum_patch.diff > enum_patch.diff
patch -p0 < enum_patch.diff

In environment.rb (or an initializer) add:

ActiveRecordEnumerations::Column.use_strings = true

Be careful of folder babies in buckets! photo by Patrish McLean

Back in the ‘burgh at Carnegie Mellon’s radio station WRCT, a really awesome poster with a baby crawling to its death in a bucket reminded you of how dangerous buckets can really be. Especially when DJing techno.  I can’t work with Amazon S3 without wishing all my baby resources and folders and files would crawl in to an S3 death bucket and vanish. OMG AMAZON!

So anyhoo, to create folders instead of just files (or what appears to be folders in the awesome S3 firefox organizer), do a PUT with:

This bitty rails code demonstrates this using the S3 plugin:

  S3_FOLDER_EXT = '_$folder$'
  def exists_in_S3?(file_name, bucket_name, options ={})
    connect_to_S3 

    file_search_name = file_name.dup
    file_search_name << S3_FOLDER_EXT if options[:folder]

    AWS::S3::Bucket.objects(bucket_name, :prefix => file_search_name).any?

  end

  def create_S3_folder(file_name, bucket_name, options={})
    connect_to_S3
    folder = AWS::S3::S3Object.store(file_name + S3_FOLDER_EXT, '', bucket_name,
      {:access => :public_read,
       :content_type => "binary/octet-stream",
       :cache_control => 'max-age=3600,post-check=900,pre-check=3600' }.update(options))
  end

  def connect_to_S3
    unless AWS::S3::Base.connected?
      AWS::S3::Base.establish_connection!(:access_key_id => 'blah', :secret_access_key => 'giraffe!')
    end
  end

At this point, you should ask yourself why you want to create folders in the first place. Because I want to stick my head in a bucket right now.

Event.find :all, :select => 'substring(name, 1, 10) as name'

This produces the follow SQL query using MySQL SUBSTRING function:

select substring(name, 1, 10) as name from events; 

ActiveRecord will pull the truncated name (the first 10 characters) in the function data into the ActiveRecord field name. This can be very useful for offloading some work to MySQL by using its operations rather than performing the same functions in memory with Rails code.

Aliasing methods can be useful when combined with the GROUP BY functionality. For example, the following query will count the number of events that start on the same day.

events = Event.find :all, :select => 'DATE(start_time) as start_time, count(*) as num_events', :group => 'DATE(start_time)'

Notes and Caveats

Be aware that several issues exist when using MySQL aliasing with ActiveRecord.

Eager Loading

One note is is that ActiveRecord doesn’t support :select with eager loading. This means that the :select clause is ignored when :include is specified. To work around this problem (and get a performance benefit by selecting fewer columns), install one of the many eager loading plugins referenced on this ticket. Perhaps there are better ones, but I wrote eload select and have stuck to it.

script/plugin install http://arperftoolkit.rubyforge.org/svn/trunk/eload_select

Aliased Column vs. MySQL Function

Another caveat is that due to a MySQL “Not a Bug”, the full function DATE(start_time) must be specified instead of just the alias start_time.  This might seem obvious, but the  ORDER BY worked the same way. However, ORDER BY clause does the opposite and uses the function aliased to name instead of the column name. For example,here the data is sorted on the second letter of the title (the substring function).

Event.find :all, :select => 'substring(title, 2) as title',   rder =>'title'

which produces this SQL query:

select substring(title, 2) as title from events order by title;

script/plugin install http://arperftoolkit.rubyforge.org/svn/trunk/conditional_page_cache

At the top of the controller, indicate which actions you wish to cache with conditionally_cache.

class MyController < Application

  conditionally_cache :action_one, :action_two

  def cache_page?(current_action_name)
    @user.demo?
  end

  def cache_action?(action_name)
    !@user.admin?
  end

end

To use page caching, add a method cache_page?(current_action_name) on the controller. The page will be cached if this method returns true.

Similarly, return true from method cache_action?(current_action_name) when you want to action cache the page. This method is the same as the one used by the action caching plugin which works great along with this one.