Snow Leopard upgrades a number of scripting languages (including Ruby) to be 64bit which is all well and good until… you try and use a gem with native extensions. Any gems which mention “Building native extensions” during the install process will need to be upgraded. There already exist a number of upgrade guides for databases and the like, but ruby-debug seems to have been forgotten.
To get ruby-debug (and debugging in a rails application) to work:
sudo gem uninstall ruby-debug sudo gem uninstall linecache sudo gem install ruby-debug
You’ll have fresh new copies of linecache and ruby-debug ready for you to use on Snow Leopard.
Key Value Coding is a simple extension to the observer pattern. Rather than blindly observing ‘change’ in an object (where any property could have changed), observers register to be informed about changes to specific ‘keys’ or properties of an object. This allows you to bind different elements in your web app and have them all magically stay in sync. An object representing a record of some sort (say an address contact) can inform all the user interface elements observing it of a change to any of its values. You could have a text field that sets the email property of the record; all elements observing the property (such as a P tag) are automatically informed whenever the property changes, and have a chance to update themselves. The pattern decouples model and view, allowing you to add, edit and remove views without ever having to change your model.
Implementing the pattern is easy in JavaScript because all objects act like hashtables. To glue the whole process together we need a few things:
- An interface for communicating with observers
- An interface for registering observers
- A way to intercept changes to values
An interface for communicating with observers
Observers are informed whenever a property they are watching changes. Because an observer may be watching more than one property of an object (and even more than one object), we need to supply the key (property name) along with the value, as well as the object being updated. It’s often also useful to know the previous value of the property, so observers can test to see if a value was actually updated, or merely re-assigned to.
For example, say we have an object representing a square element on a page that can be resized and dragged. The object could have four properties: top, left, width, height. The code that implements the dragging and resizing assigns values to all four of these properties live as the square’s position and dimensions are being changed. Something as simple as a P element could be observing the top property and change its text to match the value of the property whenever it changes. But during a resize, the top and left values won’t change; only the width and height will be updated. Because the resize operation assigns to all four properties, we’ll trigger a notification on all four, even though the values for top and left haven’t actually changed.
To be able to support this behaviour, we can use a callback defined like this:
keyWillUpdate(object, key, previousValue, newValue)
The P element could implement this function like this:
document.getElementById("our_p_tag").keyWillUpdate = function(object, key, previousValue, newValue) {
if(previousValue == newValue) return;
this.innerHTML = newValue;
}
This will ignore updates where the value has been assigned to but not changed.
An interface for registering observers
When an observer registers with a KVC object we need to know which property it is observing, and to keep track of all the observers for each property. A simple register function on an object could look like:
function registerObserver(observer, key) {
if(this.observers == null)
this.observers = {};
if(this.observers[key] == null)
this.observers[key] = [observer];
else
this.observers[key].push(observer);
}
Here we use a hashtable (observers) to store the list of observers for each property. The key of the observers table is the property being observed, and the value is an array of objects observing the property. When it comes time to notify observers of an assignment to a property, we can easily iterate through the array.
A way to intercept changes to values
Unfortunately there’s no sneaky way to do this cleanly in Javascript. Other languages allow you to catch method calls that don’t match a defined method (e.g method_missing in Ruby), but we can’t do that in Javascript. Instead we need to define a set function that must be called when assigning to a value for our KVC interface to work correctly. Because no KVC operations are performed when retrieving a value, technically we don’t need a corresponding get function, but for completeness and consistency it’s good to have. We could implement these functions like this:
function get(key) {
return this[key];
}
function set(key, value) {
if(this.observers != null && this.observers[key] != null) {
currentValue = this[key];
for(var i = 0; i < this.observers[key].length; i++)
this.observers[key][i].keyWillUpdate(this, key, currentValue, value);
}
this[key] = value;
}
A call to a KVC object like square.set(‘top’, 20) will now inform all observers of top property of the new assignment to that property.
Why is this useful?
For a simple web app KVC may seem like overkill, however the benefits are immense for medium to large apps. I recently wrote an app that had a draggable calendar interface – click to create events on a day, drag events around to indicate days and time, and resize events to indicate length of time. There were two bits to the UI for each event: the draggable box on the calendar, and a set of text fields where you could manually type in times and other options for the event.
Keeping the two in sync could have been difficult, and if a third UI element was added to track the event it would have been even more unmanageable. Using KVC, all UI elements observed the start and end times for each event. When a text field updated the start time of an event, the draggable box observing the event was informed of the change and updated it’s display accordingly. Vice versa, when the event box was dragged within the calendar, it updated the times in the event model, and the text fields showing the start and end times were immediately updated to reflect the new times.
The model didn’t need to ‘know’ about either of the UI views, and neither view had to know about each other. It decoupled all elements from each other and made synchronisation simple. Adding a third UI element to track the event would have been easy, and again, would ‘magically’ stay in sync with the rest of the elements.
Sample code
A sample page showing a sample implementation of a KVC object constructor, a checkbox setting a property in a KVC object, and a P tag observing the property.
I came across this screen capture I made in 2003 when I was using Powerpoint in the Classic environment in OS X. I have no idea what conditions made the easter egg appear, and I couldn’t replicate it (choosing ‘Save’ again after cancelling brought up the correct list of files). “Moof” is the name Clarus (the dogcow) makes, Clarus being the mascot of Apple’s dev tech support group.
Mongrel has an annoying ‘feature’ where it will bail out of startup if it finds a PID file already existing where it wants to write one. Without checking if the PID is actually stale or not, it will assume a server already exists and exit. Mongrel cluster has a –clean flag that will remove all existing PID files, but using Mongrel cluster makes it difficult to use SMF to keep all individual Mongrel instances up.
To get around this, we need to remove the PID file as part of the start script in our SMF manifest. e.g:
rm -f /path/to/pid/file
Unfortunately, chaining commands together with ‘;’ will cause SMF to think the script is exiting prematurely – rm will return 0 and exit, making SMF think the service has died. Instead we need to ‘hack’ it to work. Mongrel ignores anything on STDIN, so we can pipe the rm command to Mongrel, and all will work nicely:
rm -f /path/to/pid/file | /opt/local/bin/mongrel start ...
Capistrano is a great tool for running batches of shell commands on remote servers. It’s mostly used in the Rails community, but is helpful for deploying non-rails apps, or even running commands unrelated to web apps at all.
All our apps are run on Solaris servers and managed by SMF, and we use a series of manifests to control each individual thin instance (named thin/port4000…thin/port4014). Getting Capistrano to play with SMF (so we can restart the app servers with the new code) is luckily pretty simple. We could create our own spin and reaper scripts, but it’s nicer to keep the commands in the deploy script. We can tell Cap to run our commands like this:
namespace :deploy do
task :start, :roles => :app do
run "/usr/sbin/svcadm enable network/thin/*"
end
end
The stop task is similar, but calling svcadm with disable instead. The majority of the time we will be deploying a new version of the app to production, and it’s the restart task that gets called. Restart can be composed of the stop and start tasks:
namespace :deploy do
task :restart, :roles => :app do
stop
start
end
end
By default, SMF only allows the root user to control SMF managed processes. For security we disable root logins over ssh, so we need a way for lower privileged users to control our process instances. Role Based Access Control (RBAC) is a clean way of allowing a user to perform specific tasks. To begin, we need to add a new authorisation. Add this line to the end of /etc/security/auth_attr:
solaris.smf.manage.thin:::Manage Thin::
This is simply to tell Solaris there is a new authorisation – we could have named it anything. To be able to control SMF using this authorisation, we need to apply it the instances we want to control. There are two types of authorisation we need to apply to each instance: the action authorisation, and the value authorisation. Action is used to perform an action on an instance (e.g restart), value is used to change a value on an instance (e.g set the enabled/disabled state). We need to do this once for each thin instance we are controlling:
svccfg -s network/thin/port4000 setprop general/value_authorization=astring: 'solaris.smf.manage.thin' svccfg -s network/thin/port4000 setprop general/action_authorization=astring: 'solaris.smf.manage.thin'
Finally we need to apply this authorisation to the user we want to give control to:
usermod -A solaris.smf.manage.thin admin
Admin will now be able to completely start and stop our thin instances now:
svcadm restart network/thin/port4000 svcadm enable network/thin/port4000 svcadm disable network/thin/port4000 svcadm refresh network/thin/port4000
Solaris can sometimes seem a little foreign to people used to Linux or OS X, but its network performance and admin tools make it an excellent platform for hosting web apps. We decided on using nginx as our static file server, and load balancer, and thin as our rails app server. I won’t go in to details of getting these all up and running on Solaris, there’s plenty of resources already available for that.
nginx
Depending on the way you compile nginx, it may already default to using Solaris eventports. Event port is Solaris’ fast asynchronous blocking IO call, similar to epoll in Linux and kqueue in BSD/OS X. To ensure you use it, add a definition to events:
events {
worker_connections 2048;
use eventport;
}
One of the great admin tools in Solaris is SMF (Service Management Facility). It’s similar to launchd on OS X – built in to the kernel, it keeps your app up and running. Unlike monit there’s no lag time between checks on the process; the instant a process dies SMF restarts it. You (unfortunately) use XML files called manifests to tell SMF what to watch, and how to start/stop/refresh it. You can also define dependencies so your app doesn’t start up before network services are ready, for instance. This is the manifest we use for nginx:
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='nginx'>
<service name='network/nginx' type='service' version='0'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='fs' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<dependency name='net' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/network/loopback'/>
</dependency>
<exec_method name='start' type='method' exec='/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf' timeout_seconds='60'>
<method_context working_directory='/usr/local/nginx/'>
<method_credential user='root' group='root'/>
</method_context>
</exec_method>
<exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'>
<method_context/>
</exec_method>
<exec_method name='refresh' type='method' exec='/bin/kill -HUP `cat /usr/local/nginx/logs/nginx.pid`' timeout_seconds='60'>
<method_context/>
</exec_method>
</service>
</service_bundle>
- There is a single instance of this process
- Wait until the file system is started
- Wait until the network is started
- Define the start method (command line that’s run to start the service). We also set the CWD and current user.
- Define the stop method (using the special :kill helper)
- Define how to refresh the nginx config (send nginx a HUP signal)
Save this and import it into your system manifest with:
svccfg import nginx.xml
This will check the validity of the XML file, import it, and start the service.
thin
There’s no config files required for thin, instead we supply the config as flags to the process. The SMF manifest we use for thin places the multiple thin processes in their own “thin” hierarchy under network/thin.
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='thin/port4000'>
<service name='network/thin/port4000' type='service' version='0'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='fs' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<dependency name='net' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/network/loopback'/>
</dependency>
<dependent name='thin_port4000' restart_on='none' grouping='optional_all'>
<service_fmri value='svc:/milestone/multi-user'/>
</dependent>
<exec_method name='start' type='method' exec='/usr/local/bin/thin -e production -d -p 4000 -c /app/current -l /app/current/log/production.log -P /app/current/tmp/pids/thin4000.pid start' timeout_seconds='60'>
<method_context working_directory='/var/log'>
<method_credential user='root' group='root'/>
<method_environment>
<envvar name='PATH' value='/usr/bin:/bin:/usr/local/bin'/>
</method_environment>
</method_context>
</exec_method>
<exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'>
<method_context/>
</exec_method>
</service>
</service_bundle>
One of these manifests is used per thin instance we want to keep up. So, for instance, if we had a second thin server running on port 4001, we would make a copy of this manifest and change the appropriate port numbers throughout the file (both for the name of the service, and in the start method). This way SMF will only restart a single thin process at a time if any fail.
By placing the processes under their own hierarchy, we can reference all processes with one pattern. e.g to restart every thin server, we can issue one command, regardless of how many thin processes we have set up:
svcadm restart `svcs -H -o FMRI network/thin/*`
I recently needed to pass a WSSE authentication header with some SOAP messages sent from a Rails app to a government service. Ruby’s native SOAP library is soap4r. It’s featureful, but under-documented, so it took some fiddling to get WSSE to work.
WSSE is a pretty simple standard for sending a username and password to a service in a SOAP message’s header. The layout is:
<wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext" soap:mustUnderstand="0">
<wsse:UsernameToken>
<wsse:Username>Username</wsse:Username>
<wsse:Password>Password</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
Constructing the WSSE headers
Adding headers in soap4r requires constructing an object that responds to a method that will return the headers to be added. A helper class provided for this is SOAP::Header::SimpleHandler. SimpleHandler allows us to respond with a hash that is converted into XML.
SimpleHandler also expects subclasses to call its initialize method with an enclosing tag. We can use xsd4r (the XML parser/constructor used by soap4r) to create the tag and define its namespace:
XSD::QName.new(NAMESPACE, 'Security')
Our call back method is a hash containing the username and password to be sent:
{"UsernameToken" => {"Username" => USERNAME, "Password" => PASSWORD}}
The entire class looks like this:
require 'soap/header/simplehandler'
class WsseAuthHeader < SOAP::Header::SimpleHandler
NAMESPACE = 'http://schemas.xmlsoap.org/ws/2002/07/secext'
USERNAME = 'username'
PASSWORD = 'password'
def initialize()
super(XSD::QName.new(NAMESPACE, 'Security'))
end
def on_simple_outbound
{"UsernameToken" => {"Username" => USERNAME, "Password" => PASSWORD}}
end
end
Adding the WSSE headers
Adding the headers to a soap4r driver is quite easy (this has only been tested with a driver that has been auto-generated from a WSDL file).
require 'soap_driver.rb' require 'wsse_authentication.rb' d = Driver.new ... d.headerhandler << WsseAuthHeader.new()
Postgres has support for 8 byte integers with bigint’s, and ActiveRecord natively supports reading & writing these values in Postgres (both on 32 and 64 bit machines, thanks to Fixnum and Bignum). Using them in AR migrations is actually pretty easy as well, but there’s some fudgery required to use 8 byte ints as an id if we don’t want to patch AR.
To create an 8 byte field:
create_table :example do |t| t.integer :eight_byte_integer, :limit => 8 end
To create a table with an 8 byte id without patching:
create_table :example do |t| ... fields end change_column :example, :id, :integer, :limit => 8
Note: id sequences are automatically created as bigint’s, so all we’re doing here is allowing the table’s id field to store all the values the sequence will create.
Alternatively, we can patch ActiveRecord to automatically make all primary keys bigserial’s. It’s better to patch a frozen version of Rails (in your vendor folder) than your installed gem.
file: vendor/rails/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
@@ -283,7 +283,7 @@
def native_database_types #:nodoc:
{
- :primary_key => "serial primary key",
+ :primary_key => "bigserial primary key",
:string => { :name => "character varying", :limit => 255 },
:text => { :name => "text" },
:integer => { :name => "integer" },
