07 2 / 2014

POV: Rediscovering the Joy of Design

Thanks to Rediscovering the Joy of Design, this was truly interesting week for me. Those articles by Adam Hawkins made me rethink my application design, which lead to full scale refactoring of an existing app I was working on. Proposed solutions solved so many problems, and removed all those smelly pieces of code that I didn’t know how to make right. Not to mention that my test suite now runs at the speed of light: 207 examples in a fraction of a second, and yes, most of them are integration tests.

However, there are few things that I have approached differently, and which I would like to write about here.

Form Objects

Form objects serve single purpose: to accept and coerce user input. These objects need to translate what user meant by typing something in. For example, user may type in 2014-01-01. That is clearly a date, but in the context of due_date user probably meant: “by the end of that day”. So, the form should coerce it to TimeWithZone object and shift it to end of day. Luckily, this is quite easy to achieve with Virtus custom attributes.

class Form
  include Virtus.model
  attribute :due_date, Attribute::TimeWithZoneEndOfDay
end

Since form objects are very simple, and highly contextual, I like to define them within a controller that uses them.

class TasksController
  class Form
    include Virtus.model
    # ...
  end

  def new
    @form = Form.new
  end
end

There was a debate on whether forms should handle input validation. After all, that is another responsibility. I wanted to keep my forms simple, so I decided that I will keep them validation-free. Mutations just made this possible for me.

Mutations

Mutations are changes of application state. Adam refers to these as “Use Cases”. One thing I didn’t like about Adam’s approach is passing the form object to use case. This makes reusing use cases in different places harder (e.g. both in web application and its api). For example, in web application user will provide date string for due_date, but in api, we may require user to pass numeric timestamp. That means we’ll have two different form objects, and our use case will need to know how to handle both.

I took another approach. I went with awesome mutations gem. It allows us to specify required and optional parameters. It also supports input validation. So, instead of passing form object as a whole to mutation, we’ll just pass its attributes.

# ...
def create
  @form = Form.new params[:task]
  Tasks::Create.run! @form.attributes, current_user: current_user
end
# ...

When mutation is ran (with !), it will raise an exception with summary of invalid inputs. We can rescue from this exception and present validation errors in our web app, or just respond with 400 in our api.

Authorizations

After lots of thinking (and tons of experiments with different designs), i decided that natural place for authorization logic is within the mutation itself (like Adam suggested). There was one extra requirement: I wanted to be able to check whether specific mutation can be performed by given user upfront (e.g. whether or not to show “edit” button in the UI, or even render the edit page). In order to do so, I simply moved the authorize! call to be class method:

# ...
def execute
  task = TaskRepo.find(id)
  self.class.authorize!(current_user, task)
  task.attributes = inputs.slice(:title, :due_time)
  task.save
end 

def self.authorize!(current_user, task)
  current_user.project_admin?(task.project_id)
end
# ...

Now, we can conditionally show the Edit link:

# ... helper method
def mutable?(mutation, *args)
  mutation.authorize!(current_user, *args)
  true
rescue PermissionDeniedError
  false
end
# ... in views
<% if mutable?(Tasks::Update, task) %>
  <%= link_to 'Edit', '#' %>
<% end %>
# ...

Uniqueness validations

How about uniqueness validation? Mutations::Command provides validate method that we can use to perform custom validations. The code is simple, and it could look like this:

# ...
def validate
  add_error(:email, :taken, "Email is taken") if email_taken?
end

def email_taken?
  # query repo to find out whether email is already taken
end
# ...

The other option would be to implement this kind of check on repo level:

class UserRepo
  def save
    # raise error if email is taken
    super
  end
end

and then to rescue from exception within your mutation to add_error and raise Mutations::ValidationException. Which approach to take really depends on how important the uniqueness is and whether multiple use cases will need to handle this situation.

01 2 / 2014

YouTube-like ids for your ActiveRecord models

Recently I was asked by the client to obfuscate record ids in url and replace them with youtube-like identifiers (e.g. k8jFl). I was searching for a way to do it without having to generate and store such ids into database.

I came accross hashids gem which does just that. It can convert numeric id to string, and vice-versa.

It turns out to be very simple to integrate this into your rails app. All you have to do is to create following initializer (/app/config/initializers):

require 'hashids'

module HashidsSupport
  module ClassMethods
    def hashids
      Hashids.new(table_name, 5)
    end

    def encrypt_id(id)
      hashids.encrypt(id)
    end

    def decrypt_id(id)
      hashids.decrypt(id).first
    end
  end

  def self.included(base)
    base.extend ClassMethods
  end

  def encrypted_id
    self.class.encrypt_id(id)
  end

  def to_param
    encrypted_id
  end
end

ActiveRecord::Base.send :include, HashidsSupport

Hashids.new(table_name, 5) means that table_name will be used as salt, and that id should have minimum lenght of 5. Changing that line line will change your ids, which will make former urls invalid.

That would override to_param method for each of your models to return encrypted id instead of numeric one. The only thing left to do is to decrypt it in your controller actions before trying to find record:

@user = User.find User.decrypt_id(params[:id])

However, this is somewhat cumbersome. It would be better if we could somehow override the find function to do this automatically? Unfortunately this is different for every version of rails, but here is how you can do it in rails 4.1, by monkeypatching ActiveRecord::FinderMethods:

module ActiveRecord
  module FinderMethods
    alias_method :orig_find_one, :find_one
    def find_one(id)
      if id.is_a?(String)
        orig_find_one decrypt_id(id)
      else
        orig_find_one(id)
      end
    end
  end
end

Now we can use the find method like we normally do, and we don’t need to change single line of our app. As bonus benefit, former links (with numeric ids) will still work since find method is capable of handling both.

12 1 / 2014

Loading all i18n translations on page at once

Recently airbnb posted cool article on hammerspace. Following line got my attention:

Fetching all translations for a page at once is not straightforward, since the strings required to render a page are not known ahead of time.

I have battled similar problem in past, and even though it is true that we cannot know them all upfront, we can, however register and encode them during runtime, and then load them all at once and decode individually.

I have played with it for few hours in an proof of concept rails app (https://github.com/assembler/bulk18n).

The application overrides rails translate (t) helper to return encoded version of translation and store its key and arguments in memory. For example t('greeting', user: 'User 1') would store the translation request at index 0 in registry and save optional parameters as well. It returns an object with following string representation: \u0002 0 \u0003. It just have two invisible utf chars and translation index in between (in this case 0).

Given this template:

<p><%= t 'greeting', user: 'User 1' %></p>
<p><%= t 'greeting', user: 'User 2' %></p>

It will produce this:

<p>\u00020\u0003</p>
<p>\u00021\u0003</p>

Next, it processes entire response.body (in around_action filter) and just decodes these values to proper translations. At this point we can have ALL the keys that we need to translate, and we create the dictionary by doing SINGLE query to our preferred data store. Then we can simply decode them all using that dictionary.

Since we’re also storing additional options passed (:user in this example), we’re able to do the interpolation. The end result is as expected:

<p>Hello User 1</p>
<p>Hello User 2</p>

There is one final problem to tackle. What if we want to perform additional string manipulations of translated values. For example, we want to upcase translated string:

<p><%= t('greeting', user: 'User 1').upcase %></p>

To solve the problem, returned value from t method is really not a string. It is an object which is represented as a string, but captures all method_missing calls and saves them to be replayed later on. Kinda like null object but the one which memorizes everything and takes revenge later :)

Feel free to check out the test app at https://github.com/assembler/bulk18n. Its just proof of concept app used to demonstrate an idea. I think that this concept has potential in other areas as well (such as cache personalization in rails i wrote about earlier).

16 9 / 2013

Starting up fake_dynamo automatically with rspec test suite

All you have to do is create support file (e.g. /spec/support/dynamo.rb) which will boot the fake_dynamo before test suite, and shut it down after it:

RSpec.configure do |config|
  dynamo_thread = nil

  config.before(:suite) do
    FakeDynamo::Storage.db_path = '/usr/local/var/fake_dynamo/db.fdb'
    FakeDynamo::Logger.setup(:warn)
    FakeDynamo::Storage.instance.load_aof

    dynamo_thread = Thread.new do
      FakeDynamo::Server.run!(port: 4567, bind: '127.0.0.1') do |server|
        if server.respond_to?('config') && server.config.respond_to?('[]=')
          server.config[:AccessLog] = []
        end
      end
    end
  end

  config.after(:suite) do
    FakeDynamo::Storage.instance.shutdown
    dynamo_thread.exit
  end
end

23 8 / 2013

Memory usage monitor in Ruby

Here is useful utility I came up with which measures memory consumption in different thread while the process is running. You can use it to log memory consumption and find out memory peaks.

require 'thread'

class MemoryUsageMonitor
  attr_reader :peak_memory

  def initialize(frequency: 0.25)
    @frequency = frequency
    @peak_memory = 0
  end

  def start
    @thread = Thread.new do
      while true do
        memory = `ps -o rss -p #{Process::pid}`.chomp.split("\n").last.strip.to_i
        @peak_memory = [memory, @peak_memory].max
        sleep @frequency
      end
    end
  end

  def stop
    Thread.kill(@thread)
  end
end

Usage is simple:

require_relative 'lib/memory_usage_monitor'

mm = MemoryUsageMonitor.new
mm.start

sum = 0
items = []
5_000_000.times do |n|
  sum += n
  items << n.to_s if rand > 0.8
end

mm.stop
puts "Peak memory: #{mm.peak_memory/1024} MB"

Tags:

Permalink

13 8 / 2013

Storing latest Rails log messages in Honeybadger context

More often than not, exception messages and stack trace are not enough when dealing with context-related bugs. Here is how you can insert rails log messages into honeybadger context.

First, we need to create decorate rails logger with CachedLogger decorator. It could look like this:

class CachedLogger < SimpleDelegator
  def initialize(*args)
    @@__chache = Buffer.new(20)
    super
  end

  %w[debug info warn error fatal].each do |level|
    class_eval %{
      def #{level}(*args)
        @@__chache.push args.map(&:inspect).join(" | ")
        super
      end
    }
  end

  def read_cache
    @@__chache.read
  end
end

module RailsCachedLogging
  class Railtie < ::Rails::Railtie
    config.after_initialize do
      ::Rails.logger = CachedLogger.new(::Rails.logger)
    end
  end
end

Whenever debug, info, warn, error or fatal method is called, arguments (notably: log message) will be pushed into buffer. Buffer has limited capacity to avoid memory leaks. Implementation of buffer can be as follows:

class Buffer
  def initialize(capacity)
    @capacity = capacity
    @values = []
  end

  def push(obj)
    @values.push obj
    @values.shift if @values.length > @capacity
    obj
  end

  def read
    @values
  end
end

Natural place to store these files would be in config/initializers/logger.rb. Now, when exception occurs, you can update the context like this:

rescue Exception => exc
  Honeybadger.context({ log: Rails.logger.read_cache })
  Honeybadger.notify_or_ignore(exc)
end

You can tweak this to your needs, but essentially thats it :)

28 5 / 2013

Benchmarking gem load times

There is an awesome gist written by Pan Thomakos, and later refined by Andrew Kane here: https://gist.github.com/ankane/5022636

Here is simple fish shell function that can be used to run this script in any project you need:

function benchmark_bundler
  set -x BUNDLE_GROUPS $argv
  curl -fsSL https://gist.github.com/raw/5022636/benchmark.rb | ruby
end

Just put it in your ~/.config/fish/config.fish file and off you go.


UPDATE: rails4 fork

curl -fsSL https://gist.github.com/raw/6121664/benchmark.rb | ruby

27 3 / 2013

Monitoring memory usage on long-running scripts

There is a simple way to monitor memory usage on long-running scripts. You can just create new thread which will output current memory usage to console in short time intervals:

Thread.new do
  while true do
    memory = `ps -o rss -p #{Process::pid}`.chomp.split("\n").last.strip.to_i
    puts "Memory: #{memory/1024} MB"
    sleep 0.25
  end
end

Tags:

Permalink

20 3 / 2013

More readable specs with dynamic date methods

Often times, in my specs, i use variables like jan_01 to represent dates. It makes my specs much more readable. For example:

date = SequentialDate.new(jan_01)
expect(date.next).to eq(jan_02)

I used to create these variables by hand, but there is much easier way to do it by utilising pure awesomeness of ruby. Just define following module in your support rspec dir:

require 'date'
module Support
  module DateHelpers
    def method_missing(name, *args, &amp;block)
      begin
        Date.parse(name.to_s)
      rescue ArgumentError
        super
      end
    end
  end
end

And require it in rspec config file:

RSpec.configure do |config|
  config.include Support::DateHelpers
end

The beauty of it is that it wil parse whatever Date.parse is able to handle. Some examples are:

  • jan_01
  • jan_1st
  • jan_01_2013
  • _3rd_feb_2013
  • _01_JAN_2013
  • february_14th_2013

13 3 / 2013

Super fast way of launching postgres console for rails apps

Database console can be run with bundle exec rails dbconsole command. However, this operation performs relatively slow (due to bundler and deps).

We can make it much faster with simple ruby script:

#!/usr/bin/env ruby

require 'yaml'
require 'fileutils'

database_config = YAML.load_file(Dir.pwd + '/config/database.yml')['development'];
database_name = database_config['database']

exec('psql', "-d#{database_name}")

Just save the script somewhere in your $PATH and give it u+x permission.

I like to place scripts like these in my ~/dev/bin folder (included in $PATH). I gave this one db name, so the only thing i need to do to fire up postgres console is to type db. Simple and extremely fast.