Metaprogramming All The Way Down

A few months ago I started working my way through some five years of Giant Robots Smashing Into Other Giant Robots episodes. In episode 3 Josh Clayton, the maintainer of FactoryBot, described one particular feature as involving “metaprogramming all the way down”. The phrase piqued my curiosity, so I cloned the repository and began poking around.

What follows is a tour of some of my favorite parts of FactoryBot, with an emphasis on metaprogramming techniques. I hope the reader will learn a few new tricks, and will get familiar enough with the library to explore more on their own and perhaps contribute.

This post is NOT a guide on how to use FactoryBot. For that I recommend reading this guide and watching this video.

What is self?

FactoryBot offers a fairly straightforward DSL for creating factories. Calling the factory method with a symbol :user creates a factory for the User class.

class User
end

FactoryBot.define do
  factory :user
end

p FactoryBot.build(:user)
#=> #<User:0x007ff9b2595980>

But where does the factory method come from inside that block? Normally when we call a method with a block, that block will retain access to the variables and methods from the scope in which it was defined. self inside the block will be the same as self directly outside the block.

module FactoryBot
  def self.define
    yield
  end
end

p self
#=> main

FactoryBot.define do
  p self
  #=> main
end

But there is no factory method defined on main. Instead, the factory method definition appears in FactoryBot::DSL. So how do we have access to a method defined in a totally different context?

Ruby’s instance_eval allows precisely that. Rather than evaluate the block in its usual context with yield or Proc#call, FactoryBot uses instance_eval to change the block’s evaluation context. self inside the block becomes the instance_eval receiver, in this case an instance of FactoryBot::DSL.

module FactoryBot
  def self.define(&block)
    DSL.new.instance_eval(&block)
  end

  class DSL
    def factory(name)
      # ...
    end
  end
end

p self
#=> main

FactoryBot.define do
  p self
  #=> FactoryBot::DSL
end

instance_eval was not the only reasonable solution here. Passing the FactoryBot::DSL instance as an argument to the block would have worked just fine, and would have been a little faster. But it also would have added unnecessary noise to our factories. I think the trade off makes sense here; we are only likely to call define a handful of times anyway, and instance_eval allows for a cleaner DSL.

module FactoryBot
  def self.define
    yield DSL.new
  end

  class DSL
    def factory(name)
      # ...
    end
  end
end

FactoryBot.define do |dsl|
  dsl.factory :user
end

You can read more about FactoryBot’s use of instance_eval (and instance_exec) here. The important thing to remember when looking through the source code is that self is not always what you think it is.

Undefining Methods

The factory method builds a FactoryBot::Factory, populates its FactoryBot::Definition, and registers the new factory in a hash-like object for later lookup. To populate the factory definition with attributes, associations, etc., we can call the factory method with a block. Can you guess what self is inside this block? Let’s take a look:

FactoryBot.define do
  factory :user do
    p self
    # What is self?
  end
end

FactoryBot.build(:user)

#=> NoMethodError: undefined method `name' for :user:Symbol

Oops! That didn’t work at all. Thanks to instance_eval, self inside the block is an instance of FactoryBot::DefinitionProxy. But you won’t have much luck inspecting an instance of that class. You will have to either take my word for it or go check out the source code. That is because the class uses undef_method to remove all but a few allow-listed methods. It’s not easy to get a good look at an object that doesn’t respond to to_s, inspect, etc.

class DSL
  def factory(name, &blk)
    factory = Factory.new(name)

    proxy = DefinitionProxy.new(factory.definition)
    proxy.instance_eval(&blk) if block_given?

    FactoryBot.register_factory(factory)
  end
end

class DefinitionProxy
  UNPROXIED_METHODS = %w(__send__ __id__ nil? send object_id extend instance_eval initialize block_given? raise caller method)

  (instance_methods + private_instance_methods).each do |method|
    undef_method(method) unless UNPROXIED_METHODS.include?(method.to_s)
  end

  def initialize(definition)
    @definition = definition
  end

  # ...
end

Undefining methods on the proxy prevents those methods from interfering with any attributes on the object being constructed. Want to build a factory for a class like this:

class User
  attr_accessor :to_s, :inspect
end

FactoryBot.define do
  factory :user do
    to_s "why you overwrite me?"
    inspect "this seems like a bad idea"
  end
end

user = FactoryBot.build(:user)

puts user.class
#=> User

puts user
#=> why you overwrite me?

p user
#=> this seems like a bad idea

No Problem! (But please don’t actually write a class like that.) FactoryBot does provide a way to define attributes conflicting with reserved words or methods, but undefining methods minimizes the need for that.

Method Missing

Although FactoryBot::DefinitionProxy begins by undefining methods, it still responds to any method we call on it. username and email are not defined on the class, but calling them does not raise a NoMethodError.

FactoryBot.define do
  factory :user do
    username 'dodecadaniel'
    email 'daniel@example.com'
  end
end

This is because FactoryBot::DefinitionProxy overrides method missing. While Object#method_missing raises a NoMethodError, FactoryBot::DefinitionProxy#method_missing creates a FactoryBot::Declaration and adds it to the factory’s definition.

class DefinitionProxy
  # ...

  def method_missing(name, value)
    declaration = Declaration::Static.new(name, value)
    @definition.declare_attribute(declaration)
  end
end

I am leaving some things out, but this should give you an idea what the purpose of the definition proxy is. It offers an expressive DSL for adding attributes, associations, traits, and so on to the factory’s definition. Without method_missing factories might look like this:

FactoryBot.define do
  factory :user do
    add_attribute 'username', 'Daniel'
    add_attribue 'email', 'daniel@example.com'
  end
end

This is not terrible. It does simplify the FactoryBot code quite a bit; getting rid of method_missing would make undef_method unnecessary. But the calls to add_attribute add noise to our factories. As with instance_eval, method_missing provides for a cleaner, more elegant DSL.

If we ditched the proxy altogether, and avoided both method_missing and instance_eval, we might have factories like this:

 FactoryBot.define do |dsl|
   dsl.factory :user do |definition|
     defintion.declare_attribute(Declaration::Static.new('username', 'Daniel')
     definition.declare_attribute(Declaration::Static.new('email', 'daniel@example.com')
   end
 end

Gross! I just want to build a user; why do I need to know about definitions and declarations? I don’t think FactoryBot would be nearly as popular as it is today if factory definitions were that clumsy.

Strategy Pattern

After defining a factory, we can construct our object using a number of built-in methods:

FactoryBot.attributes_for(:user)
FactoryBot.build(:user)
FactoryBot.build_stubbed(:user)
FactoryBot.create(:user)

These methods are easy enough to define. We need to look up the factory, call its compile method, do a few other things that I left out here, and then call the appropriate strategy method.

module FactoryBot
  def self.attributes_for(name)
    factory = FactoryBot.factory_by_name(name)
    factory.compile
    # ...
    factory.attributes_for
  end

  def self.build(name)
    factory = FactoryBot.factory_by_name(name)
    factory.compile
    # ...
    factory.build
  end

  def self.build_stubbed(name)
    factory = FactoryBot.factory_by_name(name)
    factory.compile
    # ...
    factory.build_stubbed
  end

  def self.create(name)
    factory = FactoryBot.factory_by_name(name)
    factory.compile
    # ...
    factory.create
  end

  class Factory
    def attributes_for
      # ...
    end

    def build
      # ...
    end

    def build_stubbed
      # ...
    end

    def create
      # ...
    end
  end
end

But there are some problems with this code. For one, it is not particularly DRY. This could make it more difficult to make changes that affect all of the strategies. If, for example, we need to add a step after factory.compile (e.g. factory = factory.with_traits(@traits)) we will need to add it to each one of these methods. FactoryBot solves this problem by moving the shared logic into FactoryBot::FactoryRunner.

module FactoryBot
  def self.attributes_for(name)
    FactoryRunner.new(name, :attributes_for).run
  end

  def self.build(name)
    FactoryRunner.new(name, :build).run
  end

  def self.build_stubbed(name)
    FactoryRunner.new(name, :build_stubbed).run
  end

  def self.create(name)
    FactoryRunner.new(name, :create).run
  end

  class FactoryRunner
    def initialize(name, strategy)
      @name = name
      @strategy = strategy
    end

    def run
      factory = FactoryBot.factory_by_name(@name)
      factory.compile
      # ...
      factory.public_send(@strategy)
    end
  end
end

DRYing the code improves matters, but this still requires FactoryBot::Factory to have knowledge of every construction strategy. Adding a new strategy would require adding methods to both FactoryBot and FactoryBot::Factory. That would make it impossible for us to add custom strategies without altering the source code or monkey patching.

Instead, FactoryBot uses the Strategy Pattern. Rather than provide separate methods for each strategy, FactoryBot::Factory offers a single run method that takes the name of a strategy as its first argument. It then looks up the strategy at runtime and calls its result method.

class FactoryRunner
  def run
    factory = FactoryBot.factory_by_name(@name)
    factory.compile
    # ...
    factory.run(@strategy)
  end
end

class Factory
  def run(strategy_name)
    strategy = FactoryBot.strategy_by_name(strategy_name)
    # ...
    strategy.result
  end
end

FactoryBot also provides a method to register strategies at runtime. As long as the strategy responds to the right methods, the factory’s run method is happy. The following DuckStrategy would work just fine. If it walks like a duck…

class DuckStrategy
  def initialize(*args)
  end

  def association(*args)
  end

  def result(*args)
    :duck
  end
end

FactoryBot.register_strategy(:duck, DuckStrategy)

FactoryBot.duck(:user)
#=> :duck

Of course only a quack would write a strategy like this. FactoryBot gets us started with some reasonable default strategies, which I find are suitable for most cases.

Dynamically Defining Methods

Notice that we do not need to define FactoryBot.duck in order to use our custom strategy from the previous example. Calling FactoryBot.register_strategy takes care of defining it for us.

Since it is possible for us to define strategies with any name, FactoryBot needs a way to define methods dynamically at runtime. That is exactly what define_method is for.

module FactoryBot
  def self.register_strategy(strategy_name, strategy_class)
    strategies.register(strategy_name, strategy_class)

    define_method(strategy_name) do |name|
      FactoryRunner.new(name, strategy_name).run
    end
  end
end

The example above doesn’t quite work. define_method defines instance methods, but FactoryBot offers class methods for each strategy. There are a few ways around this (I’ll write about singleton classes at some point), but FactoryBot actually does not define the strategy methods directly on the FactoryBot module. Instead it defines them on the FactoryBot::Syntax::Methods module using module_exec and then extends FactoryBot with that module.

module FactoryBot
  def self.register_strategy(strategy_name, strategy_class)
    strategies.register(strategy_name, strategy_class)

    Syntax::Methods.module_exec do
      define_method(strategy_name) do |name|
        FactoryRunner.new(name, strategy_name).run
      end
    end
  end

  extend Syntax::Methods
end

Having the strategy methods packaged up in a module makes it possible to include just those methods into your test suite. I have always liked writing create instead of FactoryBot.create, even if it does make it less obvious where the method is coming from.

RSpec.configure do |config|
  config.include FactoryBot::Syntax::Methods
end

descrbe User do
  let!(:user) { create(:user) }
end

Registering a strategy also defines _pair and _list methods for that strategy. Check out FactoryBot::SyntaxMethodRegistrar for the full implementation.

Building your object

After looking up the appropriate strategy, Factory#run instantiates a FactoryBot::Evaluator. Each factory gets its own evaluator class (more on this in a bit) with methods available for each attribute on that factory. Overriding attributes amounts to overriding methods on the evaluator. If you have ever used FactoryBot callbacks you may have seen an evaluator instance as the second argument to the block.

FactoryBot.define do
  factory :user do
    username "'username' becomes a method on the evaluator"

    after(:build) do |user, evaluator|
      puts evaluator.username
    end
  end
end

FactoryBot.build(:user)
# => 'username' becomes a method on the evaluator

FactoryBot::Evaluator is a lot of fun. It uses nearly every technique discussed so far, including define_method, undef_method, method_missing, and instance_eval. I encourage you to spend some time with this class.

Factory#run passes the evaluator along to a FactoryBot:AttributeAssigner, along with the constant for your class (in my examples that would be the User constant). After initializing your class, the attribute assigner assigns each attribute by calling the relevant evaluator method, then passes the value to the relevant setter method on your class.

class AttributeAssigner
  def initialize(evaluator, build_class)
    @build_class = build_class
    @evaluator = evaluator
    @attribute_list = evaluator.class.attribute_list
  end

  def object
    instance = @build_class.new

    @attribute_list.each do |attribute|
      value = @evaluator.send(attribute)
      instance.public_send("#{attribute}=", value)
    end
  end
end

Once the attribute assigner is built and ready, each strategy behaves in its own way. Since the object is already built at this point, the build strategy only needs to trigger the after_build callback. The build_stubbed strategy, on the other hand, still has to do all the work of setting the id, disabling persistence methods, etc.

Anonymous Subclasses

FactoryBot allows you to create child factories that inherit attributes from their parent factory.

FactoryBot.define do
  factory :user do
    username "dodecadaniel"
    email "daniel@example.com"

    factory :composer do
      username "composerinteralia"
    end
  end
end

composer = FactoryBot.build(:user)
composer.username
#=> "dodecadaniel"
composer.email
#=> "daniel@example.com"

composer = FactoryBot.build(:composer)
composer.username
#=> "composerinteralia"
composer.email
#=> "daniel@example.com"

Can you think of an object-oriented approach that might allow for children to inherit behavior from their parent? This sounds a lot like Inheritance, and that is exactly what FactoryBot uses. I mentioned above that each factory gets its own evaluator class with instance methods for the attributes of that factory. Inheriting attributes is fairly simple; each child factory needs an evaluator class that inherits from the evaluator class of its parent.

But how does FactoryBot know how many evaluator classes to create, and what class names to use? Actually it doesn’t need to know any of that. FactoryBot creates anonymous subclasses dynamically at runtime using Class.new.

class Evaluator
end

child_evaluator_class = Class.new(Evaluator)
p child_evaluator
#=> #<Class:0x007fde3cda8848>
p child_evaluator_class.superclass
#=> Evaluator

grandchild_evaluator_class = Class.new(child_evaluator_class)
p grandchild_evaluator_class
#=> #<Class:0x007fde3cc28c70>
p grandchild_evaluator_class.superclass
#=> #<Class:0x007fde3cda8848>
p grandchild_evaluator_class.superclass.superclass
#=> Evaluator

# etc.

Null Object pattern

To create the evaluator class for a factory, FactoryBot does something like this:

class Factory
  def evaluator_class
    @evaluator_class ||= Class.new(parent.evaluator_class)
  end
end

But this code only works for child factories. In a top-level factory parent will be nil. We can fix this by checking for nil.

class Factory
  def evaluator_class
    @evaluator_class ||= Class.new(parent.nil? ? Evaluator : parent.evaluator_class)
  end
end

But now we will need to check for nil every time we need to use parent. There is a better way. I was planning to write more about this, but Josh Clayton already did.