Que, JSON, ActiveSupport, and “stack level too deep”

Working on one of our Rails apps, ensuring all of the transactional email is asynchronous to user interactions, I came across this:

Error

Since we still haven’t upgraded from Rails 4.1 to 4.2, I’m using Que without the ActiveJob API. All the specs were passing but the error just showed up when doing manual tests.

Que serializes the job arguments with JSON, so we have to stick with simple objects. I thought I knew that. But mistakes happen. And Que by default runs in :sync mode during tests, with no need to serialize the job’s arguments. This means that a job like this, although unprepared for :async execution, will make tests pass:

class MyJob < Que::Job
  def run(model, record_id)
    record = model.find(record_id)
    # Do some work
  end
end

order = Order.where(...)  # Order being an ActiveRecord::Base model
MyJob.enqueue(order.class, order.id)

Try that on an environment with asynchronous execution and you may get the “stack level too deep” error. Why? Because I wasn’t serializing things as I have thought.

What I really should have written:

class MyJob < Que::Job
  # model_name being a string.
  def run(model_name, record_id)
    model = model_name.constantize
    record = model.find(record_id)
    # Do some work!
  end
end

order = Order.where(...)
MyJob.enqueue(order.class.name, order.id)

Que delegates the serialization to ActiveSupport, something like:

ActiveSupport::JSON.encode(Order)

Part of the process is to JSONify the data, a process that will end up calling Order.as_json.

# From rails/activesupport/lib/active_support/json/encoding.rb
def jsonify(value)
  case value
  when String
    EscapedString.new(value)
  when Numeric, NilClass, TrueClass, FalseClass
    value
  when Hash
    Hash[value.map { |k, v| [jsonify(k), jsonify(v)] }]
  when Array
    value.map { |v| jsonify(v) }
  else
    jsonify value.as_json
  end
end

value.as_json will default to traverse Order.instance_values to create a Hash. Order.instance_values is an ActiveSupport core extension method that maps the object’s instance variable names to their values. The traversal looks like:

subset = Order.instance_values
Hash[subset.map { |k, v| [k.to_s, v.as_json] }]

This will keep calling as_json on Order.instance_values. And then, on each of the instance values that are not basic data types (e.g. ActiveRecord::ConnectionAdapters and Arel::Table) the process will get repeated over and over.

For example, Arel::Table instances reference engine instances that have references to the same table that references the same engine that has references to the same table that reference the same engine… BOOM!

Using pry, to compare it against something that works, we can see the difference between the code behind Order#as_json and Order.as_json. Look at the owner of each method: ActiveModel::Serializers::JSON vs. Object. You can also see how Order#as_json instead of using Order.instanve_values, relies on ActiveRecord::Serialization#serializable_hash, a method that returns a record’s attributes as a hash with the explicit purpose of serialization.

[7] pry(main)> $ Order.new.as_json

From: /Users/iconpin/.rvm/gems/ruby-2.1.1/gems/activemodel-4.1.1/lib/active_model/serializers/json.rb @ line 88:
Owner: ActiveModel::Serializers::JSON
Visibility: public
Number of lines: 14

def as_json(options = nil)
  root = if options && options.key?(:root)
    options[:root]
  else
    include_root_in_json
  end

  if root
    root = self.class.model_name.element if root == true
    { root => serializable_hash(options) }
  else
    serializable_hash(options)
  end
end

[8] pry(main)> $ Order.as_json

From: /Users/iconpin/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.1/lib/active_support/core_ext/object/json.rb @ line 46:
Owner: Object
Visibility: public
Number of lines: 7

def as_json(options = nil) #:nodoc:
  if respond_to?(:to_hash)
    to_hash.as_json(options)
  else
    instance_values.as_json(options)
  end
end
[9] pry(main)>

So, Order.as_json comes from the core extensions to Object made by ActiveSupport while Order#as_json is way more specific, more intentional: it is defined in ActiveModel. It seems like an side effect that we have Order.as_json available.

A minimal way to reproduce the problem:

require "active_support/core_ext"

class Thing
  @a_thing = Thing
end

Thing.as_json  # BOOM!

So, the default serialization strategy used by ActiveSupport is to recursively traverse the instance values creating a JSON-friendly Hash. Does it always makes sense? Who is being silly–if anyone? ActiveSupport::JSON for traversing variables? Rails for having these kind of circular relationships at runtime? Me for calling .as_json on Order? Is this just a little price to pay for the good parts of Rails and ActiveSupport?

This reminds me a bit of how Gary Bernhardt’s presented his (punny) Base gem:

this is Ruby! We just call methods and things happen! We don’t have to worry about it! Isn’t this great?!

Share this:

Leave a Reply

Your email address will not be published. Required fields are marked *