Testing a New Page Loaded (or not) with Capybara

When writing feature specs, I usually test a new page has loaded using a custom matcher that checks the current URL path, <title>, and headline of the expected page, as shown in these example be_*_page matchers:

...

expect(page).to be_dashboard_page
click_link "Help"
expect(page).to be_help_page

...

matcher :be_dashboard_page do
  match_unless_raises do |page|
    expect(page).to have_current_path "/dashboard"
    expect(page).to have_title "Your Dashboard"
    expect(page).to have_css "h1", text: "Dashboard"
  end
end

matcher :be_help_page do
  match_unless_raises do |page|
    expect(page).to have_current_path "/help"
    expect(page).to have_title "Help"
    expect(page).to have_css "h1", text: "How can we help?"
  end
end

Very rarely, I encounter a situation where the be_*_page technique isn't the right tool for the job.

Perhaps I need to test that a new page has not loaded, say when exercising HTML5 input validations (HTML5 browser native validation errors cannot be accessed through the DOM). Or maybe I need to test the previous page has reloaded (and so the page content and URL path do not change). This can happen when exercising server-side validations and I want to be explicit about asserting the page reloaded, to make the test intent clearer.

In these validation-exercising moments, I would ordinarily test the displayed error messages.

But what do you do if the form error message is the same across page reloads (say due to consecutive form submits that are integral to the test)?

Because the browser driver works asynchronously, you can't assume the page would have reloaded instantly after clicking a submit button.

After clicking the submit button the old page content will still be present for a short time. Not accommodating for this delay in your test design will result in false positive passes where the previous content of the page is being asserted on before the new page has loaded.

Here's a (contrived) test that illustrates the problem when a form submission does not change the content of the page, but keeps showing the same error message:

visit "/subscribe"

expect(page).to be_subscribe_page

fill_in "Email", with: "invalid-email"

click_button "Submit"

# Note the error message text expectation after the next button
# click is the same as this one:
expect(page).to have_flash :alert, text: "You have form errors"

click_button "Submit"

# Sometimes the page will *not* have reloaded by the time this
# expectation runs, but the expectation still passes (incorrectly)
# as its testing the previous page content:
expect(page).to have_flash :alert, text: "You have form errors"

(Admittedly, there are ways to fix this in the UI, for example by having a dismissable flash that is closed before clicking the submit button, but pretend we need to write test coverage for existing behaviour).

So here's how this can be resolved, by using a custom replace_page block matcher:

expect(page).to have_flash :alert, text: "You have form errors"

-click_button "Submit"
+expect { click_button "Submit" }.to replace_page

expect(page).to have_flash :alert, text: "You have form errors"

And here's the replace_page matcher:

# Matcher checks if the original page has been unloaded
# and replaced with a new page:
#
# expect { click_link '' }.to replace_page
# expect { click_button 'Submit' }.not_to replace_page
matcher :replace_page do
  supports_block_expectations

  match do |interaction|
    # Generate a unique page id:
    page_id = "test-page-id-#{SecureRandom.hex}"

    execute_script <<-JS.strip_heredoc
      window.addEventListener("beforeunload", function(event) {
        // Precaution (may not be necessary) to ensure that during unload
        // the state is not set to undefined. Only want undefined state
        // to be present on the new page.
        window.__test_page_load_state = "before-unloading";
      });

      // Store the load state and the page id as part of
      // the current page:
      window.__test_page_load_state = "loaded";
      document.documentElement.classList.add("#{page_id}");
    JS

    # Execute the block passed to `expect`:
    interaction.call

    # Return true if the original page has been replaced:
    page_replaced = evaluate_script <<-JS.strip_heredoc
      window.__test_page_load_state === undefined && !document.documentElement.classList.contains("#{page_id}")
    JS
  end
end

Note the above matcher only works with JavaScript-capable Capybara drivers, so it won't work with the :rack_test driver. My guess is its possible to get something similar working with the :rack_test driver (get in touch if this is something you've done or would like help figuring out).

How to Get Production-like Error Responses in Your Rails Tests

In the Rails test environment, when your app responds with an error, it will show a detailed exception page (the one with the stacktrace and console) to help you debug it.

Showing the detailed exception page is perfect for unexpected errors - it helps you debug. However, for expected error responses that you want to test (say an API error or a 404 Not Found), for consistency, you'll usually want your app to respond with the same responses your app would give in production.

In these cases where you need production-like error responses to test against, you can temporarily set the Rails exception page configuration to match typical production values. Below is the code to help you do this in an RSpec suite.

# File: spec/support/error_responses.rb
# Ensure this file is require-d by rails_helper.rb or spec_helper.rb
module ErrorResponses

  def respond_without_detailed_exceptions
    env_config = Rails.application.env_config
    original_show_exceptions = env_config["action_dispatch.show_exceptions"]
    original_show_detailed_exceptions = env_config["action_dispatch.show_detailed_exceptions"]
    env_config["action_dispatch.show_exceptions"] = true
    env_config["action_dispatch.show_detailed_exceptions"] = false
    yield
  ensure
    env_config["action_dispatch.show_exceptions"] = original_show_exceptions
    env_config["action_dispatch.show_detailed_exceptions"] = original_show_detailed_exceptions
  end

end

RSpec.configure do |config|
  config.include ErrorResponses

  config.around(realistic_error_responses: true) do |example|
    respond_without_detailed_exceptions(&example)
  end
end

Source: https://github.com/eliotsykes/rails-testing-toolbox/blob/master/error_responses.rb

Add :realistic_error_responses (or realistic_error_responses: true) to your spec metadata to get the production-like error responses we've been talking about, for example:

describe 'Sessions API', :realistic_error_responses do
  ...
end

(Alternatively, you can call the respond_without_detailed_exceptions method directly, passing it a block inside of which you make the request, but I prefer the metadata technique, I find it more intention-revealing and it doesn't add as much distracting noise to the test.)

How to Cleanly Test Rails Rake Tasks with RSpec

When you need to test Rake tasks for a Rails app, its convenient to be able to write short, snappy specs like so:

# File: spec/tasks/send_invoices_spec.rb
require "rails_helper"

describe "rake billing:send_invoices", type: :task do

  it "preloads the Rails environment" do
    expect(task.prerequisites).to include "environment"
  end

  it "runs gracefully with no subscribers" do
    expect { task.execute }.not_to raise_error
  end

  it "logs to stdout" do
    expect { task.execute }.to output("Sending invoices\n").to_stdout
  end

  it "emails invoices" do
    subscriber = create(:subscriber)

    task.execute

    expect(subscriber).to have_received_invoice
  end

  it "checks in with Dead Mans Snitch" do
    dead_mans_snitch_request = stub_request(:get, "https://nosnch.in/c2354d53d2")

    task.execute

    expect(dead_mans_snitch_request).to have_been_requested
  end

  matcher :have_received_invoice do
    match_unless_raises do |subscriber|
      expect(last_email_sent).to be_delivered_to subscriber.email
      expect(last_email_sent).to have_subject 'Your invoice'
      ...
    end
  end

end

To be able to do this for your project, create the following file spec/support/tasks.rb (ensuring its require-d by rails_helper.rb or spec_helper.rb):

# File: spec/support/tasks.rb
require "rake"

# Task names should be used in the top-level describe, with an optional
# "rake "-prefix for better documentation. Both of these will work:
#
# 1) describe "foo:bar" do ... end
#
# 2) describe "rake foo:bar" do ... end
#
# Favor including "rake "-prefix as in the 2nd example above as it produces
# doc output that makes it clear a rake task is under test and how it is
# invoked.
module TaskExampleGroup
  extend ActiveSupport::Concern

  included do
    let(:task_name) { self.class.top_level_description.sub(/\Arake /, "") }
    let(:tasks) { Rake::Task }

    # Make the Rake task available as `task` in your examples:
    subject(:task) { tasks[task_name] }
  end
end


RSpec.configure do |config|

  # Tag Rake specs with `:task` metadata or put them in the spec/tasks dir
  config.define_derived_metadata(:file_path => %r{/spec/tasks/}) do |metadata|
    metadata[:type] = :task
  end

  config.include TaskExampleGroup, type: :task

  config.before(:suite) do
    Rails.application.load_tasks
  end
end

Source: https://github.com/eliotsykes/rails-testing-toolbox/blob/master/tasks.rb

Now write your task specs in the spec/tasks/ directory like the example send_invoices_spec.rb given earlier, in which you will be able to call

  • task.execute to run the task
  • task.prerequisites to access the tasks that task depends on
  • task.invoke to run the task and its prerequisite tasks

Do you have questions on this or related to Rails testing? Drop me a line at https://www.wetestrails.com/ask-a-question/.

Faster RSpec: Profiling FactoryGirl

One way to speed up a slow test suite is to optimize how FactoryGirl is used, but where do you focus your efforts?

Before tuning your factory usage, consider measuring which factories are adding the most time to your test runs. You may discover the time spent in factories is fine and decide to speed up another area of your suite.

Introduce FactoryGirlProfiler below to an RSpec project. It will only report when the --profile/-p flag is passed to the rspec command:

# File: spec/support/factory_girl_profiler.rb
# Ensure this file is required by spec_helper.rb or rails_helper.rb
class FactoryGirlProfiler

  attr_accessor :results

  def self.setup
    profiler = self.new

    RSpec.configure do |config|
      config.before(:suite) { profiler.subscribe }
      config.after(:suite) { profiler.dump }
    end
  end

  def initialize
    self.results = {}
  end

  def subscribe
    ActiveSupport::Notifications.subscribe("factory_girl.run_factory") do |name, start, finish, id, payload|
      factory, strategy = payload.values_at(:name, :strategy)

      factory_result = results[factory] ||= {}
      strategy_result = factory_result[strategy] ||= { duration_in_secs: 0.0, count: 0 }

      duration_in_secs = finish - start
      strategy_result[:duration_in_secs] += duration_in_secs
      strategy_result[:count] += 1
    end
  end

  def dump
    puts "\nFactoryGirl Profiles"
    total_in_secs = 0.0
    results.each do |factory_name, factory_profile|
      puts "\n  #{factory_name}"
      factory_profile.each do |strategy, profile|
        puts "    #{strategy} called #{profile[:count]} times took #{profile[:duration_in_secs].round(2)} seconds total"
        total_in_secs += profile[:duration_in_secs]
      end
    end
    puts "\n Total FactoryGirl time #{total_in_secs.round(2)} seconds"
  end

end

RSpec.configure do |config|
  config.add_setting :profile_factories, default: false
  config.profile_factories = config.profile_examples? || ARGV.include?('--profile') || ARGV.include?('-p')
  FactoryGirlProfiler.setup if config.profile_factories?
end

Source: https://github.com/eliotsykes/rails-testing-toolbox/blob/master/factory_girl_profiler.rb

Now run rspec --profile and review the output to find results formatted like below:

FactoryGirl Profiles

  invoice
    create called 4 times took 0.18 seconds total

  product
    build called 7 times took 0.07 seconds total
    create called 17 times took 0.22 seconds total

  user
    build called 12 times took 0.06 seconds total
    build_stubbed called 2 times took 0.01 seconds total
    create called 28 times took 1.51 seconds total

 Total FactoryGirl time 2.05 seconds

This will show the factories that are taking up the most time and help you decide where to focus your optimization work.

RSpec Refactoring Patterns: Extract Matcher

If forced to pick only one technique for refactoring code, I'd go with Extract Method. And if I were made to choose similarly for refactoring specs, it’d be Extract Method’s descendant Extract Matcher.

Motivations

You’ve got one or more expectations verifying state, but its unclear from the level of abstraction what the intention of these expectations is. By extracting well-named matchers, you can clarify the intention and so make the spec easier to understand.

Another motivation for Extract Matcher is to reduce duplication of repeated expectations or groups of expectations.

Example

A feature spec contains an expectation for checking the flash message after a user logs in:

expect(page).to have_css ".notice", text: "You're logged in"

For a developer familiar with Capybara, RSpec, and CSS, this expectation is fine, its reasonably clear what its testing. It reveals the intention to "assert the user successfully logged in". If there's any issue with it, it is that plainer language could demonstrate the intention without the noise of a CSS selector.

Here's the first attempt at muting the noise by extracting a have_flash matcher.

expect(page).to have_flash :notice, text: "You're logged in"

The have_flash matcher can be written in the same spec below any examples using the RSpec DSL for custom matchers.

feature "..." do
  scenario "..." do
    ...
    expect(page).to have_flash :notice, text: "You're logged in"
    ...
  end

  matcher :have_flash do |level, options|
    match_unless_raises do |page|
      expect(page).to have_css ".#{level}", options
    end
  end

end

have_flash is one level of abstraction higher than have_css as its language is one step closer to the language a user might use when thinking about how to tell if they are logged-in, and one step farther away from the CSS language the developer and web browser use to produce the page.

Can we get any closer to the language a user viewing our web page would use to make the test clearer?

Here's a couple alternatives for how the level of abstraction could be brought closer again to the user, and further away from the technical implementation of the page:

# Option 1:
expect(page).to have_notice "You are logged in!"

# Option 2:
expect(page).to have_logged_in_notice

In this case is the readability benefit of moving to higher and higher levels of abstraction worth the extra effort? The returns on readability are diminishing but still there. This is one for you to decide based on your own style and experience in the moment (or just toss a coin, there's not much to choose between them).

I want to show you one more example of extracting a matcher to demonstrate how you can use it to group multiple expectations into a single, more intention-revealing expectation:

expect(page).to have_title "Login"
expect(page).to have_current_path "/login"
expect(page).to have_css "h1", "Enter details to login"

The one intention of the above 3 expectations is to assert that the user is on the login page. Lets extract this intention into a be_login_page matcher:

feature "..." do
  scenario "..." do
    ...
    expect(page).to be_login_page
    ...
  end

  matcher :be_login_page
    match_unless_raises do |page|
      expect(page).to have_title "Login"
      expect(page).to have_current_path "/login"
      expect(page).to have_css "h1", "Enter details to login"
    end
  end
end

Reusing Matchers

In the above examples we've extracted matchers in to the same spec that they are used in.

What if we wanted to reuse those matchers throughout our specs?

To make a matcher reusable, move it to its own file in the spec/support/matchers/ directory and ensure its require-d by spec/rails_helper.rb (or spec/spec_helper.rb) and config.include it.

To make the be_login_page matcher available to all feature specs:

# File: spec/support/matchers/be_login_page.rb
module BeLoginPage
  extend RSpec::Matchers::DSL

  matcher :be_login_page do
    match_unless_raises do |page|
      expect(page).to have_title "Login"
      expect(page).to have_current_path "/login"
      expect(page).to have_css "h1", "Enter details to login"
    end
  end
end

RSpec.configure do |config|
  # Makes be_login_page matcher available to only feature specs.
  # To make it available to all specs, drop the `type` option.
  config.include BeLoginPage, type: :feature
end

Ask here if you've got questions on this technique or questions on testing Rails apps and I'll try to help you out.

Testing CodeMirror Editor with Capybara & RSpec

CodeMirror is a JavaScript text editor that I recently needed to test, but it was a little tricky getting that test automated with Capybara.

CodeMirror converts a given textarea into the editor field and hides the textarea which causes the test to have a tough time entering text into it.

See the fill_in_editor_field(text) method below for how you can enter text into CodeMirror in your feature specs, and the have_editor_display(options) method for how you can assert the editor is displaying the expected text:

# Important, `:js` metadata
# required for Capybara to run this
# using a driver for a
# JavaScript-capable browser.
feature '...', :js do

  scenario '...' do
    ...
    fill_in_editor_field "Hello World"
    expect(page).to have_editor_display text: "Hello World"
    ...
  end

  private

  def fill_in_editor_field(text)
    within ".CodeMirror" do
      # Click makes CodeMirror element active:
      current_scope.click

      # Find the hidden textarea:
      field = current_scope.find("textarea", visible: false)

      # Mimic user typing the text:
      field.send_keys text
    end
  end

  def have_editor_display(options)
    editor_display_locator = ".CodeMirror-code"
    have_css(editor_display_locator, options)
  end
end

This solution has been tested in Chrome 54 on Mac OS X.

Beat Spec File Name Blunders

Have you ever written a spec and forgotten to give its file name the required _spec.rb suffix? That spec will rarely (if ever) be run and you may never find out…

Perhaps surprisingly, a default-configured, bare rspec command only runs spec files ending with _spec.rb inside the spec/ directory:

For example, a spec file at spec/models/foo.rb will not be run by a typical, plain rspec command.

What can make this slip-up hard to detect is when you give the full file path as an argument to rspec (e.g. rspec spec/models/foo.rb) then the spec will be run and you won’t get a heads-up about the file name.

# Only runs specs withs paths like spec/**/*_spec.rb
# spec/models/foo.rb would _not_ be run:
rspec

# Explicit path argument will run foo.rb:
rspec spec/models/foo.rb

The good news is you can protect yourself against these spec file name oversights by adding an enforcer to your projects:

# Suggested steps:
#   1. Create file: `spec/support/spec_file_name_enforcer.rb`
#   2. Include the below config in spec_file_name_enforcer.rb
#   3. Ensure test env requires `spec/support/*` files
RSpec.configure do |config|
  config.before(:suite) do

    files_without_spec_suffix = Dir.glob('spec/**/*').select do |filename|
      # Customize regular expression patterns below as needed. Common
      # non-spec file paths under spec/ will not raise an error.
      File.file?(filename) && 
        !filename.match(/_spec\.rb\z/) &&
        !filename.match(%r{\Aspec/(support|factories|mailers/previews)/}) &&
        !filename.match(%r{\Aspec/(spec_helper\.rb|rails_helper\.rb|examples\.txt)\z})
    end

    if files_without_spec_suffix.any?
      raise 'Spec files need _spec.rb suffix: ' +
        files_without_spec_suffix.join(', ')
    end
  end
end

Running rspec on a project using the above configuration will raise an error any time a suspect file in spec/ is detected without the _spec.rb suffix.

You can check the spec file names are being enforced in your own project by creating an empty file with touch spec/models/foo.rb. Run rspec and check it reports a “Spec files need _spec.rb suffix: spec/models/foo.rb” error.

Testing EpicEditor with Capybara & RSpec

EpicEditor is a JavaScript-dependent Markdown editor that uses nested iframes and can be embedded in Rails forms to replace textarea fields.

Digging into these nested iframes to input text within a test can be a little tricky. See the fill_in_editor_field(text) method below for how you can enter text into EpicEditor in your feature specs, using nested calls to Capybara’s within_frame:

# Important, `js: true` metadata
# required for Capybara to run this
# using a driver for a
# JavaScript-capable browser.
feature '...', js: true do

  scenario '...' do
    ...
    fill_in_editor_field '*Some* markdown.'
    ...
  end

  private

  def fill_in_editor_field(text)
    # Use CSS attribute prefix selector
    # (^=) to find iframe with id
    # beginning with 'epiceditor-' to
    # match first parent iframe in page
    parent_iframe = find('iframe[id^=epiceditor-]')

    within_frame(parent_iframe) do
      child_iframe_id = 'epiceditor-editor-frame'

      within_frame(child_iframe_id) do
        find('body[contenteditable]').set(text)
      end
    end

  end

end

n.b. You may need to disable Turbolinks on inbound links to forms using EpicEditor so the editor loads consistently.

Testing Ace Editor with Capybara & RSpec

I’ve been working with a developer who is putting together a Rails app to give other developers easy access to an in-browser Ruby development environment. You can check it out here.

One of the challenges we faced was how to test the Ace editor that users will enter their Ruby code into.

Here’s how we solved it:

# Important, `js: true` metadata
# required for Capybara to run this
# using a driver for a
# JavaScript-capable browser.
feature '...', js: true do

  scenario '...' do
    ...
    fill_in_editor_field "puts 'Hello World'"
    expect(page).to have_editor_display text: "puts 'Hello World'"
    ...
  end

  private

  def fill_in_editor_field(text)
    find_ace_editor_field.set text
  end

  # Ace uses textarea.ace_text-input as
  # its input stream.
  def find_ace_editor_field
    input_field_locator = ".ace_text-input"
    is_input_field_visible = false
    find(input_field_locator, visible: is_input_field_visible)
  end

  # Ace uses div.ace_content as its
  # output stream to display the code
  # entered in the textarea.
  def have_editor_display(options)
    editor_display_locator = ".ace_content"
    have_css(editor_display_locator, options)
  end
end

This solution has been tested in Firefox 40.0.3 on Mac OS X.