CodeMash 2013 Precompiler Day 2, Session Two


This afternoon’s session is Real-world JavaScript Testing with Justin Searls. This is supposed to be fairly hands on so I apologize if my notes aren’t very good since I will be focusing on doing. 🙂

We will be using a tool that TestDouble (the presenter’s company) built called lineman. It will run tests upon the file being saved. The test framework we are using is Jasmine, which has it’s roots in RSpec. QUnit is also a reasonable alternative.

Jasmine has spies, which are test doubles. Invoked by calling spyOn(<object>, <methodname>) – so, spyOn(loading, ‘show’).

Inject HTML into your tests with jasmine-fixture to avoid your testing being too reliant on the DOM.

You can use JSLint or JSHint to do static analysis of your JavaScript. CoffeeScript code is guaranteed to pass JSLint.

jasmine-given gives you a given-when-then syntax for your tests which many people consider more readable and intuitive.

And then we started pairing and writing some code and tests. Good stuff!


CodeMash 2013 Precompiler Day 2, Session One


Today’s first session is Making Test Automation with Web Applications Work with Jeremy Miller. He’s the guy behind StoryTeller, which is the automated testing tool we use at work.

What makes a good test? Any test that isn’t repeatable is useless.

Test with the finest grained mechanism that tells you something important. Don’t test business logic through the screen. It does make sense to test your UI “magic” (conventions, routing, presentation logic, etc.).

White box testing. The goal of testing is not to make sure your application is perfect. It is about reducing risk to an acceptable level. So, white box testing is very valid, since it tends to be cheaper.

Collapse your application into a single process/appdomain. Even if you have multiple services – bring them all together into one process. You solve your functionality problems and ignore the communication issues. You then can test the communication separately.

Test data sets suck. Define your test data in your test in a declarative fashion.

Diving straight into code and Storyteller. I kind of feel sorry for anyone who hasn’t seen Storyteller before, because there is no explanations about what they are doing with it. They are using RavenDB, a document database that can run in process. This helps them easily create new databases for each test without a lot of the relational overhead. They just put in the documents they need for the test.

There are lots of issues with test data. If you aren’t careful it will lead to brittle tests. This is why you should strive to tie it to the tests themselves instead of defining the traditional large sets of data. If you set up your data as part of your test, it becomes clearer to those trying to understand what the test is testing.

There exists an embedded SMTP server you can use to test email. They also use a tool called PortFinder that finds an open port you can use to listen on, which could be very useful for testing things that use TCP/IP.

If your tests are running all of the time (in continuous integration) then they are worse than worthless because they hold you back.

They use webdriver to run their UI layer and have defined what they call “Gestures” in Storyteller to perform standard UI operations.

Provide element handlers for each type of web element to facilitate making less brittle tests.

Separate your test expressions from you screen drivers. Break out the screen driving logic so that your actual tests are testing logic, It reduces brittleness.

Thread.Sleep is the easiest way to make your test suck, take too long, and unreliable. Build reasonable waiting tools that are more graceful.

It can be valuable to have a stub date/time class so that you can have better control over time. You may even want to consider taking over the system clock to give you friendly timestamp-sensitive tests.

At this point I had to bail. Too much magic, not enough meat.