How deep do you test?

I've been talking a lot recently about what the right level of testing is on my current project. Previously, on the Java projects I worked on, I was a full blown mocker, anything but the class I was testing would be likely be mocked and I would try to do pure "unit tests", testing only the code in one class at a time. Mockito was my friend and the world was good.

Now I'm working on a WebOS app so I'm back in the land of JavaScript and dynamic languages. I've never done any real JS testing before (or any dynamic language for that matter), so I'm trying to figure this all out for the first time.

The question of how deep to test keeps coming up because we're running into some some significant test complexity. We're not mocking many of our own classes, only the WebOS classes, so our tests are really halfway between integration tests and true unit tests.

This has its advantages, especially in a dynamic language where parameters are only checked at runtime. If we were doing pure unit tests and mocking the world away, we could be creating a fantasy where our class thinks it's running just fine, but isn't passing the right parameters or could possibly even be calling mocked functions that don't exist.

On the other hand, without mocking, our test complexity goes way up, especially since we're dealing with a lot of async behavior. Consider the following test:

it("should call the callback delegate", function() {
  var foo = new Foo();
  var myCallback = function() {};

  foo.doWork(myCallback);

  expect(myCallback).wasCalled();
});

Here we have a simple case where we are testing a doWork function on the Foo class and expecting that it calls a delegate at some point in the execution.

However, in reality, doWork might call some other class that relies on an XHR and asynchronously loading something from a datastore. Now our test looks something like:

it("should call the callback delegate", function() {
  var foo = new Foo();
  var myCallback = function() {};

  foo.doWork(myCallback);

  Tests.AJAX.Requests.fakeResponseFor(SOME_XHR_REQUEST); // Succeed pending XHR, return stub data
  Tests.Datastore.get.succeedAll(); // Succeed pending async datastore get requests
  Tests.Datastore.add.succeedAll(); // Succeed pending async datastore add requests

  expect(myCallback).wasCalled();
});

As you can see, the test complexity goes up very fast. Almost half the lines of code don't even have to do with the class we're testing!

Mocking could solve this problem much more cleanly by mocking whatever helper Foo uses to do the async logic:

it("should call the callback delegate", function() {
  var foo = new Foo();
  foo.helper = mock('HelperClass');
  var myCallback = function() {};

  foo.doWork(myCallback);

  expect(myCallback).wasCalled();
});

The argument against mocking in the tests above is that it could hide some poor design decisions, especially in our app when dealing with async behavior. For example, you might need a whenReady delegate to execute some code when async data is ready, and if you're mocking everything, that might not be apparent. However, the tradeoff is a lot of test code that detracts from what you are really trying to test.

The option of creating a stub class that automatically succeeds or fails all async behavior was brought up, but the legitimate concern was you could be hiding bigger problems. You could mistakenly try to code it to work synchronously and it would pass in the mocked world, but not in reality.

Also, being true to TDD, we could setup doWork to return a value instead of use a callback. This code would likely succeed with auto-succeed stub classes, but would fail completely when run in the real world:

it("should get a value from #doWork", function() {
  var foo = new Foo();
  var result = foo.doWork(); // doWork will fail in an asynchronous setup

  expect(result).equals('myValue');
});

I could rename this post to "How do you test async behavior?", but I think that my async example is just one of many things that mocks/stubs either help or hurt for the reasons above.

So, how deep do you test your code? Does the language affect your decision? Do you have any general rules of thumb for when to mock or when to do integration tests?