Want to show your appreciation?
Please a cup of tea.
Showing posts with label Unit Test. Show all posts
Showing posts with label Unit Test. Show all posts

Wednesday, August 03, 2011

Use Linq Expression for Unit Test Assertion

I’m going to use NUnit to illustrate this technique but the same can possibly be applied to any other unit test frameworks.

Although NUnit has a large set of Constraints for you to use, and most of them are intuitive for simple uses cases, but there is still a learning curve for new comers. It can actually become quite complex and a bit difficult to choose which constraint to use when.  The outcome can also be obscure sometimes. Below is an example from a nunit-discuss topic.

Assert.That(array, Contains.Item.With.Property("x").LessThan(100).And.GreaterThan(50));

has a different meaning than this one...

Assert.That(array, Contains.Item.Property("x").LessThan(100).And.Greater.Than(50));

On the same nunit-discuss topic, David Schmitt mentioned the use of Expression. Yes, there are already libraries out there to do assertions using Expression, and they support various unit testing framework. For example, PowerAssert.Net, ExpressionToCode, Satisfyr and etc.

But if you are not ready to jump on any of those (yes, they are quite young and immature), then you can also roll you own implementation very easily. Below is an extension method based on NUnit:

    public static class NUnitExtension
    {
        public static void AssertThat<T>(this T target, Expression<Func<T, bool>> criteria)
        {
            Assert.That(target, new ExpressionConstraint<T>(criteria));
        }
 
        private class ExpressionConstraint<T> : Constraint
        {
            private readonly Expression<Func<T, bool>> _expression;
 
            public ExpressionConstraint(Expression<Func<T, bool>> expression)
            {
                _expression = expression;
            }
 
            public override bool Matches(object actual)
            {
                this.actual = actual;
                return (actual is T) && _expression.Compile()((T)actual);
            }
 
            public override void WriteDescriptionTo(MessageWriter writer)
            {
                writer.WriteValue(_expression);
            }
        }
    }

Then the earlier example can be  rewritten to something like below.

array.AssertThat(a => a.Any(i => i != null && 5 < i.x && i.x < 10));
array.AssertThat(a => a.All(i => i != null && i.x < 10) && a.Length > 5);

When it fails, you still get meaningful information:

   1:  Test 'MyTest.Test' failed: 
   2:    Expected: <a => (a.All(i => ((i != null) && (i.x < 10))) && (ArrayLength(a) > 5))>
   3:    But was:  < <X(7)>, null, null, null, null, null >
   4:      MyTest.cs(68,0): at NUnitExtension.AssertThat[T](T target, Expression`1 criteria)
   5:      MyTest.cs(60,0): at MyTest.Test()

Now you may ask if this can be part of NUnit itself. Good question! Since Expression is only supported by .Net 3.5 & 4.0 but NUnit needs to support old .Net frameworks, it’s a decision NUnit team had to make if it worth the effort to release a .Net 3.5 specific dll file that contains this one extra feature. But anyway, it is very easy to copy and paste above extension method to your own test project and you are good to go.

Update (8/6/2011): While NUnit already support lamba via PredicateConstraint, the major difference is it uses a delegate instead of Expression. So when test fails the expect message loses the detail. For example, when below assert fails,

Assert.That(new []{new X{x=1}}, Has.Some.Matches<X>(i => 5 < i.x && i.x < 10));

unlike the Expression example that has expression printed as expected, you get an error message with no detail of the lamba logic.

   1:  Test 'MyTest.Test' failed: 
   2:    Expected: some item value matching lambda expression
   3:    But was:  < <X(1)> >
   4:      AccessorTest.cs(27,0): at SharpCut.UnitTests.AccessorTest`1.Test()

Wednesday, November 04, 2009

Unit Test Value Object – Reusable Test Fixture

To Test Or Not Test

Shall I unit test value object? Here are some arguments that I found when googling for an answer to the question.

  1. Value object are so simple that you don’t need to test them.
  2. Test value object is tedious, basically copy and pasting and you can make the same mistake that you did in the value object itself.
  3. If you test every other part of the application, you would have tested your value object.

And there are also opposite opinions.

  1. Value objects are not as simple as you thought. How about initial property value, equals, hashcode and cloneable? (Yes, that reminds me that I made quite some mistakes in value objects)
  2. Unit test is tedious in nature. Tedious is not a reason for not doing it. But it only means that we need to find a better way.
  3. This is about unit testing, not integration testing. Value object is a unit that should be tested alone.

In summary, value objects are relatively simple so that the cost of manually writing test cases for them are too high to justify the benefit but not testing them leaving potential bugs.

The Reusable Test Fixture

Compromising the quality is a no in my book so the solution is to develop reusable tests to replace the repetitive and tedious work. This is what programmers are good at after all!

Let’s take a look at a very simple value object.

public class Foo
{
    public int Id { get; set; }
 
    public string StringProperty { get; set; }
 
    public long LongProperty { get; set; }
}

Writing a unit test for this class manually is plain boring. I should be able just do it in one line with a reusable test fixture.

[TestFixture] public void FooTest : SomeMagicReusableTestFixture<Foo> {}

This magic reusable test fixture is called NewableValueObjectTestFixture. It sets each property with a few (three minimal) values and read it back to make sure it gets the same value previously set. Of course, it does much more then just that…

Default Property Value

The above example is probably too simple. What if my property has default value? say the Id property is initialized to –1. No problem! The reusable test fixture also verifies the initial value of properties too. It expects all the properties are initialized to the default value of the type, or a value specified by DefaultValueAttribute.

The code below will fail the test because the the default value for int type is 0, not –1.

    private int _id = -1;
 
    public int Id
    {
        get { return _id; }
        set { _id = value; }
    }

If you intended to initialized to –1, you should tell this to the test fixture. The information is also useful to other .Net tools.

    private int _id = -1;
 
    [DefaultValue(-1)]
    public int Id
    {
        get { return _id; }
        set { _id = value; }
    }

The code above passes the test happily.

Equals and GetHashCode

NewableValueObjectTestFixture and its brother ValueObjectTestFixture test the Equals and GetHashCode methods too. They basically ensure that:

  1. The value object does not equals to null.
  2. The value object does not equals an arbitrary object (new object()).
  3. The value object equals to itself.
  4. The value objects are not equal if any one of their properties are not equal.
  5. When two value objects have same property values and Equals method returns true. The GetHashCode methods must return same value.

The implementation in the Object class complies to the above rules so the your value object passes the tests by default. But if you decide to override any of them, the test helps to ensure you have implemented Equals and GetHashCode properly, otherwise the test will fail. Below is generated by Resharper and it passes the test.

    public override bool Equals(object obj)
    {
        if (ReferenceEquals(null, obj)) return false;
        if (ReferenceEquals(this, obj)) return true;
        if (obj.GetType() != typeof(Foo)) return false;
        return Equals((Foo)obj);
    }
 
    public bool Equals(Foo other)
    {
        if (ReferenceEquals(null, other)) return false;
        if (ReferenceEquals(this, other)) return true;
        return other._id == _id &&
            Equals(other.StringProperty, StringProperty) &&
            other.LongProperty == LongProperty;
    }
 
    public override int GetHashCode()
    {
        unchecked
        {
            int result = _id;
            result = (result * 397) ^ (StringProperty != null ? StringProperty.GetHashCode() : 0);
            result = (result * 397) ^ LongProperty.GetHashCode();
            return result;
        }
    }

A few days later if I add another property to the Foo class and forget to added it to the Equals method, the test fixture will effectively catch it and fails the test. This is exactly what I wanted, how many times did I forget this?

Cloneable

If my value object implements ICloneable, the test fixture will help to check that as well. It clones the object and test all readable properties to make sure the cloned one has the same property values. Below is my typical clone implementation and it works well with the test fixture.

public class Foo : ICloneable
{
    public int Id { get; set; }
 
    public string StringProperty { get; set; }
 
    public long LongProperty { get; set; }
 
    object ICloneable.Clone()
    {
        return Clone();
    }
 
    public Foo Clone()
    {
        return (Foo)MemberwiseClone();
    }
}

Type of Property

Since the test fixture needs to generate test data, it had limited support for the type of the properties. For example, a property of interface type would be difficult for ValueObjectTestFixture to create test data of it.

Although, out of the box, ValueObjectTestFixture only support properties of c# build-in data types plus DateTime and TimeSpan, but it provides two extension mechanisms. You can either use a mock framework to generate mock objects for interface type and class type, or override the TestData method to provide your own test data for types that are not supported by the default implementation. Of course, nothing prevents you from making mixed use of both mechanisms.

Future enhancement may have ValueObjectTestFixture support enum, concrete class and/or maybe struct.

Using Mock Framework to Generate Test Data

ValueObjectTestFixture accepts an IMockTestDataProvider instance. I have included an implementation using RhinoMocks. But you can easily do the same using any other mock frameworks. RhinoMockTestDataProvider mocks any interfaces, or classes that are not sealed and has a default constructor. We may be able to remove the requirement of a default constructor in future but for now that is it. :)

Take a look at below value object:

public class ValueObject
{
    public IList<string> StringIList { get; set; }
 
    public List<TimeSpan> TimeSpanList { get; set; }
 
    public ICollection<int> IntICollection { get; set; }
 
    public IDictionary<string, object> StringObjectIDictrionary { get; set; }
 
    public Dictionary<int, DateTime> IntDataTimeDictionary { get; set; }
 
    public Component Component { get; set; }
 
    // other properties, equals, hash code, clone and etc.
}

To test this, the test class is as simple as below:

    [TestFixture]
    public class MockProviderTest : NewableValueObjectTestFixture<ValueObject>
    {
        public MockProviderTest()
        {
            MockProvider = new RhinoMockTestDataProvider();
        }
    }

The complete example can be found in Google code: MockProviderTest.cs

Provide Your Own Test Data

If you have property type that is neither supported out-of-box, nor by a mock framework. You can always provide it yourself by overriding the method TestData. Below is an example from CustomTest.cs:

        protected override IEnumerable TestData(PropertyInfo property)
        {
            // ValueObjectTestFixture doesn't support enum at this moment
            // so we have to provide our own test data.
            if (property.PropertyType == typeof(Operation))
            {
                return new[]
                           {
                               Operation.Subtraction, 
                               new Operation(), 
                               Operation.Addition, 
                               Operation.Muiplication, 
                               Operation.Division
                           };
            }
            return base.TestData(property);
        }

Extensibility

While ValueObjectTestFixture makes simple things simpler, it ensures complex things possible too. Here are some additional features that will come handy when testing not so simple value objects.

Value Object without Default Constructor

What if the value object doesn’t have a default constructor? No problem, have the test case inherit from ValueObjectTestFixture. The only difference between ValueObjectTestFixture and NewableValueObjectTestFixture is that the former forces you to override an abstract method: NewValueObject, which you are required to return a new instance of the value object under test. Below example is from CustomTest.cs:

        protected override CustomObject NewValueObject()
        {
            return new CustomObject("TestObject");
        }

Test Special Properties Myself

There are times that value object has non-simple properties. For example, when a read only property is initialized by the constructor:

public class CustomObject
{
    private readonly string _name;
    public CustomObject(string name)
    {
        _name = name;
    }
 
    public string Name { get { return _name; } }
}

Or, when a property is calculated from others:

public int Result
{
    get
    {
        switch (Operation)
        {
            case Operation.Addition:
                return LeftOperand + RightOperand;
            case Operation.Subtraction:
                return LeftOperand - RightOperand;
            case Operation.Muiplication:
                return LeftOperand * RightOperand;
            case Operation.Division:
                return LeftOperand / RightOperand;
            default:
                throw new InvalidOperationException("Unsupported operation " + Operation);
        }
    }
}

ValueObjectTestFixture provides two ways to exclude those properties from being tested by it. But then you need to write the test cases yourself.

One way is to use one of the ExcludeProperties methods. Typically you call it in the constructor of your test fixture. Below is from CustomText.cs:

    [TestFixture]
    public class CustomTest : ValueObjectTestFixture<CustomTest.CustomObject>
    {
        public CustomTest()
        {
            // Exclude the Name property from being tested by ValueObjectTestFixture.
            ExcludeProperties("Name");
        }
        // ... ...
    }

This completely excludes the property from all the tests administered by ValueObjectTestFixture.

Another less aggressive approach is to override one of the XyzCandidates methods. They are listed below to provide a custom list of properties for the corresponding tests:

Example from CustomText.cs:

        public override IEnumerable<PropertyInfo> EqualsTestCandidates()
        {
            // Exclude the Tag property from equality test.
            return from p in base.EqualsTestCandidates() where p.Name != "Tag" select p;
        }

You opinions is important. Let me know what you think :)

Thursday, September 10, 2009

How NUnit Handles Test Fixture as Inner Class

The Problem

Will NUnit execute the test fixture defined as inner class when I test run the enclosing test fixture? The answer is, it depends (as of version 2.5.2). In most of cases it won't, but if the enclosing test fixture is a generic type, it will.

Here is the test case to reproduce this:

namespace Demo
{
    [TestFixture] public class A { //plain normal
        [Test] public void Bar() { }
        [TestFixture] public class Inner {
            [Test] public void Foo() {}
        }
    }
 
    [TestFixture(typeof(int))] public class B<T> { // generic
        [Test] public void Bar() { }
        [TestFixture(typeof(int))] public class Inner {
            [Test] public void Foo() {}
        }
    }
 
    [TestFixture("dummy")] public class C { // parameterized non-generic
        public C(string dummy){}
        [Test] public void Bar() { }
        [TestFixture(typeof(int))] public class Inner {
            [Test] public void Foo() { }
        }
    }
}

And the result:

C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.A" /labels
Selected test: Demo.A
***** Demo.A.Bar
 
Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0262504 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
 
C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.B<T>" /labels
Selected test: Demo.B<T>
***** Demo.B`1+Inner[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Foo
***** Demo.B`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Bar
 
Tests run: 2, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0337505 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
 
C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.C" /labels
Selected test: Demo.C
***** Demo.C.Bar
 
Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0350005 seconds
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

My Thoughts

Now which behavior do I prefer? I would like to include the inner test fixture classes when running the enclosing one. Why? that's the only reason so far I need to create inner test fixture class. And here is the use case:

  1. I have MyCollection<T> : ICollection<T>, ICollection that is the subject under test.
  2. I have common reusable test fixture CollectionTestFixture<T> and CollectionTestFixture contain functional test cases for types that implement ICollection<T> and/or ICollection.
Test implementation A, Create three(3) separate test fixtures
  • MyCollectionGenericTests<T> : CollectionTestFixture<T>
  • MyCollectionNonGenericTests : CollectionTestFixuture
  • MyCollectionTests<T> // specific test cases

Pros:

  • Simple
  • Works

Cons:

  • Must run 3 test fixture to test one class
  • Difficult (impossible if using TD.Net) to run overall code coverage for MyCollection<T> class
  • Test result are not grouped together when there is MyCollectionSomeElseTest.
Test implementation B, use dedicated namespace
  • MyCollection.GenericTests<T> : CollectionTestFixture<T>
  • MyCollection.NonGenericTests : CollectionTestFixuture
  • MyCollection.Tests<T> // specific test cases

Pros:

  • Works
  • Can easily run all tests for MyCollection and get overall coverage

Cons:

  • Creating one folder (matching namespace) for one class to be tested result in very busy project layout and hard to navigate when there are a lot of such scenarios.
  • There will be a lot of repeated type names, e.g. GenericTests<T>. Otherwise MyCollection.MyCollectionGenericTests<T> makes the label reads much longer which is a bad thing.
Test implementation C, use inner classes
  • MyCollectionTests<T> // specific test cases
  • MyCollectionTests.AsGeneric<T> : CollectionTestFixture<T>
  • MyCollectionTests.AsNonGeneric : CollectionTestFixuture

Pros:

  • Can easily run all tests for MyCollection and get overall coverage
  • No additional directories to create and navigate.
  • Common methods can be easily reused by inner classes.
  • Labels read naturally

Cons:

  • Doesn’t work for non-generic enclosing test fixture, but hoping it will work…
The Winner

It is not difficult to conclude that the best implementation is C if NUnit can support it consistently. And IMHO, inner classes are members of enclosing class from the language (logical) perspective although CLR implemented (physical) the other way. I wish test framework implements the logical meaning because that’s what most developers familiar with.

Test Fixture Aggregation

I recall there was a talk about the ability to quickly create a new test fixture by aggregating test cases from multiple reusable test fixtures. The inner class approach is my proposal to the problem. Together with the ability to specify including/excluding of test cases in the inherited fixture may provide a complete solution.

My Wishes

  1. Change NUnit to always consider test cases of inner class as an integral part of the enclosing test fixture. And execute then when the enclosing test fixture is specified to execute.
  2. NUnit-GUI should organize the inner class under the enclosing class in its tree structure.
  3. This one is minor and not important to me but it would be nice if NUnit can hide the implementation detail of CLR and translate to the dot notation for inner class like what the language does.

Thursday, July 09, 2009

Some Utilities Extend Rhino.Mocks

I have recently blogged a few posts about how to let Rhino Mocks do more for us. They are listed below:

All those features require reader either change the source code of Rhino Mocks or download a special version of Rhino Mocks that I built. Those changes are quite invasive.

By carefully looking at the code, I found that there are some features can be extract into an extension library without touching the Rhino Mocks itself. Actually, only the ordered expectation requires source code change. Hence, I decided to build a extension library so that everybody can use. You can either drop the source into your test project or download the Rhino.Mocks.Extension.dll and add the reference to it.

Here is the current feature list:

  1. Create multi mock for AAA: Mockery.GenerateMultiMock
  2. Create partial mock for AAA: Mockery.GeneratePartialMock, Mocker.GeneratePartialMultiMock
  3. Ability to use test framework's assert: Assert.IsTrue(mock.ActivityOf(x=>x.Foo()));
  4. Use operator to check "or" condition: mock.ActivityOf(x=>x.Foo()) | mock2.ActivityOf(x=>x.Bar())
  5. Support assert for one and exactly one is called: Mockery.ExactOneOf(m.ActivityOf(...), ...).AssertOccured;

Please see my earlier posts for the detail of feature 3-5.

Wednesday, July 08, 2009

Introduce A Powerful AAA Syntax for Rhino.Mocks

I was again (last one was about ordered expectations for AAA) inspired by a question posted on the Rhino Mocks support email group, expect a call to any one of overloaded method. How do we do that? I have proposed a solution and that was pretty much what I did before. When we talk about TDD, we write the test before implementation. And sometimes, designer writes the test cases for somebody else to complete the implementation.

In many cases, all we care is that one of the send methods of the notification service is called. Which exact send method to call is all up to the implementation.

The Syntax

There got to be a better why to do this! The same may not necessary only apply to overloaded methods. For any methods, I should be able to just write something like this:

    ma.AssertWasCalled(a=>a.Foo()) or mb.AssertWasCalled(b=>b.Bar());

The experience gained from my work on the ordered expectation tells me that this is very possible. All I need to figure out is the syntax for this. May be

    ma.AssertWasCalled(a=>a.Foo()).Or(mb.AssertWasCalled(b=>b.Bar()));

or even better to use operator overload:

    ma.AssertWasCalled(a=>a.Foo()) | mb.AssertWasCalled(b=>b.Bar());

Well, the problem is that AssertWasCalled fails right away when a.Foo() is not called. Why do we have to fail it? Why cannot we leave it to the test framework to do the assert? But anyway, in order to not break the existing contract of AssertWasCalled, we'll have to use something different. How about this?

    Assert.IsTrue(ma.ActivityOf(a=>a.Foo()) || mb.ActivityOf(b=>b.Bar()));

Have more methods, not problem!

    Assert.IsTrue(foo.ActivityOf(a=>a.Foo(Arg<Something>.Is.Anything)) || 
        foo.ActivityOf(a=>a.Foo(Arg<Something>.Is.Anything, Arg<Another>.Is.NotNull)) || 
        foo.ActivityOf(a=>a.Foo(Arg<Something>.Is.Anything, Arg<Another>.Is.NotNull, Arg<int>Is.Anything)));

This syntax give us a lot of flexibility. I can use operator overload for ordered expectation as well.

    Assert.IsTrue(ma.ActivityOf(a=>a.Foo()) < mb.ActivityOf(b=>b.Bar()));

And there are unlimited possibilities, some examples below:

    // chained ordering
(m.ActivityOf(...) > m.ActivityOf(...) > m.ActivityOf(...)).AssertOccured;
    // mixed or ordering
(m.ActivityOf(...) | m.ActivityOf(...)) < m.ActivityOf(...)).AssertOccured
    var sent = (m.ActivityOf(a=>a.Sent(x)) | m.ActivityOf(a=>a.Sent(x, y)));
    Assert.IsTrue(sent); // assert sent is called
    Assert.IsTrue(sent < m.ActivityOf(a=>a.Close())); // assert sent called before close

Implementation

This is exciting, isn't it? How difficult to implement this? I give it a try by building on top of the changes that I made for the ordered expectation in the previous post. It turn out to be quite easy. Although the code below is no where close to perfect, and indeed it is quick and dirty to some extend, but it served well as proof of concept to introduce this technique.

Here we go the code. The Activities class is the core. It provides implementation for all the operator overload.

    public class Activities : IComparable<Activities>
    {
        private static readonly CallRecord[] _emptyRecords = new CallRecord[0];
        private readonly IList<CallRecord> _callRecords = _emptyRecords;
        private readonly ExpectationViolationException _exception;

        public Activities(IList<CallRecord> callRecords)
        {
            if (callRecords == null || callRecords.Count == 0)
                throw new ArgumentException(
                    "Must not be null or empty.", "callRecords");
            _callRecords = callRecords;
        }

        public Activities(ExpectationViolationException exception)
        {
            if(exception==null) throw new ArgumentNullException("exception");
            _exception = exception;
        }

        public bool Occured
        {
            get { return _callRecords.Count > 0; }
        }

        public Activities OccuredBefore(Activities other)
        {
            if (!Occured) return this;
            if (!other.Occured) return other;

            var thisLast = GetLast();
            CallRecord otherFirst = other._callRecords[0];

            return thisLast.Sequence < otherFirst.Sequence ? other
                : new Activities(NewOrderException(thisLast, otherFirst));
        }

        private ExpectationViolationException NewOrderException(
            CallRecord before, CallRecord after)
        {
            return new ExpectationViolationException(
                "Expected that call " + before.Method +
                " occurs before call " + after.Method +
                ", but the expectation is not satisfied.");
        }

        public Activities OccuredAfter(Activities other)
        {
            if (!Occured) return this;
            if (!other.Occured) return other;

            CallRecord otherLast = other.GetLast();
            CallRecord thisFirst = _callRecords[0];
            return otherLast.Sequence < thisFirst.Sequence ? other
                : new Activities(NewOrderException(otherLast, thisFirst));
        }

        public Activities Or(Activities other)
        {
            if (Occured) return this;
            if (other.Occured) return other;
            return new Activities(new ExpectationViolationException(
                this._exception.Message + "\nor\n" + other._exception.Message));
        }

        public Activities First {
            get {
                return Occured ? new Activities(new CallRecord[] {_callRecords[0]}) : this;
            }
        }

        public Activities Last {
            get {
                return Occured ? new Activities(new CallRecord[] { GetLast() }) : this;
            }
        }

        private CallRecord GetLast()
        {
            return _callRecords[_callRecords.Count - 1];
        }

        public int CompareTo(Activities other)
        {
            if (ReferenceEquals(this, other)) return 0;
            return OccuredBefore(other) ? -1 : 1;
        }

        public static implicit operator bool(Activities activities)
        {
            return activities.Occured;
        }

        public static Activities operator <(Activities a1, Activities a2)
        {
            return a1.OccuredBefore(a2);
        }

        public static Activities operator >(Activities a1, Activities a2)
        {
            return a1.OccuredAfter(a2);
        }

        public static Activities operator |(Activities a1, Activities a2)
        {
            return a1.Or(a2);
        }

        public static Activities operator ^(Activities a1, Activities a2)
        {
            return OneOf(a1, a2);
        }

        public static bool operator true(Activities a)
        {
            return a.Occured;
        }

        public static bool operator false(Activities a)
        {
            return !a.Occured;
        }

        public void AssertOccured()
        {
            if (_exception != null) throw _exception;
        }

        internal static Activities ExactOneOf(params Activities[] activitiesList)
        {
            Activities one = null;

            foreach (var activities in activitiesList)
            {
                if (!activities.Occured) continue;
                if (one == null) one = activities;
                else
                    return new Activities(
                        new ExpectationViolationException(
                            "Both " + one._callRecords[0].Method +
                            " and " + activities._callRecords[0].Method +
                            " was called"));
            }
            if (one != null) return one;

            StringBuilder sb = new StringBuilder("None of below is satisfied:");
            foreach (var activities in activitiesList)
            {
                sb.Append('\n').Append(activities._exception.Message);
            }
            return new Activities(new ExpectationViolationException(sb.ToString()));
        }
    }

The other class holds a few extension methods that glues this new API to existing Rhino Mocks.

    public static class Mockery
    {
        public static Activities ActivityOf<T>(this T mock, Action<T> action, 
            Action<IMethodOptions<object>> setupConstraints)
        {
            try
            {
                return new Activities(mock.AssertWasCalled(action, setupConstraints));
            }
            catch (ExpectationViolationException e)
            {
                return new Activities(e);
            }
        }

        public static Activities ActivityOf<T>(this T mock, Action<T> action)
        {
            return ActivityOf(mock, action, DefaultConstraintSetup);
        }

        public static Activities ActivityOf<T>(this T mock, Function<T, object> func, 
            Action<IMethodOptions<object>> setupConstraints)
        {
            return ActivityOf(mock, new Action<T>(t => func(t)), setupConstraints);
        }

        public static Activities ActivityOf<T>(this T mock, Function<T, object> func)
        {
            return ActivityOf(mock, func, DefaultConstraintSetup);
        }

        public static void Assert(Activities activities)
        {
            activities.AssertOccured();
        }

        public static Activities ExactOneOf(params Activities[] activitiesList)
        {
            return Activities.ExactOneOf(activitiesList);
        }

        private static void DefaultConstraintSetup(IMethodOptions<object> options)
        {
        }
    }

You can see that the implementation of the first ActivityOf method is very quick and dirty. The idea was to keep this POC code simple enough to illustrate the technique.

The Failure Message

There is one minor issue with using test framework's Assert.IsTrue. When the assert fails, it simply tells you that you expected true but got false. I don't see this as a big issue because with today's tool you can easily click on the error to jump to the exact line of code. In most of time, the code gives much more information then the message itself. But hey, it is always good to have detailed message right?

Careful reader must already found the Assert method in the Mockery class, that is the one used to replace the Assert.IsTrue. It provides "better" message then true/false. The reason for the quotation marks around the word better is that at this moment, the implementation can sometime provide ambiguous message. Again this is just a POC, in order to provide accurate message, we will need to make deeper changes into the Rhino Mocks.

Below are some examples from the unit test:

            Mockery.Assert(mb.ActivityOf(b => b.Bar()).First
                .OccuredBefore(ma.ActivityOf(a => a.Act()).First)
                .OccuredBefore(mb.ActivityOf(b => b.Bar()).Last));

            (mb.ActivityOf(b => b.Bar()).First
                < ma.ActivityOf(a => a.Act()).First
                < mb.ActivityOf(b => b.Bar()).Last).AssertOccured();
            Mockery.Assert(foo.ActivityOf(f => f.Foo(1)) | 
                foo.ActivityOf(f => f.Foo(Arg<int>.Is.Equal(1), Arg<int>.Is.Anything)));

            (foo.ActivityOf(f => f.Foo(Arg<int>.Is.Equal(1), Arg<int>.Is.Anything)) | 
                foo.ActivityOf(f => f.Foo(1))).AssertOccured();

C# 2.0 / VS2005 Support

It was said that the AAA for C# 2.0 is ugly and tedious. I tried to using this new API with C# 2.0 syntax. The result is not too bad at all. Let's take a look at what we have in the unit test cases:

            Mockery.Assert(Mockery.ActivityOf(ma, delegate(IA a) { a.Act(); }).Last
                .OccuredAfter(Mockery.ActivityOf(mb , delegate(IB b) { b.Bar(); }).Last)
                .OccuredAfter(Mockery.ActivityOf(ma, delegate(IA a) { a.Act(); }).First));

            (Mockery.ActivityOf(ma, delegate(IA a) { a.Act(); }).Last
                > Mockery.ActivityOf(mb, delegate(IB b) { b.Bar(); }).Last
                > Mockery.ActivityOf(ma, delegate(IA a) { a.Act(); }).First).AssertOccured();

            Mockery.Assert(
Mockery.ActivityOf(foo, delegate(IFoo f) { f.Foo(1); }) | Mockery.ActivityOf(foo, delegate(IFoo f) { f.Foo(Arg<int>.Is.Equal(1), Arg<int>.Is.Anything); }));

Give It A Try

If you want to give it a try, I have build a version of Rhino Mocks based on last trunk source with the latest Castle DynamicProxy2 at this moment. You can find them here:

http://code.google.com/p/kennethxublogsource/downloads/list

There are also other features in the build like:

  • Create multi mock for AAA: Mockery.GenerateMultiMock
  • Create partial mock for AAA: Mockery.GeneratePartialMock and Mocker.GeneratePartialMultiMock

Update (7/9/2009): For people don't want a modified version of Rhino.Mocks, I have made an extension library without the feature of ordered expectation: Some Utilities Extend Rhino.Mocks

Saturday, June 27, 2009

Rhino.Mocks Ordered Expectations for AAA Syntax

There was a question posted to the Google Group about how to make ordered expectations using AAA syntax in Rhino.Mocks. As people being increasingly aware of over specification using the record/reply model. Arrange-Action-Assert model is getting adopted by more and more developers. In the question, Alex was asking if is a way to make the ordered expectations using AAA syntax instead of the workaround he used below which is, IMHO, quite tedious to write and understand.

        public void ViewTitleSetBeforeShownInWorkspace()
        {
            //arrange
            var mockView = MockRepository.GenerateMock<IView>();
            bool viewTitleSet = false;
            mockView.Stub(x => x.Title = Arg<string>.Is.Anything)
                .WhenCalled(a => { viewTitleSet = true; });
            mockView.Stub(x => x.ShowInWorkspace("MyWorkspace"))
                .WhenCalled(a => { Assert.IsTrue(viewTitleSet, 
                    "View title should have been set first"); });

            //act
            var target = new Presenter(mockView);
            target.Initialize();

            //assert
            mockView.AssertWasCalled(x => x.ShowInWorkspace("MyWorkspace"));
        }

I agree with Ayende Rahien that ordered expectation is rare. But it does happen and I wish there is a way to do it when I need it. I again agreed that duplicating full ordering support from record/reply into AAA is too complex than worth. It would be not only over killed but also easily abused.

But, maybe there is a simpler way to the goal. All I need is to ensure one method should have been called before another, I want a way to verify this as easy as AssertWasCalled.

How about something like below:

            mockBefore.AssertWasCalled(b => b.MethodBefore())
                .Before(mockAfter.AssertWasCalled(a => a.MethodAfter()));

I loves to use open source tools because if nobody doing it then I do it myself. I checked out the Rhino.Mocks source code from Subversion repository. It turned out to be quite straightforward to make it work. Thanks to Ayende for the nice design. Going around the code and making changes were very easy. I was a bit afraid of scratching the crystal when making those changes. Ayende, please forgive me if I did.

Rhino.Mocks does record each call so that it can later use the information to verify the expectations. But the current design only records call parameters. I changed it to record a newly introduced CallRecord object.

    public class CallRecord
    {
        private static long sequencer = long.MinValue;

        internal CallRecord()
        {
            Sequence = Interlocked.Increment(ref sequencer);
        }
        internal object[] Arguments { get; set; }
        internal long Sequence { get; private set; }
        internal MethodInfo Method { get; set; }
    }

The key in this object is the sequencer. Every time a new CallRecord is created, the sequencer was increased in a thread safe manner. And the current sequence is recorded with the call. So each CallRecord object in the system has a unique Sequence value that can be used to determine the time order of the calls at later time.

The Arguments property is used by the original Rhino.Mocks to check expectations. Having Method property here was due to my laziness. There should be a better way to handle this for a lighter weight of CallRecord.

Next is to make the AssertWasCalled methods to return the CallRecord. Since the AssertWasCalled can match multiple method invocations. I made them return an IList<CallRecord>.

With the CallRecord information on hand, an extension method below can easily compare the order of the calls:

        public static void Before(this IList<CallRecord> beforeCalls, IList<CallRecord> afterCalls)
        {
            long maxBefore = long.MinValue;
            CallRecord latestBeforeCall = null;
            foreach (var call in beforeCalls)
            {
                var sequence = call.Sequence;
                if (sequence > maxBefore)
                {
                    maxBefore = sequence;
                    latestBeforeCall = call;
                }
            }

            long minAfter = long.MaxValue;
            CallRecord earliestAfterCall = null;
            foreach (var call in afterCalls)
            {
                var sequence = call.Sequence;
                if (sequence < minAfter)
                {
                    minAfter = sequence;
                    earliestAfterCall = call;
                }
            }
            if (maxBefore>minAfter)
            {
                throw new ExpectationViolationException(
                    "Expected that calls to " + latestBeforeCall.Method + 
                    " occurs before " + earliestAfterCall.Method + 
                    ", but the expectation is not satisfied.");

            }
        }

Again, I was lazy, the exception message should have been better formed, for example including the more detailed method signature and the real parameters. (Update: the API has been enhanced)

With all those in place, the final bit is to create a test cases to verify it works.

    [TestFixture] public class BeforeExtensionMethodTest
    {
        public interface IBefore  { void MethodBefore(); }

        public interface IAfter { void MethodAfter(); }

        [Test] public void Before_succeeds_if_beforeCalls_occured_before_afterCalls()
        {
            var mockBefore = MockRepository.GenerateStub<IBefore>();
            var mockAfter = MockRepository.GenerateStub<IAfter>();
            mockBefore.MethodBefore();
            mockBefore.MethodBefore();
            mockAfter.MethodAfter();
            mockAfter.MethodAfter();
            mockAfter.MethodAfter();
            mockBefore.AssertWasCalled(b => b.MethodBefore())
                .Before(mockAfter.AssertWasCalled(a => a.MethodAfter()));
        }

        [ExpectedException(typeof(ExpectationViolationException))]
        [Test] public void Before_chokes_if_one_of_beforeCalls_occured_after_any_of_afterCalls()
        {
            var mockBefore = MockRepository.GenerateStub<IBefore>();
            var mockAfter = MockRepository.GenerateStub<IAfter>();
            mockBefore.MethodBefore();
            mockAfter.MethodAfter();
            mockBefore.MethodBefore();
            mockAfter.MethodAfter();
            mockAfter.MethodAfter();
            mockBefore.AssertWasCalled(b => b.MethodBefore())
                .Before(mockAfter.AssertWasCalled(a => a.MethodAfter()));
        }
    }

The full patch for r2212 of https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/trunk/mocks/ can be downloaded here. Alex's original problem in the beginning now can be rewritten neatly.

        public void ViewTitleSetBeforeShownInWorkspace()
        {
            //arrange
            var mockView = MockRepository.GenerateMock<IView>();

            //act
            var target = new Presenter(mockView);
            target.Initialize();

            //assert
            mockView.AssertWasCalled(x => { x.Title = Arg<string>.Is.Anything; })
                .Before(mockView.AssertWasCalled(x => x.ShowInWorkspace("MyWorkspace")));
        }

Update 7/18: I made my customized version of Rhino Mocks available if you want to give it a try. It contains other enhancements.