How NUnit Handles Test Fixture as Inner Class

The Problem

Will NUnit execute the test fixture defined as inner class when I test run the enclosing test fixture? The answer is, it depends (as of version 2.5.2). In most of cases it won't, but if the enclosing test fixture is a generic type, it will.

Here is the test case to reproduce this:

namespace Demo

{

    [TestFixture] public class A { //plain normal

        [Test] public void Bar() { }

        [TestFixture] public class Inner {

            [Test] public void Foo() {}

        }

    }

    [TestFixture(typeof(int))] public class B<T> { // generic

        [Test] public void Bar() { }

        [TestFixture(typeof(int))] public class Inner {

            [Test] public void Foo() {}

        }

    }

    [TestFixture("dummy")] public class C { // parameterized non-generic

        public C(string dummy){}

        [Test] public void Bar() { }

        [TestFixture(typeof(int))] public class Inner {

            [Test] public void Foo() { }

        }

    }

}

And the result:

C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.A" /labels

Selected test: Demo.A

***** Demo.A.Bar

Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0262504 seconds

  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.B<T>" /labels

Selected test: Demo.B<T>

***** Demo.B`1+Inner[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Foo

***** Demo.B`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Bar

Tests run: 2, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0337505 seconds

  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

C:\Demo>nunit-console.exe Demo.Tests.dll /run="Demo.C" /labels

Selected test: Demo.C

***** Demo.C.Bar

Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 0.0350005 seconds

  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0

My Thoughts

Now which behavior do I prefer? I would like to include the inner test fixture classes when running the enclosing one. Why? that's the only reason so far I need to create inner test fixture class. And here is the use case:

1. I have MyCollection<T> : ICollection<T>, ICollection that is the subject under test.
2. I have common reusable test fixture CollectionTestFixture<T> and CollectionTestFixture contain functional test cases for types that implement ICollection<T> and/or ICollection.

Test implementation A, Create three(3) separate test fixtures

• MyCollectionGenericTests<T> : CollectionTestFixture<T>
• MyCollectionNonGenericTests : CollectionTestFixuture
• MyCollectionTests<T> // specific test cases

Pros:

• Simple
• Works

Cons:

• Must run 3 test fixture to test one class
• Difficult (impossible if using TD.Net) to run overall code coverage for MyCollection<T> class
• Test result are not grouped together when there is MyCollectionSomeElseTest.

Test implementation B, use dedicated namespace

• MyCollection.GenericTests<T> : CollectionTestFixture<T>
• MyCollection.NonGenericTests : CollectionTestFixuture
• MyCollection.Tests<T> // specific test cases

Pros:

• Works
• Can easily run all tests for MyCollection and get overall coverage

Cons:

• Creating one folder (matching namespace) for one class to be tested result in very busy project layout and hard to navigate when there are a lot of such scenarios.
• There will be a lot of repeated type names, e.g. GenericTests<T>. Otherwise MyCollection.MyCollectionGenericTests<T> makes the label reads much longer which is a bad thing.

Test implementation C, use inner classes

• MyCollectionTests<T> // specific test cases
• MyCollectionTests.AsGeneric<T> : CollectionTestFixture<T>
• MyCollectionTests.AsNonGeneric : CollectionTestFixuture

Pros:

• Can easily run all tests for MyCollection and get overall coverage
• No additional directories to create and navigate.
• Common methods can be easily reused by inner classes.
• Labels read naturally

Cons:

• Doesn’t work for non-generic enclosing test fixture, but hoping it will work…

The Winner

It is not difficult to conclude that the best implementation is C if NUnit can support it consistently. And IMHO, inner classes are members of enclosing class from the language (logical) perspective although CLR implemented (physical) the other way. I wish test framework implements the logical meaning because that’s what most developers familiar with.

Test Fixture Aggregation

I recall there was a talk about the ability to quickly create a new test fixture by aggregating test cases from multiple reusable test fixtures. The inner class approach is my proposal to the problem. Together with the ability to specify including/excluding of test cases in the inherited fixture may provide a complete solution.

My Wishes

1. Change NUnit to always consider test cases of inner class as an integral part of the enclosing test fixture. And execute then when the enclosing test fixture is specified to execute.
2. NUnit-GUI should organize the inner class under the enclosing class in its tree structure.
3. This one is minor and not important to me but it would be nice if NUnit can hide the implementation detail of CLR and translate to the dot notation for inner class like what the language does.