Glad to know that the patch I provided near two years ago are now finally applied to the source control. http://trac-hacks.org/ticket/3566
Saturday, May 15, 2010
Friday, April 30, 2010
SpringExtension: Extends DbProvider to Provide Hook to Connection Open and Close
This is inspired by the need of populating ODP.Net's OracleConnection.ClientId property with current thread identity when Spring.Net's ADO Support is used in the data access layer.
The major difference between Java’s DataSource and its Spring.Net’s counterpart, DbProvider, is that the connection returned from DbProvider is not open. With Java’s DataSource, you can easily make a wrapper to set up the connection before it’s used by the application code. But things become difficult with DbProvider, you just cannot do this. If you open the connection in the wrapper, the subsequent call to open the connection by the Spring.Net framework will fail (you wish it will check the status before calling open? nope, that’s not the case).
But there is a workaround. We can use an event handler to setup the connection at the moment Spring.Net framework opens the connection. Although IDbConnection doesn't provide any mean to let you react when a database connection is opened and/or closed, but the System.Data.Common.DbConnection object, which is the base class of most ADO implementation, has an event called StateChange. This gives us the hook to extend the DbProvider so that application code can react on connection open and/or close event. We can do many things with this, for example call a stored procedure to setup the connection options, tell database the current thread’s identity name, even setup the in-memory database for unit testing and etc.
The good new is we have implemented the hook in SpringExtension, so you don’t have to re-do the work. Using SpringExtension, you can easily achieve what I have described in my earlier post about setting oracle CLIENT_IDENTIFIER database session information, by just below configuration settings. No c# program needs to be written.
<object id="DbProvider" type="Spring.Data.Common.ExtendedDbProvider, Spring.Data.Extension">
<property name="TargetDBProvider" ref="TargetDbProvider"/>
<property name="ConnectionStateListener">
<object type="Spring.Data.Support.OdpNetClientIdentifierSetter, Spring.Data.Extension"/>
</property>
</object>
<db:provider id="TargetDbProvider" provider="OracleODP-2.0" connectionString="${DB.IMM.ConnectionString}"/>
You can further customize the action by replacing the OdpNetClientIdentifierSetter with any implementation of IDbConnectionStateListener, defined in SpringExtension as:
/// <summary>
/// Interface to receive the database connection state change.
/// </summary>
/// <author>Kenneth Xu</author>
public interface IDbConnectionStateListener
{
/// <summary>
/// To be called when the <paramref name="connection"/> state changed
/// from <paramref name="original"/> to <paramref name="current"/>.
/// </summary>
/// <param name="connection">
/// The database connection that its state has changed.
/// </param>
/// <param name="original">
/// The original state of the connection before change.
/// </param>
/// <param name="current">
/// Current state of the connection.
/// </param>
void AfterStateChange(IDbConnection connection, ConnectionState original, ConnectionState current);
}
Or if all you care is open and/or close event, you can extend DbConnectionStateListenerBase by override any of the below two virtual methods:
/// <summary>
/// This method is called after database is opened.
/// </summary>
/// <param name="connection">
/// The connection that was just opened.
/// </param>
public virtual void AfterConnectionOpen(IDbConnection connection)
{
}
/// <summary>
/// This method is called after database is closed.
/// </summary>
/// <param name="connection">
/// The connection that was just closed.
/// </param>
public virtual void AfterConnectionClose(IDbConnection connection)
{
}
Welcome comments and suggestions
Wednesday, April 28, 2010
SpringExtension: Declarative Transaction Management and Unit Test
Declarative transaction management is one of the compelling reasons to using Spring.Net. But there are still cases where you need to programmatically rollback a transaction without throwing an exception. Spring.Net let you do so with below method call:
TransactionInterceptor.CurrentTransactionStatus.SetRollbackOnly();
The problem with this call emerges when you unit test your business objects containing such statement. You got an exception when you run your test because there is no transaction context exists.
The solution to this problem is dependency injection. We need to introduce an interface that can give us the transaction status.
public interface ITransactionStatusProvider
{
ITransactionStatus CurrentTransactionStatus { get; }
}
We can then use the interface to rollback current transaction in our business objects:
1: public class BusinessObject
2: {
3: ITransactionStatusProvider TransactionStatusProvider { get; set; }
4: public void BusinessMethod()
5: {
6: // Do some work.
7: if (something is wrong)
8: {
9: TransactionStatusProvider.CurrentTransactionStatus.SetRollbackOnly();
10: }
11: }
12: }
In line 3, we declared a property so the provider can be injected; In line 9, we used the provider to rollback the transaction in error condition.
Since both provider and status are interfaces, we can easily mock them in our unit test:
1: [Test] public void BusinessMethodRollBackTransactionWhenSomethingIsWrong()
2: {
3: var mockProvider = MockRepository.GenerateMock<ITransactionStatusProvider>();
4: var mockStatus = MockRepository.GenerateMock<ITransactionStatus>();
5: mockProvider.Stub(p => p.CurrentTransactionStatus).Return(mockStatus);
6:
7: var sut = new BusinessObject {TransactionStatusProvider = mockProvider};
8: // sut.SetUpWrongCondition
9: sut.BusinessMethod();
10:
11: mockStatus.AssertWasCalled(x=>x.SetRollbackOnly());
12: }
The last thing left is to write a real implementation that actually works with the Spring.Net’s declarative transaction management:
public class TransactionInterceptorStatusProvider : ITransactionStatusProvider
{
public ITransactionStatus CurrentTransactionStatus
{
get { return TransactionInterceptor.CurrentTransactionStatus; }
}
}
We then use Spring.Net’s dependency inject feature to set the the BusinessObject.TransactionStatusProvider to an instance of TransactionInterceptorStatusProvider. Problem resolved!
This is part of SpringExtension, both source and binary are available on Google Code.
Tuesday, January 19, 2010
Always Override Equals of Your Value Type
ReSharper had been reminding me to override the Equals method and operator for value types that I have created. Every time I took the advice and let ReSharper do the work for me.
But I kept wondering how bad is the default implementation? The default implementation of Equals method is in the ValueType abstract class that every .Net value type inherits from. Initially I though it must use some sort of native memory comparison so it got to be fast. But after I looked at the code in Reflector, I’m no longer sure how true is this. It calls a native method CanCompareBits to check if current instance eligible for FastEqualsCheck. If yes great, otherwise it falls back to reflection. I don’t know what qualifies for fast equals check, but I start to realized that members with Equals method overridden cannot not use bit comparison. The reflection code told me that it uses Equals to compare members.
Enough talk, lets test the performance with two simplest value types, one has Equals overridden and another relays on the default implementation.
I used NUnit test case for simplicity but the code below can be easily changed to a console application.
1: private const int _sampleSize = 100;
2:
3: [Test] public static void Main()
4: {
5: var defaultEqualsStruct = new DefaultEqualsStruct[_sampleSize];
6: var overriddenEqualsStruct = new OverriddenEqualsStruct[_sampleSize];
7: for (int i = 0; i < _sampleSize; i++)
8: {
9: defaultEqualsStruct[i] = new DefaultEqualsStruct { Foo = i };
10: overriddenEqualsStruct[i] = new OverriddenEqualsStruct { Foo = i };
11: }
12: RunBenchMark(defaultEqualsStruct, new DefaultEqualsStruct { Foo = _sampleSize });
13: RunBenchMark(overriddenEqualsStruct, new OverriddenEqualsStruct { Foo = _sampleSize });
14: }
15:
16: public static void RunBenchMark<T>(T[] items, T e)
17: {
18: Console.WriteLine("==========" + typeof(T).Name + "=========");
19: // Warn up
20: Assert.IsTrue(ArrayIndexOf(items, items[5]));
21: Assert.IsTrue(LoopEquals(items, items[5]));
22: Clock(null, 1, () => Assert.IsFalse(ArrayIndexOf(items, e)));
23: Clock(null, 1, () => Assert.IsFalse(LoopEquals(items, e)));
24: // Run benchmark.
25: const int repeat = 100000;
26: Clock("Array.IndexOf", repeat, () => ArrayIndexOf(items, e));
27: Clock("Loop + Equals", repeat, () => LoopEquals(items, e));
28: }
29:
30: private static void Clock(string message, int repeat, Action a)
31: {
32: Stopwatch stopwatch = new Stopwatch();
33: stopwatch.Start();
34: for (int i = 0; i < repeat; i++) a();
35: stopwatch.Stop();
36: if (message != null)
37: Console.WriteLine(message + ": " + stopwatch.ElapsedTicks);
38: }
39:
40: private static bool ArrayIndexOf<T>(T[] items, T element)
41: {
42: return (Array.IndexOf(items, element) >= 0);
43: }
44:
45: private static bool LoopEquals<T>(T[] items, T element)
46: {
47: int size = items.Length;
48: for (int i = 0; i < size; i++)
49: {
50: if (Equals(element, items[i])) return true; // found
51: }
52: return false;
53: }
54:
55: public struct DefaultEqualsStruct
56: {
57: public int Foo { get; set; }
58: }
59:
60: public struct OverriddenEqualsStruct
61: {
62: public int Foo { get; set; }
63:
64: public bool Equals(OverriddenEqualsStruct other)
65: {
66: return other.Foo == Foo;
67: }
68:
69: public override bool Equals(object obj)
70: {
71: if (ReferenceEquals(null, obj)) return false;
72: if (obj.GetType() != typeof(OverriddenEqualsStruct)) return false;
73: return Equals((OverriddenEqualsStruct)obj);
74: }
75:
76: public override int GetHashCode()
77: {
78: return Foo;
79: }
80: }
And here goes the test result in release build:
1: ==========DefaultEqualsStruct=========
2: Array.IndexOf: 2863736
3: Loop + Equals: 1424624
4: ==========OverriddenEqualsStruct=========
5: Array.IndexOf: 289861
6: Loop + Equals: 509123
That’s really a lot of difference! Thanks ReSharper!
Monday, January 18, 2010
How Fast Are Methods In .Net Array Class
It’s very fast.
When I was porting the CopyOnWriteArrayList class from Java to .Net, I came across the implementation of the AddIfAbsent method. In Java it was written as:
1: public boolean addIfAbsent(E e) {
2: final ReentrantLock lock = this.lock;
3: lock.lock();
4: try {
5: // Copy while checking if already present.
6: // This wins in the most common case where it is not present
7: Object[] elements = getArray();
8: int len = elements.length;
9: Object[] newElements = new Object[len + 1];
10: for (int i = 0; i < len; ++i) {
11: if (eq(e, elements[i]))
12: return false; // exit, throwing away copy
13: else
14: newElements[i] = elements[i];
15: }
16: newElements[len] = e;
17: setArray(newElements);
18: return true;
19: } finally {
20: lock.unlock();
21: }
22: }
If you read the comment in the code above, the implementation creates a new array regardless of whether the element exists or not. If it finds out that the element exists during the copying of existing elements between arrays, it simply discards the newly created array.
Comparing this to the typical approach that search first then create new array if not found, it wins in the case when the element doesn’t exist by eliminating the search.
If I directly translate the same code into .Net, I won’t be able use any of utility methods provided by Array class in .Net framework. So I have to choose between a) sticking with the Java’s implementation, or b) using Array.IndexOf then Array.Copy if found.
Let’s set out and run a benchmark (I’m using NUnit test case but the code can be easily called by a Program.cs file).
Let’s only test the use case where the element is not found, to see if the Array.IndexOf+Array.Copy implementation (let’s call it “Find Then Copy”) can compete with find when copying implementation (“Copy With Find”) of Java.
1: [TestFixture(typeof(int))]
2: [TestFixture(typeof(string))]
3: public class CopyOnWriteListTest<T>
4: {
5: private static readonly T[] _items = TestData<T>.MakeTestArray(100);
6:
7: [Test] public void RunBenchMark()
8: {
9: Console.WriteLine("==========" + typeof (T).Name + "=========");
10:
11: // Confirm that Array.IndexOf use value equals, not reference equals.
12: T value = TestData<T>.MakeData(5);
13: int index = Array.IndexOf(_items, value);
14: Assert.That(index, Is.EqualTo(5));
15: Assert.That(value, Is.Not.SameAs(_items[index]));
16: Assert.That(value, Is.EqualTo(_items[index]));
17:
18: T e = TestData<T>.MakeData(_items.Length);
19: // Warn up
20: Clock(null, 1, () => FindThenCopy(e));
21: Clock(null, 1, () => CopyWithFind(e));
22: // Run benchmark.
23: const int repeat = 100000;
24: Clock("Find Then Copy", repeat, () => FindThenCopy(e));
25: Clock("Copy With Find", repeat, () => CopyWithFind(e));
26: }
27:
28: private static void Clock(string message, int repeat, Action a)
29: {
30: Stopwatch stopwatch = new Stopwatch();
31: stopwatch.Start();
32: for (int i = 0; i < repeat; i++) a();
33: stopwatch.Stop();
34: if (message != null)
35: Console.WriteLine(message + ": " + stopwatch.ElapsedTicks);
36: }
37:
38: private static void FindThenCopy(T element)
39: {
40: T[] items = _items;
41: var index = Array.IndexOf(items, element);
42: if (index >= 0) return; // found
43: int size = items.Length;
44: var newItems = new T[size + 1];
45: Array.Copy(items, 0, newItems, 0, size);
46: newItems[size] = element;
47: }
48:
49: private static void CopyWithFind(T element)
50: {
51: T[] items = _items;
52: int size = items.Length;
53: var newItems = new T[size + 1];
54: for (int i = 0; i < size; i++)
55: {
56: if (Equals(element, items[i])) return; // found
57: newItems[i] = items[i];
58: }
59: newItems[size] = element;
60: }
61: }
And here is the result:
1: ==========Int32=========
2: Find Then Copy: 129702
3: Copy With Find: 547511
4: ==========String=========
5: Find Then Copy: 274798
6: Copy With Find: 375962
Although it is two pass approach using Array class, native code is still significantly faster than looping in managed code. Value type showing larger difference is because the boxing when calling the Equals method.
Result changes a little when the array size is getting smaller. When we run it with array size of 20:
1: ==========Int32=========
2: Find Then Copy: 82526
3: Copy With Find: 112597
4: ==========String=========
5: Find Then Copy: 107412
6: Copy With Find: 108321
And 10:
1: ==========Int32=========
2: Find Then Copy: 36767
3: Copy With Find: 84236
4: ==========String=========
5: Find Then Copy: 68216
6: Copy With Find: 57618
This is because the overhead of calling the method into Array class starts to play the game. The “Copy With Find” implementation gets faster for reference type when array size is below 20. But both implementations are fast when array is small anyway so I don’t see much benefit of using “Copy With Find” approach in .Net.
I have also tested with custom reference type and value types, the result is comparable to int and string test.
Conclusion: Array class won the race! Yes, array operations using methods in Array class are indeed very fast.
Friday, December 04, 2009
Class Diagram For Major Collection Interfaces in .Net
Today, I want to show somebody a class diagram for major collection interfaces in .Net. I created one after failed Googling one. So here it is.
And below for enumerators. I hope I can show “has a” relationship but VS 2008 doesn’t let me do it. So I placed them side by side to show enumerable has enumerator.
Wednesday, November 04, 2009
Unit Test Value Object – Reusable Test Fixture
To Test Or Not Test
Shall I unit test value object? Here are some arguments that I found when googling for an answer to the question.
- Value object are so simple that you don’t need to test them.
- Test value object is tedious, basically copy and pasting and you can make the same mistake that you did in the value object itself.
- If you test every other part of the application, you would have tested your value object.
And there are also opposite opinions.
- Value objects are not as simple as you thought. How about initial property value, equals, hashcode and cloneable? (Yes, that reminds me that I made quite some mistakes in value objects)
- Unit test is tedious in nature. Tedious is not a reason for not doing it. But it only means that we need to find a better way.
- This is about unit testing, not integration testing. Value object is a unit that should be tested alone.
In summary, value objects are relatively simple so that the cost of manually writing test cases for them are too high to justify the benefit but not testing them leaving potential bugs.
The Reusable Test Fixture
Compromising the quality is a no in my book so the solution is to develop reusable tests to replace the repetitive and tedious work. This is what programmers are good at after all!
Let’s take a look at a very simple value object.
public class Foo
{
public int Id { get; set; }
public string StringProperty { get; set; }
public long LongProperty { get; set; }
}
Writing a unit test for this class manually is plain boring. I should be able just do it in one line with a reusable test fixture.
[TestFixture] public void FooTest : SomeMagicReusableTestFixture<Foo> {}
This magic reusable test fixture is called NewableValueObjectTestFixture. It sets each property with a few (three minimal) values and read it back to make sure it gets the same value previously set. Of course, it does much more then just that…
Default Property Value
The above example is probably too simple. What if my property has default value? say the Id property is initialized to –1. No problem! The reusable test fixture also verifies the initial value of properties too. It expects all the properties are initialized to the default value of the type, or a value specified by DefaultValueAttribute.
The code below will fail the test because the the default value for int type is 0, not –1.
private int _id = -1;
public int Id
{
get { return _id; }
set { _id = value; }
}
If you intended to initialized to –1, you should tell this to the test fixture. The information is also useful to other .Net tools.
private int _id = -1;
[DefaultValue(-1)]
public int Id
{
get { return _id; }
set { _id = value; }
}
The code above passes the test happily.
Equals and GetHashCode
NewableValueObjectTestFixture and its brother ValueObjectTestFixture test the Equals and GetHashCode methods too. They basically ensure that:
- The value object does not equals to null.
- The value object does not equals an arbitrary object (new object()).
- The value object equals to itself.
- The value objects are not equal if any one of their properties are not equal.
- When two value objects have same property values and Equals method returns true. The GetHashCode methods must return same value.
The implementation in the Object class complies to the above rules so the your value object passes the tests by default. But if you decide to override any of them, the test helps to ensure you have implemented Equals and GetHashCode properly, otherwise the test will fail. Below is generated by Resharper and it passes the test.
public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
if (ReferenceEquals(this, obj)) return true;
if (obj.GetType() != typeof(Foo)) return false;
return Equals((Foo)obj);
}
public bool Equals(Foo other)
{
if (ReferenceEquals(null, other)) return false;
if (ReferenceEquals(this, other)) return true;
return other._id == _id &&
Equals(other.StringProperty, StringProperty) &&
other.LongProperty == LongProperty;
}
public override int GetHashCode()
{
unchecked
{
int result = _id;
result = (result * 397) ^ (StringProperty != null ? StringProperty.GetHashCode() : 0);
result = (result * 397) ^ LongProperty.GetHashCode();
return result;
}
}
A few days later if I add another property to the Foo class and forget to added it to the Equals method, the test fixture will effectively catch it and fails the test. This is exactly what I wanted, how many times did I forget this?
Cloneable
If my value object implements ICloneable, the test fixture will help to check that as well. It clones the object and test all readable properties to make sure the cloned one has the same property values. Below is my typical clone implementation and it works well with the test fixture.
public class Foo : ICloneable
{
public int Id { get; set; }
public string StringProperty { get; set; }
public long LongProperty { get; set; }
object ICloneable.Clone()
{
return Clone();
}
public Foo Clone()
{
return (Foo)MemberwiseClone();
}
}
Type of Property
Since the test fixture needs to generate test data, it had limited support for the type of the properties. For example, a property of interface type would be difficult for ValueObjectTestFixture to create test data of it.
Although, out of the box, ValueObjectTestFixture only support properties of c# build-in data types plus DateTime and TimeSpan, but it provides two extension mechanisms. You can either use a mock framework to generate mock objects for interface type and class type, or override the TestData method to provide your own test data for types that are not supported by the default implementation. Of course, nothing prevents you from making mixed use of both mechanisms.
Future enhancement may have ValueObjectTestFixture support enum, concrete class and/or maybe struct.
Using Mock Framework to Generate Test Data
ValueObjectTestFixture accepts an IMockTestDataProvider instance. I have included an implementation using RhinoMocks. But you can easily do the same using any other mock frameworks. RhinoMockTestDataProvider mocks any interfaces, or classes that are not sealed and has a default constructor. We may be able to remove the requirement of a default constructor in future but for now that is it. :)
Take a look at below value object:
public class ValueObject
{
public IList<string> StringIList { get; set; }
public List<TimeSpan> TimeSpanList { get; set; }
public ICollection<int> IntICollection { get; set; }
public IDictionary<string, object> StringObjectIDictrionary { get; set; }
public Dictionary<int, DateTime> IntDataTimeDictionary { get; set; }
public Component Component { get; set; }
// other properties, equals, hash code, clone and etc.
}
To test this, the test class is as simple as below:
[TestFixture]
public class MockProviderTest : NewableValueObjectTestFixture<ValueObject>
{
public MockProviderTest()
{
MockProvider = new RhinoMockTestDataProvider();
}
}
The complete example can be found in Google code: MockProviderTest.cs
Provide Your Own Test Data
If you have property type that is neither supported out-of-box, nor by a mock framework. You can always provide it yourself by overriding the method TestData. Below is an example from CustomTest.cs:
protected override IEnumerable TestData(PropertyInfo property)
{
// ValueObjectTestFixture doesn't support enum at this moment
// so we have to provide our own test data.
if (property.PropertyType == typeof(Operation))
{
return new[]
{
Operation.Subtraction,
new Operation(),
Operation.Addition,
Operation.Muiplication,
Operation.Division
};
}
return base.TestData(property);
}
Extensibility
While ValueObjectTestFixture makes simple things simpler, it ensures complex things possible too. Here are some additional features that will come handy when testing not so simple value objects.
Value Object without Default Constructor
What if the value object doesn’t have a default constructor? No problem, have the test case inherit from ValueObjectTestFixture. The only difference between ValueObjectTestFixture and NewableValueObjectTestFixture is that the former forces you to override an abstract method: NewValueObject, which you are required to return a new instance of the value object under test. Below example is from CustomTest.cs:
protected override CustomObject NewValueObject()
{
return new CustomObject("TestObject");
}
Test Special Properties Myself
There are times that value object has non-simple properties. For example, when a read only property is initialized by the constructor:
public class CustomObject
{
private readonly string _name;
public CustomObject(string name)
{
_name = name;
}
public string Name { get { return _name; } }
}
Or, when a property is calculated from others:
public int Result
{
get
{
switch (Operation)
{
case Operation.Addition:
return LeftOperand + RightOperand;
case Operation.Subtraction:
return LeftOperand - RightOperand;
case Operation.Muiplication:
return LeftOperand * RightOperand;
case Operation.Division:
return LeftOperand / RightOperand;
default:
throw new InvalidOperationException("Unsupported operation " + Operation);
}
}
}
ValueObjectTestFixture provides two ways to exclude those properties from being tested by it. But then you need to write the test cases yourself.
One way is to use one of the ExcludeProperties methods. Typically you call it in the constructor of your test fixture. Below is from CustomText.cs:
[TestFixture]
public class CustomTest : ValueObjectTestFixture<CustomTest.CustomObject>
{
public CustomTest()
{
// Exclude the Name property from being tested by ValueObjectTestFixture.
ExcludeProperties("Name");
}
// ... ...
}
This completely excludes the property from all the tests administered by ValueObjectTestFixture.
Another less aggressive approach is to override one of the XyzCandidates methods. They are listed below to provide a custom list of properties for the corresponding tests:
- DefaultTestCandidates – default value test.
- ReadWriteTestCandidates – read/write test.
- EqualsTestCandidates – equals and hash code test.
- ClonedProperties – cloneable test.
Example from CustomText.cs:
public override IEnumerable<PropertyInfo> EqualsTestCandidates()
{
// Exclude the Tag property from equality test.
return from p in base.EqualsTestCandidates() where p.Name != "Tag" select p;
}
You opinions is important. Let me know what you think :)