bdUnit A BDD - Unit Test DSL using Microsoft Oslo

[Cross posted on the StormID blog]

Over the past few weeks, I have been working on bdUnit, a side project to create a DSL that would transform user stories into C# unit tests. In November last year Jonny, one of my colleagues at Storm, tasked me with creating a parser that would generate unit tests from user stories. This was to be a part (‘APE’) of the project he named ESCAPE - ‘Evolutionary Service Compiler’ and ‘Application Programming Environment’.

The Parser

This was a rather daunting task and my first step was to look around for some tools or a framework I could use. As luck would have it, I quickly came across the "Oslo": Building Textual DSLs PDC 2008 session. I was impressed by the power, flexibility and control that Oslo offered as well as the great tool included in the Oslo SDK – Intellipad. I also came across MisBehave a nice implementation of the DSL componet of Oslo, MGrammar, used to create executable specifications from user stories. I realised that a similar approach could be used to extend this to executable unit tests.

I was able to get started with considerable help from a couple of great posts by Torkel Ödegaard on Creating a WatiN DSL using MGrammar and the second part, Browser Automation DSL using MGrammar.

Here is a view of my grammar and an example input loaded in Intellipad:

The input is transformed using the token matching and syntax rules contained within the MGrammar to output what I would call ‘Structured Data’ and Microsoft have named MGraph. MGraph is very similar in concept to Xml – i.e. enabling the serialization of data in a structured format that can be extended and controlled by the developer. The advantage MGrammar has over an Xml or JSON serialization is that the provision of functional text and text pattern matching logic ‘out of the box’.

Deserializing

In order to make use of the structured data provided by the parser, I needed to deserialize the data and map it on to objects I could use in C#. Again, following Torkel and Roger Alsing’s lead, I created an Abstract Syntax Tree (AST) to resolve the data to CIL objects. My AST is below:

Give Me the Code

Resolving the data to CIL objects enabled me to use types and type properties in generating the C# unit test.

public string GenerateMethodSignature(CreateMethod method)
{
 var methodText = MethodText.Replace("##accesslevel##", Access.ToString());
 var _params = new StringBuilder();
 var signature = methodText.Replace("##methodname##", method.TargetMethod.Name);
 signature = signature.Replace("##returntype##", "void");
 var paramCount = method.TargetMethod.Objects.Count;
 for (var j = 1; j < paramCount; j++)
 {
  var parameter = method.TargetMethod.Objects[j];
  var instanceName = "";
  if (!string.IsNullOrEmpty(parameter.Instance.Value))
  {
   instanceName = parameter.Instance.Value;
  }
  else
  {
   instanceName = parameter.Name.ToLower();
  }
  var delimiter = j < (paramCount - 1) ? ", " : string.Empty;
  _params.Append(string.Format("I{0} {1}{2}", parameter.Name, instanceName, delimiter));
 }
 signature = signature.Replace("##params##", _params.ToString());
 return signature;
}

I created a basic demo app in WPF to show the user story input alongside the corresponding C# output:

With a few modifications, I was able to generate C# unit tests using the nUnit, xUnit and MbUnit frameworks.

• Example input
  
• nUnit output
  
• xUnit output
  
• MbUnit output

Notice the objects are created as Interfaces only and are instantiated within the unit tests using StructureMap. I did this so that the code could be used as a black box referenced dll within any project. Here is the compiled dll in Reflector:

Once a reference is added, concrete classes can be created and implement the bdUnit interfaces. In order for these concrete classes to be instantiated and used within the bdUnit test, they need to be maked up with the correct attribute. This allowed me to leave out any mention of these objects within the bdUnit.dll to hold the separation of concerns that would prevent the bdUnit generated code from interfering with implementation code within the client application/project.

[Pluggable("bdUnit")]
public class User : IUser
{
 public User(){}

 public IUser Spouse { get; set; }
 public bool IsARunner { get; set; }
 public int Age { get; set; }
 public bool IsDead { get; set; }
 public IList<IUser> Children { get; set; }

 public void Kill(IUser user)
 {
  user.IsDead = true;
 }

 public void Marry(IUser user)
 {
  this.Spouse = user;
  user.Spouse = this;
 }

 public void Visit(ISleepShop sleepshop)
 {
  this.IsDead = true;
 }

 public void Total()
 {
  throw new System.NotImplementedException();
 }
}

It appears that StructureMap does not support the instantiation of concrete classes across dll’s without using the decalritive attributes and .GetNamedInstance<>() but this may not be the case.

The tests can be run from the bdUnit.dll using a test-runner but unfortunately Resharper and Visual Studio only pick up tests within the source code of the application, and not those within a reference. It would be great to find a workaround for this to get the tests to run within the VS IDE.

The real application of bdUnit (which only become clear very recently!) is to use ‘Business Readable’ BDD user stories to provide the developer with a working technical specification or contract to be fulfilled by their implementation code without the former interfering with the latter If some of the bdUnit tests fail, the user stories can be modified and the interfaces/unit tests regenerated or the developer can make modifications to their code.

Next Steps

1. Developing the MGrammar. I need to allow for more complex relationships between objects, nesting methods within methods (e.g. When @Person(John) #Purchase @House(number34) (John) should #Pay (number34) ~Price and (John) should #Pay (number34) ~StampDuty) and support types such as DateTime, Guid etc.
   
2. Code Generation. This presently only supports analogues of Assert.IsTrue(clause) with comparisons being made in the clause. It would be good to make use of some of the wide variety of Asserts available.
   
3. Improve Error Feedback. This is mostly nonexistent at the moment and I definitely need to provide useful exceptions at the parsing an compiling steps.
   

James Lynch

del.icio.us Tags: microsoft oslo,mgrammar,dsl,bdd