bdUnit.Preview Demo

I have been working on a few new features over the past couple of evenings (and early mornings!) and I’ve made a short demo video of the current state of the preview app.

The video runs through the following features:

• Input syntax highlighting
• Tests generated in nUnit, xUnit and MbUnit
• Parsing error handling and reporting within the app
• Error highlighted within input text
• Clickable error links to position in input text

I was aiming to get at least some of the cool features from Intellipad but with an ‘end-to-end’ preview rather than the tree-preview/mgraph view offered by Intellipad. I took this approach from the MGrammar Xaml sample produced by SpankyJ.

I am now going to concentrate on extending the MGrammar at the heart of the project and hope to have an update on this within a week or so.

Compiling a Dll Programmatically using CSharpCodeProvider

For bdUnit, I needed to output the generated source code as a compiled dll. I was dreading this but, when I bit the bullet, the code turned out to be pretty clean and simple:

using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.IO;
using System.Text;
using bdUnit.Core;
using Core.Enum;
using Microsoft.CSharp;

namespace bdUnit.Core
{
 public class DllBuilder
 {
    private readonly string[] References =
     new[]   {
        "Rhino.Mocks", "nunit.core", "nunit.core.interfaces", "nunit.framework",
        "xunit", "MbUnit.Framework", "StructureMap", "StructureMap.AutoMocking"
       };

    public string CompileDll(string folderPath, UnitTestFrameworkEnum currentFramework)
    {
     CodeDomProvider compiler = 
      new CSharpCodeProvider(new Dictionary<string, string>{{"CompilerVersion","v3.5"}});
   
     var parameters = new CompilerParameters
       {
        GenerateInMemory = false,
        GenerateExecutable = false,
        IncludeDebugInformation = true,
        OutputAssembly = string.Format("bdUnit_{0}.dll", currentFramework),
        ReferencedAssemblies = {"System.dll"}
       };
   
     foreach (var reference in References)
     {
      parameters.ReferencedAssemblies.Add(AppDomain.CurrentDomain.BaseDirectory
                 + string.Format("\\{0}.dll", reference));
     }
     var source = GetSource(folderPath, currentFramework);
     var results = compiler.CompileAssemblyFromSource(parameters, source);
   
     if (results.Errors.HasErrors)
     {
      var errorText = new StringBuilder();
      var count = results.Errors.Count;
      for (var i = 0; i < count; i++)
      {
       errorText.AppendLine("Compilation Error: " + results.Errors[i].ErrorText);
      }
      return errorText.ToString();
     }
     return "Succesfully Generated Dll";
    }

    private static string[] GetSource(string folderPath, UnitTestFrameworkEnum framework)
    {
     Directory.SetCurrentDirectory(folderPath);
     var files = Directory.GetFiles(folderPath, "*.input");
     var source = new string[files.Length];
     for (var i = 0; i < files.Length; i++)
     {
      var paths = new Dictionary<string, string>
          {
           {"input", string.Format("{0}", files[i])},
           {"grammar", Settings.GrammarPath}
          };
      var parser = new Parser(paths);
      source[i] = parser.Parse(framework);
     }
     return source;
    }
 }
}

I setup my compile parameters with the framework and references that I need. Note that these references are only present as includes and the corresponding dlls need to be present in the bin folder where the dynamically compiled dll is used. There may also be issue at runtime if different versions of such dependancies are referenced in the dll and present in the bin folder.

I use the GetSource method to parse the inputs/user stories and construct an array of source code, each member representing a .cs file. I then pass this as an argument for CompileAssemblyFromSource and report any errors that prevent the compilation from completing. The errors are informative and cover missing/misnamed references as well as build errors in the source code itself.

In the next post I will run through further developments to the MGrammar used in bdUnit including support for new input statements and steps I have taken to refactor the grammar itself.

James Lynch

bdUnit A BDD - Unit Test DSL using Microsoft Oslo

[Cross posted on the StormID blog]

Over the past few weeks, I have been working on bdUnit, a side project to create a DSL that would transform user stories into C# unit tests. In November last year Jonny, one of my colleagues at Storm, tasked me with creating a parser that would generate unit tests from user stories. This was to be a part (‘APE’) of the project he named ESCAPE - ‘Evolutionary Service Compiler’ and ‘Application Programming Environment’.

The Parser

This was a rather daunting task and my first step was to look around for some tools or a framework I could use. As luck would have it, I quickly came across the "Oslo": Building Textual DSLs PDC 2008 session. I was impressed by the power, flexibility and control that Oslo offered as well as the great tool included in the Oslo SDK – Intellipad. I also came across MisBehave a nice implementation of the DSL componet of Oslo, MGrammar, used to create executable specifications from user stories. I realised that a similar approach could be used to extend this to executable unit tests.

I was able to get started with considerable help from a couple of great posts by Torkel Ödegaard on Creating a WatiN DSL using MGrammar and the second part, Browser Automation DSL using MGrammar.

Here is a view of my grammar and an example input loaded in Intellipad:

The input is transformed using the token matching and syntax rules contained within the MGrammar to output what I would call ‘Structured Data’ and Microsoft have named MGraph. MGraph is very similar in concept to Xml – i.e. enabling the serialization of data in a structured format that can be extended and controlled by the developer. The advantage MGrammar has over an Xml or JSON serialization is that the provision of functional text and text pattern matching logic ‘out of the box’.

Deserializing

In order to make use of the structured data provided by the parser, I needed to deserialize the data and map it on to objects I could use in C#. Again, following Torkel and Roger Alsing’s lead, I created an Abstract Syntax Tree (AST) to resolve the data to CIL objects. My AST is below:

Give Me the Code

Resolving the data to CIL objects enabled me to use types and type properties in generating the C# unit test.

public string GenerateMethodSignature(CreateMethod method)
{
 var methodText = MethodText.Replace("##accesslevel##", Access.ToString());
 var _params = new StringBuilder();
 var signature = methodText.Replace("##methodname##", method.TargetMethod.Name);
 signature = signature.Replace("##returntype##", "void");
 var paramCount = method.TargetMethod.Objects.Count;
 for (var j = 1; j < paramCount; j++)
 {
  var parameter = method.TargetMethod.Objects[j];
  var instanceName = "";
  if (!string.IsNullOrEmpty(parameter.Instance.Value))
  {
   instanceName = parameter.Instance.Value;
  }
  else
  {
   instanceName = parameter.Name.ToLower();
  }
  var delimiter = j < (paramCount - 1) ? ", " : string.Empty;
  _params.Append(string.Format("I{0} {1}{2}", parameter.Name, instanceName, delimiter));
 }
 signature = signature.Replace("##params##", _params.ToString());
 return signature;
}

I created a basic demo app in WPF to show the user story input alongside the corresponding C# output:

With a few modifications, I was able to generate C# unit tests using the nUnit, xUnit and MbUnit frameworks.

• Example input
  
• nUnit output
  
• xUnit output
  
• MbUnit output

Notice the objects are created as Interfaces only and are instantiated within the unit tests using StructureMap. I did this so that the code could be used as a black box referenced dll within any project. Here is the compiled dll in Reflector:

Once a reference is added, concrete classes can be created and implement the bdUnit interfaces. In order for these concrete classes to be instantiated and used within the bdUnit test, they need to be maked up with the correct attribute. This allowed me to leave out any mention of these objects within the bdUnit.dll to hold the separation of concerns that would prevent the bdUnit generated code from interfering with implementation code within the client application/project.

[Pluggable("bdUnit")]
public class User : IUser
{
 public User(){}

 public IUser Spouse { get; set; }
 public bool IsARunner { get; set; }
 public int Age { get; set; }
 public bool IsDead { get; set; }
 public IList<IUser> Children { get; set; }

 public void Kill(IUser user)
 {
  user.IsDead = true;
 }

 public void Marry(IUser user)
 {
  this.Spouse = user;
  user.Spouse = this;
 }

 public void Visit(ISleepShop sleepshop)
 {
  this.IsDead = true;
 }

 public void Total()
 {
  throw new System.NotImplementedException();
 }
}

It appears that StructureMap does not support the instantiation of concrete classes across dll’s without using the decalritive attributes and .GetNamedInstance<>() but this may not be the case.

The tests can be run from the bdUnit.dll using a test-runner but unfortunately Resharper and Visual Studio only pick up tests within the source code of the application, and not those within a reference. It would be great to find a workaround for this to get the tests to run within the VS IDE.

The real application of bdUnit (which only become clear very recently!) is to use ‘Business Readable’ BDD user stories to provide the developer with a working technical specification or contract to be fulfilled by their implementation code without the former interfering with the latter If some of the bdUnit tests fail, the user stories can be modified and the interfaces/unit tests regenerated or the developer can make modifications to their code.

Next Steps

1. Developing the MGrammar. I need to allow for more complex relationships between objects, nesting methods within methods (e.g. When @Person(John) #Purchase @House(number34) (John) should #Pay (number34) ~Price and (John) should #Pay (number34) ~StampDuty) and support types such as DateTime, Guid etc.
   
2. Code Generation. This presently only supports analogues of Assert.IsTrue(clause) with comparisons being made in the clause. It would be good to make use of some of the wide variety of Asserts available.
   
3. Improve Error Feedback. This is mostly nonexistent at the moment and I definitely need to provide useful exceptions at the parsing an compiling steps.
   

James Lynch

del.icio.us Tags: microsoft oslo,mgrammar,dsl,bdd