Momento Extendo

Recently I have been using an iPhone app called Momento. This is a diary app with a difference. As with any diary app, you can add diary entries complete with photos and tags. Momento also allows you to connect to web services such as Twitter, Facebook and Last.fm to populate your diary with your online activity. You can also add RSS/Atom feeds to be included in your diary. Here are the feeds I have currently in Momento:

  1. Foursquare
  2. Instapaper: Unread
  3. Instapaper: Archive
  4. Twitter
  5. Last.fm Loved Tracks
  6. Youtube
  7. Vimeo
  8. Delicious Bookmarks
  9. Quora: Followed Questions
  10. Librarything
  11. Github
  12. Assembla

I created Momento Extendo so I could use some more Twitter related feeds with Momento:

  1. Friendships
    Followed Friends
    Unfollowed Friends
  2. Lists
    New Lists
    Added List Members
    Removed List Members
    Added Subscribed Lists
    Unsubscribed Lists

I thought this would be a good opportunity to learn some Ruby an try hosting an app on Heroku. I choose Sinatra as the web framework because it was lightweight, easy to learn and took little to no time to get up and running. A couple of Heroku add-ons I used were Apigee for Twitter and Cloudant Hosted CouchDB. Apigee provide a proxied Twitter Api endpoint that gives you a whitelisted rate limit (20k reqs/hour). The free Cloudant hosted CouchDB solution offers 250 MB of storage with a hosted Futon admin.

The main purpose of the app is to regularly poll Twitter to detect changes in a user’s friends and lists in order to provide a feed of these changes which can be used with Momento. This polling is required because, at the moment, Twitter does not included timestamps with friendship-follow/unfollow or list activities. Initially, I thought about using Delayed Jobs to take care of the polling but this would involve using a background worker process for which Heroku charges for. As the app is going to have fairly low traffic and the single HTTP process or ‘Dyno’ can handle between 10 and 100 requests per second, I started looking for a solution that could ping the app periodically over HTTP.

I found Moment, a service that allows you to create a delayed job that makes an HTTP request at the time specified.

moment

I use the Daily Cron Heroku add-on to remove any completed Moment jobs.

Heroku Add-Ons

Gems

Final Result

Some items from my Twitter Friendships feed in Momento:

Photo Feb 19, 22 59 51

Keystroke Launchers – A Developer’s Best Friend

Launchy is an open source keystroke launcher that has been around since 2007. I had used it in the past but without any real purpose and I didn’t give it enough time or attention to discover its numerous applications. Recently I have being using it at work to reduce some of the time I spend looking for and clicking on toolbar and quick-launch shortcuts. When writing code, the last thing I want to do is have to reach for the mouse and dig around for a shortcut. I can run Launchy from within any application by using the Alt+Space keyboard shortcut.

Launchy assigns keywords to certain actions and runs a search for your input to find any matching keywords. The key to using Launchy effectively is to keep the number of shortcuts and commands to the bare minimum so that the keywords you use will match the shortcut/command you want to execute.

1) Apps and Folders

Launchy will index all of the shortcuts and folders in your start menu (as does the search in Vista and Windows 7 start menus).

image

image

2) Solutions

By indexing your development folder for *.sln and *.csproj files, you can open up a solution/project in Visual Studio without having to go to Windows Explorer or the VS Start Page!

image

image

3) Runner Plugin

Launchy comes with a Runner and Weby plugin that you can use to run program and web based commands. Let’s look at the Runner plugin first.

You can setup runner commands to link a program with a keyword and allow user input:

image

A command can be executed by entering [Keyword] Tab [Parameter]:

image

image

4) Weby Plugin and Custom Searches

Launchy will index your browser bookmarks by default (IE and FF) and comes with a slew of searches such as Google, Wikipedia, Amazon etc. You can add to these and I’ve added a couple of custom Twitter searches:

image

Running the search commands follows the same [Keyword] Tab [Parameter]pattern as the Runner.

hash Tab fail would give:

image

and wiki Tab launchy would give:

image

Scot Alt.Net Presentation

Last week, the guys at ScotAlt.Net were kind enough to let me give a presentation on using Microsoft Oslo to create a Domain Specific Language (DSL). My talk was preceded by an excellent overview of developing a DSL in Boo which was given by Paul Cowan.

I enjoyed Paul’s talk and I was pleased that my own presentation went fairly smoothly.

I hope to go into some more detail on some of the points covered in the talk with a series of upcoming posts.

bdUnit – It’s back

I have been really busy lately so I haven’t been able to give bdUnit the attention it deserves. This week I have started adding features again and I thought it would be a good time to put up a video on using bdUnit with a VS project…

Download bdUnit-walkthrough.wmv directly

I briefly talk about a couple of the DSL features at the end of the video but I will put up a more in-depth post on this later in the week.

bdUnit.Preview Demo

I have been working on a few new features over the past couple of evenings (and early mornings!) and I’ve made a short demo video of the current state of the preview app.

The video runs through the following features:

  • Input syntax highlighting
  • Tests generated in nUnit, xUnit and MbUnit
  • Parsing error handling and reporting within the app
  • Error highlighted within input text
  • Clickable error links to position in input text

I was aiming to get at least some of the cool features from Intellipad but with an ‘end-to-end’ preview rather than the tree-preview/mgraph view offered by Intellipad. I took this approach from the MGrammar Xaml sample produced by SpankyJ.

I am now going to concentrate on extending the MGrammar at the heart of the project and hope to have an update on this within a week or so.

Compiling a Dll Programmatically using CSharpCodeProvider

For bdUnit, I needed to output the generated source code as a compiled dll. I was dreading this but, when I bit the bullet, the code turned out to be pretty clean and simple:


using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.IO;
using System.Text;
using bdUnit.Core;
using Core.Enum;
using Microsoft.CSharp;

namespace bdUnit.Core
{
public class DllBuilder
{
private readonly string[] References =
new[] {
"Rhino.Mocks", "nunit.core", "nunit.core.interfaces", "nunit.framework",
"xunit", "MbUnit.Framework", "StructureMap", "StructureMap.AutoMocking"
};

public string CompileDll(string folderPath, UnitTestFrameworkEnum currentFramework)
{
CodeDomProvider compiler =
new CSharpCodeProvider(new Dictionary<string, string>{{"CompilerVersion","v3.5"}});

var parameters = new CompilerParameters
{
GenerateInMemory = false,
GenerateExecutable = false,
IncludeDebugInformation = true,
OutputAssembly = string.Format("bdUnit_{0}.dll", currentFramework),
ReferencedAssemblies = {"System.dll"}
};

foreach (var reference in References)
{
parameters.ReferencedAssemblies.Add(AppDomain.CurrentDomain.BaseDirectory
+ string.Format("\\{0}.dll", reference));
}
var source = GetSource(folderPath, currentFramework);
var results = compiler.CompileAssemblyFromSource(parameters, source);

if (results.Errors.HasErrors)
{
var errorText = new StringBuilder();
var count = results.Errors.Count;
for (var i = 0; i < count; i++)
{
errorText.AppendLine("Compilation Error: " + results.Errors[i].ErrorText);
}
return errorText.ToString();
}
return "Succesfully Generated Dll";
}

private static string[] GetSource(string folderPath, UnitTestFrameworkEnum framework)
{
Directory.SetCurrentDirectory(folderPath);
var files = Directory.GetFiles(folderPath, "*.input");
var source = new string[files.Length];
for (var i = 0; i < files.Length; i++)
{
var paths = new Dictionary<string, string>
{
{"input", string.Format("{0}", files[i])},
{"grammar", Settings.GrammarPath}
};
var parser = new Parser(paths);
source[i] = parser.Parse(framework);
}
return source;
}
}
}

I setup my compile parameters with the framework and references that I need. Note that these references are only present as includes and the corresponding dlls need to be present in the bin folder where the dynamically compiled dll is used. There may also be issue at runtime if different versions of such dependancies are referenced in the dll and present in the bin folder.

I use the GetSource method to parse the inputs/user stories and construct an array of source code, each member representing a .cs file. I then pass this as an argument for CompileAssemblyFromSource and report any errors that prevent the compilation from completing. The errors are informative and cover missing/misnamed references as well as build errors in the source code itself.

In the next post I will run through further developments to the MGrammar used in bdUnit including support for new input statements and steps I have taken to refactor the grammar itself.


James Lynch

bdUnit A BDD - Unit Test DSL using Microsoft Oslo

[Cross posted on the StormID blog]

Over the past few weeks, I have been working on bdUnit, a side project to create a DSL that would transform user stories into C# unit tests. In November last year Jonny, one of my colleagues at Storm, tasked me with creating a parser that would generate unit tests from user stories. This was to be a part (‘APE’) of the project he named ESCAPE - ‘Evolutionary Service Compiler’ and ‘Application Programming Environment’.

The Parser

This was a rather daunting task and my first step was to look around for some tools or a framework I could use. As luck would have it, I quickly came across the "Oslo": Building Textual DSLs PDC 2008 session. I was impressed by the power, flexibility and control that Oslo offered as well as the great tool included in the Oslo SDK – Intellipad. I also came across MisBehave a nice implementation of the DSL componet of Oslo, MGrammar, used to create executable specifications from user stories. I realised that a similar approach could be used to extend this to executable unit tests.

I was able to get started with considerable help from a couple of great posts by Torkel Ă–degaard on Creating a WatiN DSL using MGrammar and the second part, Browser Automation DSL using MGrammar.

Here is a view of my grammar and an example input loaded in Intellipad:

bdUnitMGrammar

The input is transformed using the token matching and syntax rules contained within the MGrammar to output what I would call ‘Structured Data’ and Microsoft have named MGraph. MGraph is very similar in concept to Xml – i.e. enabling the serialization of data in a structured format that can be extended and controlled by the developer. The advantage MGrammar has over an Xml or JSON serialization is that the provision of functional text and text pattern matching logic ‘out of the box’.

Deserializing

In order to make use of the structured data provided by the parser, I needed to deserialize the data and map it on to objects I could use in C#. Again, following Torkel and Roger Alsing’s lead, I created an Abstract Syntax Tree (AST) to resolve the data to CIL objects. My AST is below:

bdUnitAST

Give Me the Code

Resolving the data to CIL objects enabled me to use types and type properties in generating the C# unit test.

public string GenerateMethodSignature(CreateMethod method)
{
var methodText = MethodText.Replace("##accesslevel##", Access.ToString());
var _params = new StringBuilder();
var signature = methodText.Replace("##methodname##", method.TargetMethod.Name);
signature = signature.Replace("##returntype##", "void");
var paramCount = method.TargetMethod.Objects.Count;
for (var j = 1; j < paramCount; j++)
{
var parameter = method.TargetMethod.Objects[j];
var instanceName = "";
if (!string.IsNullOrEmpty(parameter.Instance.Value))
{
instanceName = parameter.Instance.Value;
}
else
{
instanceName = parameter.Name.ToLower();
}
var delimiter = j < (paramCount - 1) ? ", " : string.Empty;
_params.Append(string.Format("I{0} {1}{2}", parameter.Name, instanceName, delimiter));
}
signature = signature.Replace("##params##", _params.ToString());
return signature;
}

I created a basic demo app in WPF to show the user story input alongside the corresponding C# output:

bdUnitPreview

With a few modifications, I was able to generate C# unit tests using the nUnit, xUnit and MbUnit frameworks.
Notice the objects are created as Interfaces only and are instantiated within the unit tests using StructureMap. I did this so that the code could be used as a black box referenced dll within any project. Here is the compiled dll in Reflector:

bdUnitReflector

Once a reference is added, concrete classes can be created and implement the bdUnit interfaces. In order for these concrete classes to be instantiated and used within the bdUnit test, they need to be maked up with the correct attribute. This allowed me to leave out any mention of these objects within the bdUnit.dll to hold the separation of concerns that would prevent the bdUnit generated code from interfering with implementation code within the client application/project.


[Pluggable("bdUnit")]
public class User : IUser
{
public User(){}

public IUser Spouse { get; set; }
public bool IsARunner { get; set; }
public int Age { get; set; }
public bool IsDead { get; set; }
public IList<IUser> Children { get; set; }

public void Kill(IUser user)
{
user.IsDead = true;
}

public void Marry(IUser user)
{
this.Spouse = user;
user.Spouse = this;
}

public void Visit(ISleepShop sleepshop)
{
this.IsDead = true;
}

public void Total()
{
throw new System.NotImplementedException();
}
}


It appears that StructureMap does not support the instantiation of concrete classes across dll’s without using the decalritive attributes and .GetNamedInstance<>() but this may not be the case.

The tests can be run from the bdUnit.dll using a test-runner but unfortunately Resharper and Visual Studio only pick up tests within the source code of the application, and not those within a reference. It would be great to find a workaround for this to get the tests to run within the VS IDE.

The real application of bdUnit (which only become clear very recently!) is to use ‘Business Readable’ BDD user stories to provide the developer with a working technical specification or contract to be fulfilled by their implementation code without the former interfering with the latter If some of the bdUnit tests fail, the user stories can be modified and the interfaces/unit tests regenerated or the developer can make modifications to their code.

Next Steps


  1. Developing the MGrammar. I need to allow for more complex relationships between objects, nesting methods within methods (e.g. When @Person(John) #Purchase @House(number34) (John) should #Pay (number34) ~Price and (John) should #Pay (number34) ~StampDuty) and support types such as DateTime, Guid etc.
  2. Code Generation. This presently only supports analogues of Assert.IsTrue(clause) with comparisons being made in the clause. It would be good to make use of some of the wide variety of Asserts available.
  3. Improve Error Feedback. This is mostly nonexistent at the moment and I definitely need to provide useful exceptions at the parsing an compiling steps.

James Lynch

del.icio.us Tags: ,,,