One of the issues of infrequent releases.....

It's been a month since our last deployment, and we've got two features that's have been sitting in code for longer than that, and one that's been recently developed. One feature is 'hidden', the other isn't functional yet (a couple of external dependencies).

There are a couple of problems here:

• Not getting feedback on features that are ready to be released
• Keeping the information in your brain so that you can fix any bugs that arise when the feature finally gets released
• Trying to remember all the little bits that need to be completed to make the feature ready to release

A lot of this has to do with the flow of development, and perceived issues with releasing small features often. But the one that has kept on creeping up on me in the last couple of weeks is the realisation I want to keep adding new features, little ones that I think will make a difference, but aren't really necessary or proven to be required. 

This mentality comes from a "we're not going to be able to push anything new for another month", so try to pack everything in. It means that the code isn't as lean as it should be. Instead of waiting for feedback, reacting and pushing a new release, we just assume what people are after. 

Have you found this to be an issue when you're doing infrequent releases?

Raven DB Multi Map Index

I'm going to be looking at getting multiple document types returned from raven querying some properties that both objects have.

We're going to be using Animals and Vehicles, and we want to be able to figure out which animals and vehicles operate on land. These documents don't share inheritance, so we can't just search the base properties - what we need is a mapping that collects all the common properties of these documents.

The objects are:

We need to start by mapping the common fields to a new object for use in our LINQ query.
We also create a mapping for each object type that we want to index, selecting the common fields into an anonymous type.

You may have heard a lot of talk about Map/Reduce in the document database world, but for this particular query we're just interested in the Map part - we don't want to reduce the results to a common object but rather return the original objects.

Now that we've defined our Map we need to tell Raven about it which is really easy. Just after calling DocumentStore.Initialize() you call IndexCreation.CreateIndexes(typeof(AnimalsAndVehicles).Assembly, DocumentStore) which takes care of creating the index, and now it's ready to use!

To use the MultiMap: The key part of this is the As<dynamic>() - this will let us return any object from the query.

And the magic:

A couple of helpful links in putting this together and thinking about how the mapping works were the Raven Documentation and Daniel Lang's blog about Multi Map Indexes

Basic Raven DB Indexing and Querying with Lucene

We've got up and running with Raven DB for our new project, which has been a delight to use from a code perspective, but one of the big gaps for me is figuring out how to do SQL like queries. I'm not too bad at putting a SQL query together as something I do semi-regularly as part of my job, so I thought it shouldn't be too much of a stretch to start pulling some statistics.

I was wrong.

If you're using the .NET client, this is not something you particularly need to worry about as Raven will dynamically create indexes for you based on your LINQ query, which is awesome. My particular scenario is using the Raven web admin pages to do a query that I'm not particularly interested in coding - I just want to see some numbers quickly.

I'll talk you through the steps that I've gone through to get a simple query working, hopefully its of some benefit to you. To start with you should know that RavenDB search is powered by Lucene - we'll be looking at that in a bit. I'm going to start with a data set of animals :

What I'd like to see in the browser is a list of all the living land animals. What this means is we want to do a SQL equivalent of where on 2 of the fields: Lives AND Status.

To make these two columns queryable we need to make an index - unlike SQL we can't just query whatever we want whenever we want - we've got to tell Raven which fields we're going to do a query on. To do this jump over to the 'Indexes' section and 'Create an Index'. What we're going to do is create a document that maps to the columns we need to query, so give your index a name and jump into the 'Maps'.

A breakdown of the parts:

• from animals: animals is just the alias for the document we're going to refer to in the select
• in docs.SimpleIndexExample: this provides the source for the map - docs is generally all documents in the database, and SimpleIndexExample is the document type
• animals.Lives and animals.Status are the two columns we want to query on

Getting back to the point, I'm going to put together a query to get a list of all living land animals. Now that the index is created, we can click on it and use it to do a query (it'll look different depending on which version of Raven you've got running, but hopefully you'll be able to find the 'Query' section)

Creating the query is just a matter of telling Raven which field you want to query on, and value to restrict it to, so it should look something like this (by the way, you can get some auto-completion on the map field by pressing ctrl + space):

If you haven't picked it up, there's a problem with our result set - we're getting a Shark and TRex - this is not good! The problem is Lucene is looking for anything that matches the query, effectively doing an 'OR' on our statement. What we need to do is tell Lucene to require that a certain value is present - this is just putting a '+' symbol at the start of each required field:

For heaps more info about using Lucene to query have a look at the Lucene Query Parser Syntax Page

Please let me know if you have any comments, ways I can think about this better as this is still a very new area for me, or if you've found it helpful.

Task based workflow with NServiceBus and SignalR

Over the last six months our team has been working on breaking our system down into a whole lot of distinct, reusable components. Most of the work is completely automated, but there is always the case where we need some human input.

For each of these scenarios we've created very specific task based UI's, so we can taylor the screens to the exact need of the task. This has meant that we've been a lot more focused on solving a real business problem rather than just trying to jam as much information on the screen at once.

Up to this point, each task has been very distinct from the others, without too much consideration of the workflow that would really take place. There are a couple of options for how the tasks could be completed:

• Task centric users who complete just one type of task, and who don't particularly worry about the work flowing from their action
• Customer centric users who might need to call the customer to get some extra details, and are very interested in the way the rest of the work flows, as they want to be able to get all the information required to complete the whole transaction while on the phone. 

Our system currently supports task centric users, but we need a solution to customer centric solutions. For that, I had seen and heard a bit about SignalR:

Async signaling library for .NET to help build real-time, multi-user interactive web applications.

Along with listening to Hanselminutes, one of the posts that got me started was push notifications with nservicebus and signalr, which was right up my alley as we're using NServiceBus as our underlying infrastructure for messaging. The only difference in our project was using MVC3 for the web app, so I avoided the EventStream and created a class implementing SignalR's Hub - which was super easy.

I got a bit confused about how to get the Clients who are connected to my implementation of the Hub, but this stackoverflow answer from David Fowler helped me get the clients, so I could start notifying away!

Within a day I had a working notification system working! It definitely requires more refinement to make it more useful to our users, such as only notifying some people of the changes (trying to identify who 'owns' a transaction and wants to see further tasks) and some UI design to make things a bit friendlier, but it's definitely been really fund and easy to work with.

If you're working with task based UI's how do you create customer centric workflow?

Asynchronous Testing Woes

Over the last six months I've been having fun working with NServiceBus producing our new app at work. It's been great splitting our system up into functional bits (that we can hopefully reuse). Our current requirements mean we have one big saga (long running process) that uses a whole lot of handlers to get the job done.

This is awesome, as we've been able to attach a problem by talking about what the solution should be, splitting up into pairs and work on our own bit of the solution, then get back together and figure out our next bit of work, just having to define our messaging requirements collaboratively before we start.

We've also invested in writing specification tests with SpecFlow - taking a business scenario and making sure that our system can handle it from end to end. While this has been fun to learn, it's been hard work to get working due to the async nature of our system. What we've ended up doing is firing off the spec and then waiting 10 seconds until we check our auditing system to see if the saga has completed. This works fine for most of the tests, but there are often one or two specs that fail, but when you check the audits, everything has worked properly.

It would be awesome in this case to have a spec subscribe to the auditing events (currently there isn't an event published saying a saga has completed, but commands are sent to the auditor) - in that case the test could just wait until the publish, but it would also need a timeout (for when the saga actually fails).

This brings me to one problem that I've had with my specflow tests - the amount of time they take to run. These aren't like unit tests that we run all the time and only take a few seconds, the suit of tests takes minutes to run, so I'm looking to see if there are any solutions to run them in parallel. At the moment we use the NUnit library to implement the tests, but I have seen that MbUnit has a parallel library which looks interesting.

How do you do integration tests? Do you even do them?