Tuesday, March 2nd, 2010 at 15:02
Quick Tips, Testing, .NET, Bugs, C#, Events, TDD, Unit Testing
I’m currently trying to learn how to do Test Driven Development/Design (TDD) by slowly working through the techniques whilst working on a couple of new applications. Aside from the general difficulties I’ve experienced in changing my way of thinking to fit the TDD mantra (I can’t remember how many times I’ve caught myself going to write code without having a failing test first) I think what I’ve come to realise is that I really don’t know much about testing! This is probably down to not ever having dedicated a massive amount of time to it as unit testing always seems to be an afterthought on most of the projects I’ve ever worked on.
Anyway; a couple of days ago I came across a situation where I needed to test that an event fired after a thread had finished after being signalled by a method call. I eventually settled (after reading a couple of blog posts about the subject) on something like the following:
[TestMethod]
public void EndedEventFiresTest()
{
var exampleMonitor = new ExampleMonitor();
var finished = false;
var locker = new object();
exampleMonitor.MonitoringEnded += delegate(object sender, EventArgs e)
{
lock (locker)
{
finished = true;
Monitor.Pulse(locker);
}
};
exampleMonitor.BeginMonitoring();
exampleMonitor.EndMonitoring();
lock (locker)
{
Monitor.Wait(locker, 1000);
Assert.IsTrue(finished);
}
}
This seemed to work at first, but then I realised – after many runs of my test suite – that it would sometimes fail for no apparent reason!
After a while trying to work out the problem myself, I eventually gave up and asked over at Stack Overflow where someone very kindly pointed out that there was a race condition in my code. Since the call to EndMonitoring essentially just signals the thread that it needs to stop (as a pose to forcefully aborting it), it doesn’t actually guarentee that it will finish, by which time my test may have actually aquired the lock on the locker object, therefore blocking the event handler.
The resolution was to change the test to use a ManualResetEvent instance as follows:
[TestMethod]
public void EndedEventFiresTest()
{
var exampleMonitor = ExampleMonitor();
var eventSignal = new ManualResetEvent(false);
exampleMonitor.MonitoringEnded += (sender, e) => eventSignal.Set();
exampleMonitor.BeginMonitoring();
exampleMonitor.EndMonitoring();
Assert.IsTrue(eventSignal.WaitOne(1000));
}
This means the test can wait for the pre-determined amount of time (1 second in this instance) for the event to fire, if it doesn’t then the test fails, if it does, it passes. Aside from the minor detail of actually being a consistent test, the code is also a lot cleaner as well, so it’s thumbs up all round.
Saturday, February 27th, 2010 at 12:25
Working Environment, Monitors, Office
After the silly amount of overtime I’ve been doing at work lately to hit our looming project deadline, I decided it was time to treat myself with a new monitor setup at home. It took a long time to pick the right kit, but in the end I went for:
The result is amazing considering I was previously using a single 20″ Samsung display balanced on some books to raise it to a level where I didn’t get neck strain. I use a dual 4:3 19″ LCD setup at work and I missed having the ability to run Visual Studio on my main screen, and then all the supporting windows (Output, Solution Explorer, Test Results, Pending Changes etc) on the other at home. This is not certainly not an issue with this setup.
For desktop work the setup works excellently, but to really exploit the extra real-estate in games I’m going to need to upgrade my current Geforce 9800GTX+ main card for something like one of the new ATI Eyefinity line-up. All in good time.
Update: While the setup is still excellent, Aero is struggling slightly on the monitor output via the PCI graphics card. It’s only a slight issue mind you, the third screen just isn’t quiet as smooth as the other two. Once ATI sort out the DisplayPort to DVI fiasco I’ll upgrade to a single Radeon 8750.
Saturday, February 20th, 2010 at 13:07
Quick Tips, .NET, C#, InfoPath, Performance
The project I’m currently developing for at work uses InfoPath in a slightly unorthodox way. The system is a workflow solution which takes paper forms submitted by customers and then scans and processes those forms depending on the associated business process.
The forms are interpreted are converted to an XML document which matches the format of an associated InfoPath form template. InfoPath is then used to “validate” this incoming data by – up until very recently – spawning an instance of the application per form on the server where it was being processed (cringworthy, I’ll admit). The return code from the process (which happens to be the number of validation errors) is then used to detirmine if the form data is valid according to a set of relavitely complex business rules defined in the forms.
There were a few problems with this approach:
- InfoPath is really designed to be used as a data entry tool, not a business rules engine.
- Launching InfoPath processes on the server per form when there can be multiple forms which require validation per case is downright slow, taking anywhere between 10 and 30 seconds at a time per form.
- Having a direct dependancy to the InfoPath client server-side means it must be present and licenced on each of the servers in the server farm.
Obviously this was less than ideal, and more importantly, very slow. Unfortunetly this was existing architecture which was in place right at the start of the project, so only recently were we able to actually revisit it and look for a better solution.
The obvious fix was to take InfoPath out of the loop when validating the incoming data. In order to do this we implemented a very simple rules engine which implements the same rules that were defined on the InfoPath forms themselves. So when the cases pass through the process flow we can perform the same validation without having to spawn an InfoPath process each time.
The result is that cases are now processed far quicker than before, taking only a few milliseconds at most. There are some problems however:
When a form fails validation it is sent to a team of users who assess the data and attempt to correct the problem, which can be anything from a scanning error to a genuine mistake made by the customer. The process for doing this is to open the form – using InfoPath Form Services, which allows the form to be opened in a SharePoint page – correct the errors and then send the data on it’s way. The issue with this is that we needed to hook up the same business rules implemented in the engine we wrote for non-InfoPath validation to the InfoPath form itself. This was actually fairly simple because InfoPath has a concept of code-behind, which allowed us to hook into the validating events of all the fields and then fire off the appropriate validation call for that particular field as below:
private void InternalStartup()
{
this.EventManager.XmlEvents["/my:myFields/my:myExampleField"].Validating += new XmlValidatingEventHandler(this.myExampleField_Validating);
}
private void myExampleField_Validating(object sender, XmlValidatingEventArgs e)
{
// Call rules engine to check the field.
}
The problem is that – for reasons explained below – we also maintained the client-side validation rules (i.e. rules implemented directly in the form itself), so we essentially end up with two sets of validation going on.
The reason we can’t currently just remove all of the client-side validation and use only the back-end rules engine instead, is that InfoPath Form Services renders the form using a relatively complex amount of JavaScript to implement validation rules without having to actually perform a post-back after each field is changed. So as the user is correcting values they get visual feedback to say that the errors have been corrected. Conversely, we can’t not hook into the rules engine just in case something is implemented there, but not implemented in the InfoPath form itself. Otherwise the user would be told there was an error on the form but not actually told what that error was.
While we had this concept of essentially double-validation occuring, we also have a secondary issue. When the user opens the form a list of current errors is displayed. Along side this is a button which allows them to refresh this list to check they have successfully resolved all issues. This list is actually built in the code-behind for the form by itterating through the errors on the form and displaying the detail. The issue with this is that InfoPath only calls the associated validation event handler from the code-behind for the field being changed, which, while logical, actually causes a big problem when you have a complex form which has multiple dependancies between fields (i.e. field X can only be populated if field Y is etc). The result is that other fields may have been invalidated by another one changing. In order to work around this we decided to hook into the button used to refresh the error list and manually validate the entire form.
As with WinForms, each InfoPath form code-behind is generated as a partial class. The editable portion of this class is where the event handlers are attached and implemented, and the designer portion is auto-generated by InfoPath and contains the EventManager instance which is initialised below:
[global::System.Diagnostics.DebuggerNonUserCodeAttribute()]
[global::System.ComponentModel.EditorBrowsableAttribute(System.ComponentModel.EditorBrowsableState.Never)]
protected override void OnStartup()
{
base.OnStartup();
Application = ((Microsoft.Office.InfoPath.Application)(this.GetHostObject("Microsoft.Office.InfoPath.Application", "Application")));
EventManager = ((Microsoft.Office.InfoPath.EventManager)(this.GetHostObject("Microsoft.Office.InfoPath.EventManager", "EventManager")));
}
This class holds all the event handler registrations and is where I would normally expect to find an OnValidate-style protected method which would allow developers to manually invoke validation. The problem is, it doesn’t exist! In fact it appears as if the entire InfoPath namespace is relatively limited and not very developer-friendly. I imagine this is largely down to the fact that InfoPath really was never designed to be doing this sort of thing.
In order to get round this, we ended up implementing our own EventManager, XmlEvents and XmlEvent implementations which wrap the existing classes and expose an OnValidate method.
The EventManager source is as follows and is a basic wrapper which substitutes the default XmlEvents instance (essentially just a collection of XmlEvent instances) with our own:
using System.Collections.ObjectModel;
using Microsoft.Office.InfoPath;
public class ValidatingEventManager : EventManager
{
private EventManager eventManager;
private ValidatingXmlEvents xmlEvents;
public ValidatingEventManager(EventManager eventManager)
{
this.eventManager = eventManager;
this.xmlEvents = new ValidatingXmlEvents(eventManager.XmlEvents);
}
public override ControlEvents ControlEvents
{
get
{
return this.eventManager.ControlEvents;
}
}
public override FormEvents FormEvents
{
get
{
return this.eventManager.FormEvents;
}
}
public override XmlEvents XmlEvents
{
get
{
return this.xmlEvents;
}
}
}
The XmlEvents is then substituted with our own version which hooks into the creation of XmlEvent instances and records them allowing us to retrieve them when needed. This wrapper also provides our ValidateAll method which calls all registered event handlers:
using System.Collections.ObjectModel;
using Microsoft.Office.InfoPath;
public class ValidatingXmlEvents : XmlEvents
{
private XmlEvents xmlEvents;
private Collection<ValidatingXmlEvent> validatingXmlEvents = new Collection<ValidatingXmlEvent>();
public ValidatingXmlEvents(XmlEvents xmlEvents)
{
this.xmlEvents = xmlEvents;
}
public override XmlEvent this[string xpath, string dataSourceName]
{
get
{
var validatingXmlEvent = new ValidatingXmlEvent(this.xmlEvents[xpath, dataSourceName]);
this.validatingXmlEvents.Add(validatingXmlEvent);
return validatingXmlEvent;
}
}
public override XmlEvent this[string xpath]
{
get
{
var validatingXmlEvent = new ValidatingXmlEvent(this.xmlEvents[xpath]);
this.validatingXmlEvents.Add(validatingXmlEvent);
return validatingXmlEvent;
}
}
public void ValidateAll()
{
foreach (ValidatingXmlEvent validatingXmlEvent in this.validatingXmlEvents)
{
validatingXmlEvent.OnValidate(this, null);
}
}
}
Finally, the XmlEvent instance(s) are replaced so we can provide our own OnValidate method to be called from the custom XmlEvents collection above:
using System.Collections.ObjectModel;
using Microsoft.Office.InfoPath;
public class ValidatingXmlEvent : XmlEvent
{
private XmlEvent xmlEvent;
private Collection<XmlValidatingEventHandler> validatingEventHandlers = new Collection<XmlValidatingEventHandler>();
public ValidatingXmlEvent(XmlEvent xmlEvent)
{
this.xmlEvent = xmlEvent;
}
public override event XmlValidatingEventHandler Validating
{
add
{
this.xmlEvent.Validating += value;
this.validatingEventHandlers.Add(value);
}
remove
{
this.xmlEvent.Validating -= value;
this.validatingEventHandlers.Remove(value);
}
}
public override event XmlChangedEventHandler Changed
{
add
{
this.xmlEvent.Changed += value;
}
remove
{
this.xmlEvent.Changed -= value;
}
}
public override event XmlChangingEventHandler Changing
{
add
{
this.xmlEvent.Changing += value;
}
remove
{
this.xmlEvent.Changing -= value;
}
}
public override bool RaiseUndoRedoForChanged
{
get
{
return this.xmlEvent.RaiseUndoRedoForChanged;
}
set
{
this.xmlEvent.RaiseUndoRedoForChanged = value;
}
}
internal void OnValidate(object sender, XmlValidatingEventArgs e)
{
foreach (XmlValidatingEventHandler validatingEventHandler in this.validatingEventHandlers)
{
validatingEventHandler(sender, e);
}
}
}
Now all we had to do was replace the EventManager instance from the designer with our wrapped version as follows:
private void InternalStartup()
{
this.EventManager = new ValidatingEventManager(this.EventManager);
}
Seems like a lot of work just to manually invoke validation, but it works. We can now call the ValidateAll method on the ValidatingXmlEvents instance and completely revalidate the form when the refresh errors button is clicked.
Wednesday, November 25th, 2009 at 19:41
Reverse Engineering, BPM, File Formats, MFC, PODF
Note: For reference, part 1 of this article can be found here.
I already touched upon the fact I’d discovered this particuar file format I was trying to decipher used Persisted Object Data Format as it’s internal object structure, but before I get into the nitty gritty of my .NET parser implementation, I’m going to try and explain a couple of the fundamentals regarding this format.
Compression/Optimization
PODF files use a form of compression/optimization to avoid replicating class structures and other details more than once within any one instance of a file. This is achieved through the use of “class tags” to indicate whether a particular type has already been seen before, if it has, a reference tag is inserted which is essentially a pointer into a list of existing types.
As well as avoiding class structure replication, the PODF format also avoids duplication of object references by using pointers to existing object instances if they have already been seen before in the structure (i.e. they are also referenced somewhere else). All these class definitions and object references come together to form the central structure of a PODF file, what I’ve come to refer to as the “Class Item Look-up Table”.
Class Item Look-up Table
The Class Item Look-up Table (CILT) is essentially a sequential list of items (class definitions and object references) which go together to make up the content of the file being stored/read. As the file is parsed, items are added to the table so they can be referenced at later points if needed, then when they are required again an index is supplied to the given position back into the table to allow the parser to grab the item. New class definitions are marked with a special tag, and instances of new objects are represented merely as indexes into the CILT to their respective class definition.
The read sequence may go something like this:
- New class definition TypeA: Add class definition to CILT at position 1.
- New instance of TypeA: References class definition at position 1. Add instance reference at position 2.
- New class definition of TypeB: Add class definition at position 3.
- New instance of TypeB: References class definition at position 3. Add instance reference at position 4.
- Reference to TypeA instance at position 2: Nothing added to CILT as item already exists.
Note: New class definitions are always followed by an instance object of that type.
Fully understanding the Class Item Look-up Table was the key to understanding how to deserialize my target file. The only issue after this point was to understand the structure of each individual type being stored. This was the most time consuming part of the work as it involved many test cases and a lot of trial and error to produce something which was relatively stable.
The task from this point onwards was to come up with a solution for parsing the PODF structure into something useful (i.e. a .NET object). This will form the basis for the third and final part of this article.
Thursday, November 12th, 2009 at 22:14
Web Development, .NET, ASP.NET, ASP.NET MVC, Design Patterns
After a couple of well deserved days off work this week, I’ve been looking into the new(ish) alternative to WebForms for ASP.NET, ASP.NET MVC:
Just to give some brief background; ASP.NET MVC sits at the same layer that WebForms do currently, and does not replace ASP.NET, but rather offers an alternative to the traditional ASP.NET development that we’re all used to. There are some excellent overview articles and a other information at the the ASP.NET MVC site. I’m not going to go into any real detail regarding the technology here, but suffice to say that essentially the aim of ASP.NET MVC is to bring control back to the web developer whilst enforcing proper segregation of model, view and controller aspects.
Note: From this point I’m going to refer to ASP.NET MVC as just plain old MVC. I realise MVC is actually just a design pattern, but for the sake of my fingers please excuse me. I’d also like to say that while I have a fairly high level of expeirence developing WebForms applications, but I haven’t actually tried MVC yet. So some of my views may well be unfounded.
The more I read about the differences between the two approaches to ASP.NET development, the more I’m conflicted. On the one hand I love the concept of MVC for things like:
- Simplicity. It’s nice to have a new platform that provides so much control back to the developer. I also love the thought of getting rid of all the crap that WebForms generates which messes up my anally beautifully crafted mark-up.
- Segregation. No more business logic hiding in code-behind files, although it’s worth noting that there’s nothing to stop you breaking the convention to do this.
However; on the other hand I feel like MVC has the following issues:
- Control reuse. As mark-up is generated in a far more “manual” fashion, you appear to loose the ability to drag-and-drop controls to pages. This is addressed somewhat by what Microsoft refer to as “HTML helper methods” (essentially just extensions of the static Html class), but I’m still unconvinced.
- Spaghetti code. Because MVC moves away from the code-behind model of WebForms (although again, doesn’t restrict it), your “views” end up being full of in-line code which – in my opinion – just looks horrific. This actually is the biggie for me.
I think the reason I struggle so much, is that – coming from a commercial development background – I can’t see the benefit to switching away from WebForms to use MVC as it stands currently. Largely the feeling I get with development tasks in a commercial setting – especially those which aren’t customer facing – is that it’s generally more important that web applications do the right thing than that they look right. So some of the identified issues with WebForms in relation to bad mark-up, or display issues etc don’t really feel prevalent enough to warrant the additional time it would require to develop something using MVC. Yes; I realise you gain more control, and get closer to what purists would claim was “pure” web development (more akin to PHP, Ruby etc), but I’m just not convinced that this is enough to warrent the additional development time.
I guess the important point to note however; is that WebForms isn’t being replaced by MVC. Instead Microsoft are now offering an alternative to what we’re more used to while still exposing the wide and rich ASP.NET framework libraries. I’m also still very keen to give it a go, and there are still some things which I think look really interesting. I just wish there was a better alternative to using in-line code in the views!
Edit: After reading a very informative blog post, it looks like there may well be light at the end of the code-spaghetti filled tunnel for MVC in the form of Spark.
