NUnitForms : ExpectModal and MessageBoxTester
April 10th, 2005I usually don't write many tests for GUI and try to focus on the underlying code (model / services) instead, but sometimes it's really helpful. Here I've got a Form which asks for confirmation to the end-user, using a MessageBox.Show call:
/// <summary> /// The public name of the confirmation dialog, /// used for the tests /// </summary> public const string ConfirmationDialogName = "Exit?"; private void MainForm_Closing( object sender, System.ComponentModel.CancelEventArgs e) { // ask for confirmation (yes/no) DialogResult result = MessageBox.Show( " Do you really want to leave ?", ConfirmationDialogName, MessageBoxButtons.YesNo, MessageBoxIcon.Question); // cancel if no if (result==DialogResult.No) e.Cancel=true; }
I've extracted the MessageBox name to be able to tell NUnitForms how to recognize the modal window which will open.
Now to the tests : here I inherit from NUnitFormsTest which provides some useful services such as an "ExpectModal" method, and I setup two tests:
* one to check that answering Yes will really close the form,
* another one to check that answering No will leave the form opened
The two tests call ExpectModal(string modalDialogName,string modalHandler) to specify which modal dialog is expected (here our confirmation box) and to specify a handler which will respond to this modal dialog. Those "expectations" are automatically verified (through a method called Verify) by the NUnitFormsTest class, at Teardown.
When invoked, our ConfirmModal method instanciates a MessageBoxTester with the expected message box name and triggers a command (Yes or No) accordingly:
using System; using Library; using NUnit.Framework; using NUnit.Extensions.Forms; namespace Tests { public class Tests : NUnitFormTest { MainForm form; public override void Setup() { base.Setup (); form = new MainForm(); form.Show(); } [Test] public void CancelExitShouldNotExit() { ExpectModal( MainForm.ConfirmationDialogName, "ConfirmModalByNo"); form.Close(); // check that the form is not closed Assert.IsFalse(form.IsDisposed); } [Test] public void ConfirmExitShouldExit() { ExpectModal( MainForm.ConfirmationDialogName, "ConfirmModalByYes"); form.Close(); // check that the form is really closed Assert.IsTrue(form.IsDisposed); } #region test tools public void ConfirmModalByNo() { ConfirmModal(MessageBoxTester.Command.No); } public void ConfirmModalByYes() { ConfirmModal(MessageBoxTester.Command.Yes); } public void ConfirmModal(MessageBoxTester.Command command) { MessageBoxTester tester = new MessageBoxTester(MainForm.ConfirmationDialogName); if (tester!=null) tester.SendCommand(command); } #endregion } }
Well, guess it works now.
Beware of the following caveats though:
* don't use [SetUp] or [TearDown] with the NUnitFormsTest, instead override Setup() and Teardown() methods (don't forget to call the base implementation).
* if the MessageTester cannot find the expected modal dialog, an unhandled exception will be thrown (and in this case, the test will never exit... ). Using a try/catch block with a Assert.Fail under the catch statement will not work either (the dialog will remain open). I extracted the name of the dialog to minimize the risk to meet this situation (there's probably a better way to handle errors here).
* i18n : here the name of the messagebox is a hardcoded string : in real life it should be a property able to retrieve the name based on culture instead.
Versioning, probing and binding in .Net
December 22nd, 2004Richard Grimes released a very nice tutorial about how .NET locates and loads assemblies.
It's written in a clear style, and provides practical and valuable information with code snippets on the following subjects : assemblies, metadata, fusion / fuslogvw, versioning (including versioning without a strong name), rebinding/redirection, shared assemblies, publisher policies, dynamic loading, etc.
A must-read (did I mention it's free?). Thanks Richard!
System.InvalidProgram Exception
December 21st, 2004Did you ever get this exception ? I had never met it until a few days ago.
I started getting it all the time on a precise bit of managed c++ code, when invoking delegates with delegate->Invoke() calls.
As a starter I had a look at the documentation, which states that System.InvalidProgramException is "the exception that is thrown when a program contains invalid Microsoft intermediate language (MSIL) or metadata. Generally this indicates a bug in a compiler".
I almost immediately tried to run the guilty code on other machines, which proved useful : while the exception was always thrown on my machine, it was never thrown on others (although supporting the same version of .Net).
I googled a bit but could not find many things apart from generated documentation, and a post regarding JIT by Justin Rogers (http://weblogs.asp.net/justin_rogers/archive/2004/03/21/93608.aspx).
I then suspected CASPOL and the framework security, double-checked my enterprise/machine/user configurations, compared it to others, without more luck : even with similar configurations, the behavior remained the same. I recompiled (debug/release) the solution a couple of times, but it didn't change.
Then I started thinking JIT again, code instrumentation and what could affect my code in some way.
I noticed that I had installed a test coverage tool (CoverageEye) a few days before, and had never run the guilty code since then. I uninstalled it, and the code started working again.
What was happening is that this tool instruments the code at runtime with generated MSIL, which in that case was invalid, hence the exception.
Don't get me wrong : I'm not blaming this specific tool at all - I reported the issue to the author and will take time to reproduce it. I'm aware it could have happened with many other tools such as profilers, coverage tools, or even AOP implementations ;)
I found myself rather lucky to spot this one! Taking a break (breath, air, etc) helped me a lot to change of "scope". We should have breaks more often.
NAnt 0.85-RC1, TestDriven.Net 1.0 final
November 30th, 2004Quick one today :
* NAnt 0.85-rc1 is released, see full release notes here. Loads of stuff added (expressions, filterchain, better solution task support etc).
* Spread the word : TestDriven.Net 1.0 final is out, along with a new website! The best add-in in town to ease unit testing in VisualStudio.Net. The only cost today is an online registration with a short survey! Definitely a must-have, once again.
NUnitForms and TestDriven.Net
November 25th, 2004Yesterday I started adding test to a windows form application. Until now I only tested non graphic libraries, but was concerned about how to lock the behavior of a graphic application. Graphic applications (both web and non-web) are usually very likely to regress from a functional point of view, and seem to generate a lot of support ("Didn't you fix that already ?"), maybe because an inspired gui tester has that kind of black magic skills to find new, undiscovered ways to use your application ("This *cannot* happen!! How did you do that?")
Anyway. I started small - a few tests, with a small application.
For regular (non graphic) tests, I used classic NUnit tests. And when I started testing forms, I checked out NUnitForms, which is an "NUnit extension for unit and acceptance testing of Windows Forms applications".
Setting up
For a practical point of view, when testing libraries I usually create a separate test project, and most of the time unit test the public parts of the tested assemblies which are located in other projects in the solution.
But this time, at least under VS.Net 2003 this won't work : I can't reference an "application project" assembly from another project (although it is allowed by the c# compiler itself, the IDE is not letting us do it).
So I decided to put the tests inside the application, in a separate Tests sub-namespace. This way I can cope with the IDE limitation, plus avoid the visibility issues.
There's plenty of other ways to do that of course (and debates around it as well), but that's not the purpose of this post.
To finish the setup, I just added nunit.framework.dll and NUnitForms.dll to the application assembly references.
Writing a test
Apart from usual NUnit test I wrote for the same application, I wanted to start testing a form. This form is called ErrorReportingForm, has a LinkLabel called _close which should manage to close the form, and an ErrorText property which value should be dumped into a TextBox called _errorMessage, along with other informations (including the list of assemblies in current appdomain).
Here's the test code first :
using System; using NUnit.Framework; using NUnit.Extensions.Forms; using DotNetGuru.Examples.Forms; namespace DotNetGuru.Examples.Forms.Tests { [TestFixture] public class ErrorReportingFormTest { bool hasClosed=false; private void form_Closed(object sender,EventArgs args) { hasClosed=true; } [Test] public void CloseForm() { ErrorReportingForm form = new ErrorReportingForm(); form.Closed+=new EventHandler(form_Closed); form.Show(); LinkLabelTester ctrl = new LinkLabelTester("_close",form); ctrl.Click(); Assert.IsTrue(hasClosed); } [Test] public void TextBoxShouldContainMessage() { ErrorReportingForm form = new ErrorReportingForm(); form.ErrorText="This is the error I want to display"; form.Show(); TextBoxTester ctrl = new TextBoxTester("_errorMessage",form); Assert.IsTrue(ctrl.Text.IndexOf("This is the error I want to display")!=-1); } } }
First thing is there's no breaking change with NUnit, only extensions. I have to import the NUnit.Extensions.Forms namespace in addition to NUnit.Framework. Then I declare my TestFixture and Test as usual.
I wanted to ensure the close button really close the windows. What's the need for that ? I just want to detect if no exception is thrown, for instance.
With NUnitForms I just instanciate the form. To be informed that the form has closed I hook up the Closed event. Then I call Show(), which causes NUnitForms to open the form in a separate, hidden desktop.
Then I use the NUnitForms tester objects to simulate actions on the control named "_close", then check that my callback has been called.
NUnitForms (now 1.3.1) provides support for a part of the Windows Forms (including "Buttons, CheckBoxes, ComboBoxes, Labels, ListBoxes, RadioButtons, TabControls, TextBoxes, TreeViews, Context Menus, Forms, MenuItems, Modal Forms, Modal MessageBoxes, and the Mouse", the list is growing), and still leave you with the opportunity to use a generic ControlTester class if the control you want to test is not yet supported (or not at all).
I should also mention that it comes bundled with a Recorder application with confused me a bit first (you have to select which form you want to test in the dropdown list), but which is handy once you have it working: it let you interact with the form and record what's happening (in term of real, typed events, not (x;y) click coordinates like other test recording tools). You just have to copy and paste the generated code into your test project to start testing.
Running and debugging tests with TestDriven.Net
For those who don't know it yet, TestDriven.Net is a VS.NET add-in to allow running and debugging unit tests in a very comfortable way. The latest release can be found here.
The good news is : it works perfect with NUnitForms so far!
Just like any other NUnit test, I can run a NUnitForms test, debug it step-by-step, put breakpoints etc. Isn't that sweet ?
The build process
Just as seamless, my CruiseControl.Net + NAnt setup accepted the NUnitForms tests and properly executed them.
Time to leave
It is actually simpler than what I expected to *start* testing graphical user interfaces, and now the tools under .Net are getting very well integrated to achieve that.
There are a lot more questions on how to test more complex things (just like for classical nunit tests when it comes to database testing, distributed testing, nmock etc), but at least the beginning is promising and I'll explore more.
Edit (25/11/2004) : Jose Almeida has blogged about another way to setup NUnitForms which I would use for a bigger application, ie. having three projects (one bootstrap .exe, one assembly with all the app code, and one assembly with all the tests).
A read-only IDictionary wrapper for .Net 1/1.1
November 24th, 2004How to disable write operations on a given IDictionary instance (specifically, a Hashtable instance) ? Until now I couldn't find an immediate answer in .Net framework 1.0/1.1.
For some reasons using third party collections such as NCollection, which may address this feature, wasn't possible for me this time. Also, this feature seems to be discussed and adressed in .Net 2.0, but I'm targetting .Net 1.0/1.1.
I finally ended up writing a decorator à-la ArrayList.ReadOnly().
Hashtable table = new Hashtable();
table.Add("key","value");
IDictionary readOnlyTable = ReadOnlyDictionary.ReadOnly(table);
try
{
readOnlyTable.Add("anotherkey","anothervalue");
}
catch (NotSupportedException)
{
// this exception will be thrown
}
I'd love to have readers feedback on at least two points :
- on the whole approach : did anyone find a simpler or different way to achieve that ? (I'd be happy to throw away some code, as usual)
- on the wrapper code itself : it seems to be working fine, but does anyone detect some caveat here ?
The wrapper code :
using System;
using System.Collections;
namespace DotNetGuru.Collections
{
/// <summary>
/// A read-only wrapper for IDictionary. Any change to the underlying dictionary will be
/// propagated to the read-only wrapper.
///
public class ReadOnlyDictionary : IDictionary
{
#region ReadOnlyDictionary members
private IDictionary _originalDictionary;
private ReadOnlyDictionary(IDictionary original)
{
_originalDictionary = original;
}
/// <summary>
/// Return a read only wrapper to an existing dictionary.
/// Any change to the underlying dictionary will be
/// propagated to the read-only wrapper.
///
public static ReadOnlyDictionary ReadOnly(IDictionary dictionary)
{
return new ReadOnlyDictionary(dictionary);
}
private void ReportNotSupported()
{
throw new NotSupportedException("Collection is read-only.");
}
#endregion
#region IDictionary Members
public bool IsReadOnly
{
get
{
return true;
}
}
public IDictionaryEnumerator GetEnumerator()
{
return _originalDictionary.GetEnumerator();
}
public object this[object key]
{
get
{
return _originalDictionary[key];
}
set
{
ReportNotSupported();
}
}
public void Remove(object key)
{
ReportNotSupported();
}
public bool Contains(object key)
{
return _originalDictionary.Contains(key);
}
public void Clear()
{
ReportNotSupported();
}
public ICollection Values
{
get
{
// no need to wrap with a read-only thing,
// as ICollection is always read-only
return _originalDictionary.Values;
}
}
public void Add(object key, object value)
{
ReportNotSupported();
}
public ICollection Keys
{
get
{
// no need to wrap with a read-only thing,
// as ICollection is always read-only
return _originalDictionary.Keys;
}
}
public bool IsFixedSize
{
get
{
return _originalDictionary.IsFixedSize;
}
}
#endregion
#region ICollection Members
public bool IsSynchronized
{
get
{
return _originalDictionary.IsSynchronized;
}
}
public int Count
{
get
{
return _originalDictionary.Count;
}
}
public void CopyTo(Array array, int index)
{
_originalDictionary.CopyTo(array,index);
}
public object SyncRoot
{
get
{
return _originalDictionary.SyncRoot;
}
}
#endregion
#region IEnumerable Members
IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return _originalDictionary.GetEnumerator();
}
#endregion
}
}
The unit tests (threading test is not terrific; as well a good test should ensure that the underlying collection has never been modified by various operations):
using System;
using System.Threading;
using System.Collections;
using NUnit.Framework;
namespace DotNetGuru.Collections
{
/// <summary>
/// Test the read-only IDictionary wrapper.
/// </summary>
[TestFixture]
public class ReadOnlyDictionaryTest
{
Hashtable table;
ReadOnlyDictionary readOnlyTable;
[SetUp]
public void Init()
{
table = new Hashtable();
table["key"]="value";
table["key2"]="value2";
readOnlyTable = ReadOnlyDictionary.ReadOnly(table);
}
[Test, ExpectedException(typeof(NotSupportedException),"Collection is read-only.")]
public void AddShouldBeForbidden()
{
readOnlyTable.Add("test","value");
}
[Test, ExpectedException(typeof(NotSupportedException),"Collection is read-only.")]
public void AddThroughIndexerShouldBeForbidden()
{
readOnlyTable["test"]="value";
}
[Test, ExpectedException(typeof(NotSupportedException),"Collection is read-only.")]
public void ClearForbidden()
{
readOnlyTable.Clear();
}
[Test, ExpectedException(typeof(NotSupportedException),"Collection is read-only.")]
public void RemoveForbidden()
{
readOnlyTable.Remove("key");
}
[Test, ExpectedException(typeof(NotSupportedException),"Collection is read-only.")]
public void ChangeForbidden()
{
readOnlyTable["key"]="a new value";
}
[Test]
public void ShouldMirrorOriginalDictionary()
{
Assert.AreEqual(table.Count,readOnlyTable.Count);
Assert.AreEqual(table.Keys.Count,readOnlyTable.Keys.Count);
Assert.AreEqual(table.Values.Count,readOnlyTable.Values.Count);
foreach (string key in table.Keys)
Assert.AreEqual(table[key],readOnlyTable[key]);
// modify original dictionary and ensure the changes are propagated
table["newkey"]="newvalue";
Assert.AreEqual(3,readOnlyTable.Count);
Assert.AreEqual(3,readOnlyTable.Keys.Count);
Assert.AreEqual(3,readOnlyTable.Values.Count);
Assert.AreEqual("newvalue",readOnlyTable["newkey"]);
}
bool writerThreadShouldExit=false;
private void WriterThread()
{
while (!writerThreadShouldExit)
{
table["key"]="new val at "+DateTime.Now;
string key = "addedkey"+DateTime.Now;
table.Add(key,"addedvalue");
table.Remove(key);
}
}
/// <summary>
/// Threading test : a separate thread keeps writing, adding, removing objects.
/// We should be interrupted in the current thread at some point.
/// </summary>
[Test, ExpectedException(typeof(InvalidOperationException),"Collection was modified; enumeration operation may not execute.")]
public void ReadWriteConcurrent()
{
Thread thread = new Thread(new ThreadStart(WriterThread));
thread.Start();
DateTime start = DateTime.Now;
while (new TimeSpan(DateTime.Now.Ticks-start.Ticks).Seconds<3)
{
foreach (string key in readOnlyTable.Keys)
{
string val = readOnlyTable[key].ToString()+"xxx";
}
}
}
}
}
What do you think ?
Edit : thanks to Sébastien Ros for the syntax coloring tip !(http://www.manoli.net/csharpformat/)
Ignore attribute for NUnit TestFixture
November 16th, 2004The [Ignore("reason")] attribute on nunit tests is very useful to temporarily disable a test without having to remove the test itself from the code (which could lead to situations where you delete a test you didn't really want to delete, and have it lost deep in your VCM) :
[Test]
[Ignore("this test is temporarily ignored because ...")]
public void MyTest()
{
...
}
What I had never realized until today is that it's also possible to use this attribute on the whole fixture :
[TestFixture]
[Ignore("Ignoring the whole fixture")]
public class MyFixture
{
...
}
CCNet 0.7 Released
November 9th, 2004CruiseControl.Net 0.7, one of the largest releases so far, has been released.
News are :
- the new web dashboard has reporting options. Although it doesn't support everything the old webapp does, the new dashboard avoids to setup one virtual root for each ccnet project, and will soon replace the web application.
- automatic update of the source tree without using a bootstrap script (CVS, Perforce and VSS. SVN is on the way but not yet available in 0.7).
- triggers are replacing the old schedules (breaking change).
- lot of documentation updates
- multiple projects states are automatically handled (no need to specify the state file in ccnet.config anymore)
- the file merging has been removed from the xml logger, and the file merge task should be used instead
Have a careful look at the configuration documentation before upgrading.
CCNet 0.7 RC-1 is out. Feedback welcome.
November 2nd, 2004The release candidate one is out.
Most important updates are the addition of automatic retrieval of source code (ie: no need to bootstrap anymore) for some VCM, including CVS and VSS (SVN is planned), multiple integration triggers support, various fixes (including VSS french support), and more support for relatives paths. See the release notes for more information.
Please note that there are small breaking changes in scheduling syntax.
We're happily welcoming feedback on ccnet-user for any topic, and especially documentation which is under rework with the aim to simplify CCNet adoption for the newcomer.
A log4net tip
October 14th, 2004
private static readonly ILog log = LogManager.GetLogger(typeof(MyClass));
I'm writing more than once some days. Nothing difficult here, but I can save a reference to MyClass by using reflection (tip provided by Nicko Cadell) :
#region Logging Definition
private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
#endregion
Comfortable when combined with quickcode or added to a "create new class" wizard.
ASP.Net vulnerability fix : HTTP module published
October 12th, 2004Brian Goldfarb posted updated information on the reported ASP.Net vulnerability : there's now a MSI installer for an HTTP module supporting (more) secure canonicalization.
That's good news for people willing to secure their numerous applications in an automated way.
Software marketing resources
October 1st, 2004Wesner Moise posted the url of a very interesting site for those who try to market and sell software :
Software marketing resources. I'm only starting to explore it.
No need to mention that Wesner's blog (".Net Undocumented") itself is really worth looking at (oops, I've done it!).
TestDriven.Net
October 1st, 2004Jamie is doing a great job with TestDriven.Net (the new name for his NUnit add-in and companion website). Testing is sugar candy with this tool. I'm using it on a daily basis and have to admit I'm addicted.
The features I like are its robustness, its clean support for step-by-step debugging of unit tests, the possibility to pick up the runtime I want to use, and its integration with several testing frameworks (NUnit, MBUnit and the forthcoming TeamSystem framework). This leaves open the possibility to move forward.
Jamie has just set up a FlexWiki for releases, documentation, simple bug tracking and feature request.
As well, Jamie and Peli have teamed up to make MbUnit released along with the NUnit add-in. They share a common build environment based on CruiseControl.Net, MSBuild and Wix. I noticed that this combo is becoming more and more common these days.
I'm using NAnt most of the time (and contributing small bits), but I have the feeling that MSBuild will probably overcome NAnt when most of the dev teams will switch to .Net 2.0.
I'm interested in your real-life feedback about MSBuild.
Monitor multiple CCNet builds at a glance with CCTray
August 26th, 2004CruiseControl.Net offers several ways to get feedback from the continuous integration builds, including emails, web dashboard and system tray notification with an application called CCTray.
CCTray allows you to pick which project to monitor, but doesn't yet allow you to monitor several projects at a time (I'm confident this will change sooner or later).
I wrote this small NAnt script which setups and spawns multiple instances of cctray, using information found in the CCNet configuration file. This way I can be aware of the state of all builds at a glance, without checking a web page or my emails.
<?xml version="1.0" encoding="utf-8" ?>
<project xmlns="http://nant.sf.net/schemas/nant-0.85.win32.net-1.0.xsd" default="launch.trays">
<script language="C#" prefix="utils" >
<imports>
<import name="System.Xml"/>
<import name="System.Text"/>
</imports>
<code>
<![CDATA[
// return nodes values for a given xpath, separated by commas
[Function("select-nodes" )]
public static string SelectNodes(string filename,string xpath)
{
XmlDocument doc = new XmlDocument();
doc.Load(filename);
StringBuilder buffer = new StringBuilder();
foreach (XmlNode node in doc.SelectNodes(xpath))
buffer.AppendFormat("{0},",node.Value);
return buffer.ToString();
}
]]>
</code>
</script>
<target name="launch.trays">
<property name="cctray.source.folder" value="c:/integration/tools/ccnet/cctray"/>
<property name="cctray.instances.folder" value="c:/integration/cctrays"/>
<property name="ccnet.config" value="c:/integration/ccnet/ccnet.config"/>
<exec program="pskill.exe" commandline="cctray" failonerror="false"/>
<delete dir="${cctray.instances.folder}"/>
<foreach item="String" in="${utils::select-nodes(ccnet.config,'//project/@name')}" delim="," property="projectname">
<property name="ccfolder" value="${cctray.instances.folder}/${projectname}"/>
<copy todir="${ccfolder}">
<fileset basedir="${cctray.source.folder}">
<include name="*"/>
<exclude name="cctray-settings.xml"/>
</fileset>
</copy>
<copy todir="${ccfolder}" verbose="true">
<fileset basedir="${cctray.source.folder}">
<include name="cctray-settings.xml"/>
</fileset>
<filterchain>
<replacestring from="<ProjectName>MyDefaultProject</ProjectName>"
to="<ProjectName>${projectname}</ProjectName>"/>
</filterchain>
</copy>
<exec program="cmd.exe" commandline="/c start ${ccfolder}/cctray.exe"/>
</foreach>
</target>
</project>
You'll need to grab pskill from sysinternals and the latest nightly build from nant, and must launch and setup the source cctray instance properly (point to the ccnet server etc).
Though this approach obviously doesn't scale very well, as each cctray instance requires almost 20mb, it fits my needs and my configuration.
I like the new nant filter chains feature, which allows to replace tokens just like the filter task in Ant, but also arbitrary strings. It's also possible to expand nant properties with a specific filter.
I customized CCTray texts to get more precise information :
<replacestring from="Yet another successful build!" to="Another successful build for project ${projectname}!" />
<replacestring from="The build is still broken..." to="Project ${projectname} is still broken" />
<replacestring from="Recent checkins have fixed the build." to="Recent checkins have fixed project ${projectname}" />
<replacestring from="Recent checkins have broken the build." to="Project ${projectname} is now broken" />
One step further and you've got Peedy flying and speaking.
Essential XML Quick Reference available for free
August 6th, 2004One more ebook gem to download on TheServerSide.Net :
Essential XML Quick Reference
Had a quick look, seems to be a very good reference guide to XML, XPath, XSLT, XSD, DTD etc; worth the download.