No soup for me

Get latest buildable code

One of the headaches with getting latest version from source control is of course breaking the state of your local build. The unfortunate fact is that often in development teams, developers checking in code end up breaking the build, either by multiple developers checking in incompatible code or by not checking in all files necessary. I do understand it, of course. It has happened to me too, and it probably will happen again. After all, it works on my machine, right?

There are numerous ways to prevent this, however. At our R&D, we are ~20 devs using Team Foundation Server and CruiseControl.NET for continuous integration, meaning every time anyone checks in anything, a build is triggered on one of the CI servers (each team/branch has own). CCtray, a tiny system tray app lets us know the state of the build for our branch, and unless the icon is green, it is either building (orange) or failed (red) and that is usually a bad time for “Get Latest”. This is no news for many, but I felt I needed to describe it nevertheless.

Despite this safety net, the reality is that CI server build is often broken or building with uncertain outcome. Getting latest with build server in those states can result in broken code so one has to look out for green light from CCtray before getting latest, both tasks easily forgotten or delayed. This annoyed me so I decided to write (I boldly assumed there wasn’t one just like it written already) a small console app that automates this process:

• connect to a CruiseControl.NET server via remoting and get status for specified project
• if project has built ok, perform get latest from TFS, if building or broken, do nothing

Values that need to be specified are server name, project name and local working folder.

While writing this post, another issue came up: Working in offline mode. This turned out to be somewhat tedious in terms of editing files in disconnected mode from TFS and letting TFS know to pend those changes next time you come online. Best way we know about was TFS power tool command “tfpt online”, but running that command from root node of one of our branches was often equivalent of pulling the plug out of the wall. As we’re talking in the vicinity of 30.000 files in the branch, the load just got too heavy. To prevent this, one would have to keep track of the folders where changes were being made and execute the “online” command specifying only those folders, but that was bound to result in something being left out as it demands strict manual control on every part.

Command tool utility to the rescue again, and since I had one already, I thought the best way was to combine it all in a same app. So I added another switch that says to the app to iterate the specified working folder for all project files and parse them to retrieve all source code files in them. Folders that contain writable files (meaning something is up) get logged and “TFPT online” command is run only on those folders, so to prevent the server getting blasted by the workload of executing on the root level.

So, now I ended up in a two-fold purpose built command-line tool that anyone is welcome to get. It has a number of switches, but they basically group in two operation types:

1. Integration with CruiseControl.NET
2. Automating “tfpt online” command from TFS power tools

usage:
tfssync [[/s:servername] [/p:projectname] [/cc]] [/online [/filemask] [/preview]] [/wf:workingfolder] [/uu] [/log:logfile] [/verbose]

/s        CCnet Server name
/p        CCnet project name
/wf       Your local working folder
/uu       Perform TFPT undo of unchanged files before get latest
/log      Log output to file
/cc       Check status of CruiseControl.NET without taking further action
/online   Search for writable project files and perform TFPT ONLINE on containing folders
/filemask Filemask for searching for writable files (semicolon delimited, example: "*.cs;*.csproj;*.sln"
/preview  Adds /preview switch to TFPT ONLINE (no action performed)
/verbose  Outputs more detailed information

example (CCnet): tfssync /s:172.120.120.120 /p:"Build team 1" /wf:c:\dev\code /log:output.log
example (TFPT ONLINE): tfssync /wf:c:\dev\code /filemask:*.cs;*.csproj;*.sln /preview

TFS Command line integration tool

PS. Realize there are a few prerequisites:

1. Integration with CruiseControl.NET will require DLLs from ThoughtWorks (ThoughtWorks.CruiseControl.Remote.dll, ThoughtWorks.CruiseControl.CCTrayLib.dll). If you have CCtray installed, they are already in the \Program Files\CCTray folder.
2. Running the TFPT commands obviously requires TFS power tools (available free at MSDN)

del.icio.us

Posted in Programming, C#, .NET | 2 Comments »

Optimizing the Hashtable

We all love hash tables. They are the efficient cousin of arrays and are used in countless ways to quickly store and retrieve key/value pairs. However, a recent post from Jeff Atwood got me thinking and I realized that there was an aspect that maybe wasn’t as clarified as it should be to make their use even more optimized. Scott Mitchell covered this years ago, but I think it is time to recap.

The Hashtable class contains a private member variable called loadFactor that specifies the maximum ratio of items in the Hashtable to the total number of slots. A loadFactor of, say, 0.5, indicates that at most the Hashtable can only have half of its slots filled with items. That is, the other half must remain empty.

In an overloaded form of the Hashtable’s constructor, you can specify a loadFactor value between 0.1 and 1.0. Realize, however, that whatever value you provide, it is scaled down 72 percent, so even if you pass in a value of 1.0, the Hashtable class’s actual loadFactor will be 0.72. The 0.72 was found by Microsoft to be the optimal load factor, so consider using the default 1.0 load factor value (which gets scaled automatically to 0.72). Therefore, you would be encouraged to use the default of 1.0 (which is really 0.72, confusing, I know).

This might be news for some and it’s nice to be aware of it, but the point is particularly important in cases where one knows the amount of items to be placed in the hash table, which, frankly, does occur often enough to be taken into account.

So, when creating a hash table knowing there will be, say, 1000 items in it, specifying capacity of 1000 will mean that by the time you reach 720 items, the hash table will have to be expanded, because loadFactor is scaled down to 0.72, and that is a costly operation and should be avoided:

Finally, realize that expanding the Hashtable is not an inexpensive operation. Therefore, if you have an estimate as to how many items you’re Hashtable will end up containing, you should set the Hashtable’s initial capacity accordingly in the constructor so as to avoid unnecessary expansions.

In our example of 1000 items, avoiding expansion of the hash table would then mean setting the initial capacity to 1000/0.72 (approx. 1390).

However, there are other issues to take into account. One goal is to populate the hash table in the quickest fashion, but that will quite possibly not result in a best performing object when it comes to searching for keys and values in the table.

Supporting all this in real world proved to be a challenge, though. When trying to verify these arguments with a simple console application running all variations of the hash table creation, I was unable to get consistent results. In fact, the results would vary from run to run going in my favor and back. But it seems that lowering the loadFactor to 0.75 and setting capacity to capacity/0.72 improves response times when searching quite a bit, while costing very little in the creation process.

Rest of the article is the test code…

del.icio.us

Posted in Programming | 2 Comments »

Multiple “using” statements

Interesting discovery the other day: did you know you can have multiple “using” statements on top of each other without having to nest the curly brackets? Not often I need this, but a nice feature to save you some indenting space. So, instead of this:

using (MyRendererWrapper renderer = new MyRendererWrapper())
{
  using (SqlConnection conn = new SqlConnection())
  {
    using (SqlCommand c = new SqlCommand("doStuff", conn))
    {
      c.CommandTimeout = CommandTimeout;
      c.CommandType = CommandType.StoredProcedure;
      OpenConnection(conn);
      SqlDataReader dr = c.ExecuteReader();
      ...
    }
  }
}

…you could simplify it a bit further, like so:

using (MyRendererWrapper renderer = new MyRendererWrapper())
using (SqlConnection conn = new SqlConnection())
using (SqlCommand c = new SqlCommand("doStuff", conn))
{
  c.CommandTimeout = CommandTimeout;
  c.CommandType = CommandType.StoredProcedure;
  OpenConnection(conn);
  SqlDataReader dr = c.ExecuteReader();
  ...
}

It does look a bit more appealing, don’t you think?

del.icio.us

Posted in Programming | 11 Comments »

Is .NET getting an attitude problem?

I was recently doing some transaction programming, when I noticed the following property on the ServiceConfig class:

ServiceConfig.BringYourOwnTransaction

Although realizing I misinterpreted the tonality in the name, I could not help but laughing at the prospect of a name with an attitude (as in “don’t expect me to handle any transactions here, buddy. Bring your own!”).

Lets hope none of these are anywhere on the .NET roadmap:

GC.CleanYourOwnDamnMess();

System.Exceptions.StupidFunctionException();

SmtpClient.WriteLetterInstead();

del.icio.us

Posted in Programming | 4 Comments »

Unit testing and mocking

As many will agree, unit testing is greatly underrated. I am not going to go on a preaching mission for unit testing here, but the fact is that too few are using this magnificent tool for predictable and error-avoiding programming. Some seem to believe that unit and integration testing is a thing for big, complex systems, which of course is nonsense. You can have an app consisting of a single form, single table and a few functions and still find good use for some unit testing. Not to mention test-driven development which, as the name states, has tests as a pivotal part of it all. Or, as Karl Seguin beautifully put it:

Unit testing is paramount in order to achieve our goal of highly maintainable code. Unit tests empower developers with an unbelievable amount of confidence. The amount of refactoring and feature changes you’re able/willing to make when you have safety net of hundreds or thousands of automated tests that validate you haven’t broken anything is unbelievable.

Sidenote: If you haven’t read Karl Seguins “Foundations of Programming” article series, do not wait another minute.

So if you are reading this and either don’t know much about unit testing or haven’t used it so far, I strongly suggest googling, reading and last but certainly not least start using it.

Back on the subject, testing something in a class soon gets complicated. A class can derive from another class, execute static calls from other objects or otherwise depend on other classes, services or what have you. To give a basic example, testing a class called Order and its UpdateOrder() method could look like so:

[TestMethod]
public void UpdateOrderTest()
{
	// Create order item and set some properties
	OrderItem item = new OrderItem();
	item.ProductID = 1;
	item.Quantity = 1;

	// Create order and assign order item
	Order order = new Order();
	order.Items.Add(item);

	// Save order
	order.UpdateOrder();

	// Finally, if everything ok, total should be set and test passes
	Assert.IsTrue(order.Total>0);
}

As it happens, changing anything in an order automatically triggers inventory check performed on the product database by a designated class, which really is out of your testing scope. One way of isolating Order class in this example from its dependencies is known as type mocking. It is basically a way to intercept any object creation or function calls that we don’t define within our testing scope and deciding what to return instead. So, if somewhere within our UpdateOrder() method, there was a static call to Inventory.CheckItems(order.Items) that we knew would cause trouble if we didn’t know exactly what order items to test with, we could simply tell the test class to simulate this call, and for calls returning values, we could simulate those as well.

Before I go any further, there are two types of mocking techniques: reflective and natural. TypeMock.NET defines those like this:

- Reflective: Mocking each type separately and defining the behavior using string-based (reflective) expectations
- Natural TypeMocks are strongly typed and thus support Compile Time Checks, chained calls and Refactoring.

Obviously, if you have the ability (read: TypeMock.NET Professional Edition) use natural mocks whenever possible, which is what I’m going to do here as well.

So, back to our Order example, if we want to write a test that foresees and mocks the Inventory object and its call to CheckItems(), we could modify the test above to look something like this:

[TestMethod]
public void UpdateOrderTest()
{
	// Create order item and set some properties
	OrderItem item = new OrderItem();
	item.ProductID = 1;
	item.Quantity = 1;

	// Create order and assign order item
	Order order = new Order();
	order.Items.Add(item);

	// Mock the call that would mess up our test
	using (RecordExpectations recorder = RecorderManager.StartRecording())
	{
		// This is the call to mock
		// parameter just needs to match the type, content is irrelevant
		Inventory.CheckItems((new Order()).Items);
		// Since the method is bool, always return true
		recorder.Return(true).RepeatAlways();
	}

	// Save order
	order.UpdateOrder();

	// Finally, if everything ok, total should be set and test passes
	Assert.IsTrue(order.Total>0);
}

That’s it. Of course, there would probably be some mocking of the database access layer too before this went well, but hopefully you get the idea. Another thing, possibly obvious to some, that needs to be said is that tests should in general be many and smallest possible and test small chunks of your code each. That being said, something like updating order is typically complex enough procedure that it would be a better idea to write several tests that each test a single sub-process of the UpdateOrder process.

If you came this far in the article, you deserve some pointers as to what tools you might want to check out for helping you with this. Visual Studio 2005 has a decent support for unit testing, although some might disagree, and if you’re new to the concept it will be enough to start with. But there are several frameworks out there that can be helpful and for mocking in particular you would need a 3rd party to get started.

Here are some resources for testing and mocking (.NET being the platform, of course):
- NUnit
- TestDriven.NET
- NCover
- TypeMock.NET
- RhinoMocks

del.icio.us

Posted in Programming | 2 Comments »