Automated Smoke Tests

Well it appears that my foray into test automation was both timely and well received. At our retrospective this week, we looked at things we’d like to improve on as a team and test automation came out as a clear favourite.

Of particular interest was automating our smoke tests. At the moment the whole smoke test is manual and takes a couple of hours. We want to move our ‘done’ branch forward at the end of every sprint which means we need to take a milestone and test it every two weeks. This needs to be scheduled in to make sure that it’s done after our work has made it to the candidate milestone branch, but before we need to start on the next sprint. If only we could have the smoke tests run automatically after every green build!

Well, after our retrospective that has become our goal. Actually we’re not aiming to automate the whole of the smoke tests just yet, but we want to put the framework in place and perhaps get 10 or so of the web tests automated. Hopefully we’ll also make time to continue with the SpecFlow tests as well.

The myth of the one-person story

We used to be very much of the mindset that each story had one person working on it at a time. Often people would say, oh I don’t think you can get more than one person down that hole. This leads to problems.

Large stories that would take a single person more than a sprint to complete are not uncommon for us. Partly because we’re not (yet) that good at breaking stories down and I suspect partly due to the legacy code we are working with. Anyway if we only have one person working on such a story they aren’t going to get finished inside a sprint, so it’s hard to track progress which can hide big problems for too long.

One of the ways I’ve tired to help people change their mind about the one-person per story mindset is to task up stories at the beginning of a sprint like normal. Then we position the tasks to highlight parallel streams of work. So if two tasks can be done at the same time they sit on post-it notes on the same line. It’s not uncommon for there to be 3 or even 4 possible streams of development for at least part of the sprint.

Related to this is that tasks that are ‘stalling’ (taking longer than expected) can also be split once they are identified. If you identify a task that is taking longer than a day then ask what’s going on, what work is being done, what’s still left. I ask people to write their responses up as additional tasks in the same way as in the sprint planning. We can then split up these tasks amongst the rest of the team if need be and have a better chance of getting to done.

I’m sure the real solution is to get rid of the big stories, but until we acquire that skill (we’re working on it!), this process seems to be working well for us.

Test Automation II

After three days of development I have a basic framework in place and about 10 automated acceptance tests. I suspect I’m far from the best solution right now, but it works and I think it’s reasonably easy to work with. I’ll be testing that theory soon though as the next step is to get the framework reviewed by some colleagues. I’ve already had a few people take a look and suggest some changes, but now it’s time for a proper code review.

I can see already how important it is to make the automated tests maintainable. If anything I think they need to be written to a higher coding standard than the rest of the application. Throughout the 3 days I’ve been working on this I’ve been constantly refactoring. If a method feels like it’s in the wrong place I take the time to assess where it should live. If a class name no longer fits, I change it. This absolutely cannot stagnate or it will become a burden to maintain and then the tests will be useless.

I’ve made rather more use of static classes than I would normally find acceptable, however SpecFlow requires the Before/AfterScenario attributes to be on static methods whereas our base class for database tests is not. To get round this I’ve created a static wrapper round the test class which exposes just the methods the automated tests need.

I’ve also ended up with a static object store so that I can split up my step definitions into several files. The idea being that you create an object in one step definition, put it in the store then get it back out in another step definition. It seems to be a neat idea, but I’m not sure it’s practical. Let’s see what my colleagues think, in the end they’re the ones who will have to use the framework.

*** Update ***

So the code review went well. I’ve added in some more structure to make it obvious how the system should grow (and so we don’t end up with monolithic files) and I made some improvements to the underlying structure after some suggestions by my colleagues. The real test will come next sprint when we write automated tests for another feature.

Test Automation I

Over the past couple of days I’ve been trying to put into place an automated acceptance test framework. Just the beginnings really, but I have a new found respect for test automation engineers.

My aim is to just get something going, to make a start that other people can build on. I’ve taken one feature which we believed was no development effort, but which leant itself very well to automation. I’m also trying to be as agile as possible, developing just enough of the framework to meet the needs of the tests I’m writing now, but also trying to make sure I’m building decent foundations.

I’ve decided to begin my experiment using SpecFlow because it integrates nicely with Visual Studio and because I like the natural language support from cucumber. This might change in future, but I’m hoping it gives up a quick start.

F#

Last night I went to a talk by Phil Trelford organised by software east. I’d heard a bit about F# and a friend had showed me some code, but I’d never tried using it.

Before the talk I was chatting to Phil and he asked me if I enjoyed C#. I said that I did and he replied that F# would change that! And I can really see why…. All the boiler plate is gone… Classes defined in one line, getters assumed etc. Makes for really clear concise code. I can see why it would be a great language to use to define your business domain.

The bit that had me amazed though, was the built-in compile time support for units of measure. You can define your units e.g. metres and seconds and then apply them to values in the system. If you try to multiply incompatible units or assign a value to a variable with different units you get an error at compile time. That’s got to be so useful for scientific applications.

I’m sure I spent most of the talk with my mouth hanging open! Definitely going to try out some F# at home.

Moving Motivators

I tried this exercise from Jurgen Appelo in one of our weekly team time slots. You get 10 cards each with a different intrinsic motivator on them. In the first step you simply order them from most to least important. Sounds easy, but almost everyone has one or two they  struggle to place, either because they find it hard to define (honour was a particular problem for my team) or because they value them similarly.

Continue reading

Getting a Group Talking

I was chatting to a friend yesterday and we were discussing how planning meetings, retrospectives and stand-ups can be really hard work when you have a group of people that just don’t want to talk. Agile needs good communication to make things work, so how do you go about persuading a group to talk when they are naturally quiet and shy individuals?

We have a few thoughts between us and I’ve come up with a few more since that conversation. Continue reading

Just Finished Reading…

Don’t Make Me Think by Steve Krug

Cover of Don't Make Me Think

This slim book covers all the basics for designing a usable website. It is really easy to read with plenty of pictures to explain the points. You find out how people really use the web (i.e. in a rush), which key areas of a site you need to get right and a bit about how to push back against daft suggestions from higher up. There’s also a good chapter on doing usability testing on a (very) low budget.The tag line for the book says it all really ‘A common sense approach to web usability’.

Although this book was almost exclusively talking about websites and I work on a web application I still found there were many useful and relevant points for out situation. Clear navigation, obvious search and getting rid of needless words are equally as important in web applications.

The section about designing the homepage was also interesting. Steve Krug makes the point that the homepage has to make it clear what the purpose of the site is. The product I work on has a web application as part of it’s suite of tools. As a company we don’t control most of the real estate on the homepage as customers can customise it to meet their needs, however I think it is still important to make the purpose of the site clear. We do have control over the title bar and left hand side and they can convey a lot of information on their own.

Last year we did some usability testing on some summer students working for the company. This year it might be interesting to start by showing them our web application and asking them what they think it’s purpose it. Can they tell just by looking? What would they use the site for if someone pointed them at it? I’d be interested to find out.

To sum up this was a very interesting read and I would definitely recommend to anyone developing or designing websites or web applications.

Shrink-wrapped Agile

So, I’ve been wondering how Agile fits with the idea of a long-lived shrink-wrapped product? A lot of the examples in books and online talk about web or hosted solutions where you can decide where and when your software is deployed, or else they talk about one-off projects where you are working directly with a customer and can deliver software to them at regular intervals. The nature of our product presents us with a number of problems when implementing agile.

Customer

Who is our customer? Is it the company employee sponsoring the project? Is it the person who buys our software? Is it the person who actually uses the software? I’m inclined to think the answer is actually all three, but that presents a problem of it’s own.

Access

Most of our end users aren’t commissioning features, they just get told by whoever purchased the product that they should use it. So how do we reach these users and make sure they are getting something useful? I’m not sure I have a good answer for this yet, perhaps we should be making more effort to find out what they need? Usage data would probably help a lot, but how do we get it? This is not a hosted solution so we can’t access log files unless someone sends them to us.

User Documentation

By user documentation I mean the manuals and help pages that are used by people when they get stuck or want to find out about new features. Even if our UI was so intuitive that it was always obvious what to do next, we would still need conceptual help for new features. Explaining why someone might use the new functionality is important when you have a technical product like ours.

So how to we incorporate our documentation team into the agile process? There’s a lot of literature around about how QA and testing fit in, but no-one seems to mention technical writers. Ideally we would like them to document along side the feature work, but is that practical when there aren’t enough writers to go around the teams? Also later feature work may invalidate documents already written or at least mean they need an overhaul. I think maybe our real problem is that the value of good user documentation is not realised so there is no desire to improve? Perhaps that’s what we need to change?

Slack Time

So I went on the course that goes with the book and I had a great time. It was Jurgen Appelo himself who did our class so it was good to meet him and he is a really interesting and engaging person. Despite having a horrible cold I didn’t find myself drifting off or losing track.

Over the next few days I’m going to write about a few of the ideas from the course that resonated with me. The first of these is slack time.

This is something that I didn’t immediately think of when asked about takeaways, but it’s something that’s been growing on me for the past day or so.

In our team we’re definitely guilty of not leaving any slack, the temptation is to fill up the sprints as full as possible, often with a bit extra thrown in for good measure. This leads to a relentless pace with us often feeling demotivated because we can’t get everything we planned done. As scrum master I’ve tried in the past to tone things down, asked the team to put less in, but in the end I fall into the same trap of just wanting to get everything done and keep the sponsors happy.

Looking back over the last release I’m starting to realise that this is not working well and we need to change. We need occasions where we can sit back and congratulate ourselves about finishing everything we planned. We need to feel good about ourselves now and then or it doesn’t take long before you lose your energy and then you end up working even slower. There are other bad effects as well, the crammed sprints mean that our testers fall behind and never quite catch-up which means we end up with a mini-waterfall rather than agile.

The problem is that on a 3 month release cycle it doesn’t feel like we have the time to experiment. If we plan in a really ‘light’ sprint with few features how are we going to meet the sponsors expectations, it can look (and feel) like we are just slacking off. There isn’t time for things to even out, or at least that’s the way it feels.

We need to find a way to break out of this cycle and to change the way we work, we need to manage expectations of sponsors so that we have the space to make these changes and we have to convince them that it will benefit them in the long run as well.