Archive for the ‘development environements’ Category

A short story about the value of automated testing.

Saturday, July 9th, 2011

For the last few weeks, my team has been working on some pretty fundamental changes for Nokia maps on the web.

But that’s the short term. For the last six months my team has been working hard on getting a suite of automated tests up and running, and I’m proud to say we now have close to 1000 Jasmine unit tests and 200 cucumber acceptance scenarios running on every commit.

So I was delighted, when I heard that this week, one of the new changes had broken 52 of our acceptance tests.

Why should I be delighted?

Doesn’t that mean the team now has to go away and fix 52 acceptance tests? Well, yes, and that was certainly the way the team saw it, until I said:

“Now imagine we didn’t have those tests. What would have happened?”

And then we all started to realise, the power of automated testing.

So if we didn’t have automated testing, what would have actually happened?

Well, someone would have gone into the code. Found the piece of code that needed changing. And changed it. Probably tested it in the one place they knew about. Committed it. And moved on. Completely oblivious to the fact that they’d broken the application in 51 other different places.

Then the build would have gone to QA, the QA’s would have found the 52 different breakages, raised 52 new bugs, and sent them back to the developers.

Then the developers would have gone into the code, tried to fix those 52 bugs, and probably caused another 50 regressions. Which would then go to the QA team, be raised and go back to the developers for fixing. Eventually this hugely inefficient cycle would have continued until the build gets to an extremely brittle stability, a bit like building a house of cards, and it would have gone live.

However, with the tests, those bugs were caught. Immediately. And fixed. Immediately. Those 52 broken acceptance tests, saved us potentially months and months of rework.

So the moral of the story is, when you reduce the feedback cycle of a bug, with the saftey net of automated tests, it allows you to develop in confidence, giving you warning to what you’ve broken and giving you the opportunity to fix it then and there (however having the discipline to do this is another story). In Lean, this is referred to as ‘building integrity and quality into the product from the beginning of the cycle’, rather than trying to add it in at the end with a torturous QA cycle.

Tags: , , ,
Posted in agile, developement, development environements, testing | No Comments »

Quickstart tutorial to Mercurial and GoogleCode

Sunday, January 3rd, 2010

I love using new technology. I really do. It’s great, basic things you knew how to do with an older system, get reinvented and become no longer trivial. They become a long drawn out, painful process, that involves your blood pressure raising to that perfect level where it reduces you lifespan by a twenty minutes. Each time.

mercurial-logoSo when I switched to command line Mercurial from Tortoise GUI SVN, you can imagine my delight.

To be fair, I actually had already got command line SVN experience, so thought the transition might be somewhat simpler. It wasn’t. The commands may be very similar but the process itself is not.

code_logoFrustratingly it is actually very simple once you know the command-set to get you up and running, unfortunately no has written a basic, 123 guide to setting up a googleCode mercurial instance on windows XP. So I thought I’d be the first.

  1. Download and install Mercurial command line. Make sure to check the last check box on the installation screen along with the readme, this allows you to access Mercurial from the command line.
  2. Create your Google code project. This part’s easy, follow the steps enter the words and navigate youself to the main project screen. It helps if you already have a Google account.
  3. Now we get to the fun part. Open your command line in windows, by going to Start->run->type cmd in the input box and click ok. Or if you have style double click the short-cut you’ve created for yourself on the desktop.
  4. You’ll want to navigate to a folder where you can store your project. For the sake of this example, lets say your project is named testproject and is in the root of c:\. first back out to the root with: 
    cd \
  5. Then make a directory to store the project in: 
     mkdir testproject
  6. enter the directory 
     cd testproject
  7. Now you want to take a clone of your googlecode repository, to do this type the following command: 
    hg clone https://testproject.googlecode.com/hg/ testproject
  8. The last parameter is the directory to copy it into on the local file system. This should result in Mercurial spewing out some output identifying that it has correctly downloaded.
  9. To check, type: 
    cd testproject
dir
  10. If it was successful, you’ll see a directory called “.hg”
  11. If it’s there, now you want to make your first commit. Create a text file called readme and dump it in the directory with the .hg in. To add your new files type: 
    hg add *
  12. Next you’ll want to commit them, type:

    hg commit -m "added first file to repository"
  13. So you have commited your first change to the repository, but this does not mean it’s gone up to the main node on Google, it is currently just committed to your local file server. To commit the changes back up to Google’s servers, you’ll need to run the following: 
    hg push

    This will prompt you for a user name and password, the password IS NOT YOUR DEFAULT PASSWORD. Go to your accounts hosting page to find it.

  14. Now, if your like me, i.e. LAZY, you don’t want to have to enter your username and password each friggin’ time you want to push. You want an automated way to authenticate your https googlecode push actions. The way to get around this is to open the “.hg” directory and find the file “hgrc”. Edit this in your preferred text editor. Where you see the line
  15. default = https://testproject.googlecode.com/hg
  16. change it to 
  17. https://(your username here):(your password here)@testproject.googlecode.com/hg
  18. And Voila! everytime you push, it automatically skips the login and password steps.

I hope this helps, I spent sometime this weekend trawling the internet for clues as to how to do this most basic of processes but could find no simple explanation, written in a language people could understand.

Tags: , ,
Posted in developement, development environements, new technology | 3 Comments »

CSSUnit : experimenting with unit testing presentation code.

Thursday, October 1st, 2009

Not all developers are created equal.

In a perfect world, everyone would be super diligent and proficient at creating CSS, but in reality this is not the case. In some cases less experienced developers can make mistakes, create inconsistant code or not reuse exiting code. Even in other cases when more than one experienced developer works on a project, you can still end up with inconsistency and repeated code just by the dint of different working styles.

cssunitscreenshotI thought I’d start experimenting with automated testing of frontend presentation code, focussing on regression testing. This topic is not really discussed a lot, as the standard response is that there is no replacement for an eyeball test, but humans are by nature unreliable beasts and I’d like to change that and make front-end more of an accepted science.

My hope is that by trying out some techniques and bringing them into the forum I can at least start a discussion that results in an advancement of this field. Allowing us to escape some of the common traps we see at the moment and mitigate some of the risks associated with our profession in the same way back end coding has with unit testing.

Existing approaches

This is by no means the first time anyone has looked at this problem. There are other software solutions available like Hp’s WinRunner, but in my opinion they are generally unsuccessful or not fit for purpose. The existing solutions rely on doing algorithmic pixel-based comparisons of screens. Of which the process involves, a screen being designed & built. A “good version” of the screen captured and stored. Then every time the application goes through the build process, the screen is re-captured and compared for differences to the master. Any deviations are noted and the build process fails.

Now this works for static screens. But the problem is that most of the time our applications do not have static screens, the content dynamically changes and therefore, everytime it does so, the build process would fail, invalidating the automated nature of the process. Not only that, but these screens render differently in each browser, so you end up taking 4-8 captures, which exponentially increases the potential to fail incorrectly. In essence these tests are too brittle and so not actually very useful, as they break too easily.

Taking a step back

Given that doing very low level atomic checking seems to be unhelpful, let’s analyse what the actual process is, that a human developer uses to validate a page by eye.

When I look at a page against a design, I don’t compare pixel by pixel. I compare a higher level. First I look at the design. From the design I create a number of mental rules. In my head I create a list of all the different font variations, the different colors used and the rough layout. If the design has too many variations in these things, then it is inconsistent and hence bad user design, in which case I end up going back to the designers and asking for a higher level of standardisation. When we have this basic checklist of “design principles”, we can compare the font size, weight and face, compare colors, compare widths, heights and alignments to the implementation. This enables us to take a design and an implementation and quickly gauge at a high level whether it is likely to be correct.

Taking this principle changed the way I started thinking about unit testing CSS, what if we could formalise this set of design principles and turn them into a programmatic set of rules that we could test each page against?

For this, my experiment led me to create cssUnit, a framework for checking style consistency.

Where cssUnit might help.

Like all software projects, not all processes are suitable for every situation, this is no exception. You will have to evaluate whether css unit testing is right for your project, the key factors I would take into account for this are, project lifespan, number of developers and size of site.

The scenarios in which cssUnit testing will be helpful are:

  • Large corporate web presence where there is a overall style guide but many different sites/microsites maintained by lots of developers
  • where you have a very fluid team - may often be short term freelancers
  • where you are training up a young/unfamiliar team of front-end developers,
  • situations where there is overlap between front end and backend - but not necessarily the possibility to maintain quality
  • co-located or remote teams

As well as being a tool to maintain quality - unit testing is also a way of communicating and distributing knowledge in a direct fashion, few developers will bother to spend the time reading the style guide, but many will learn the rules through failing tests.

What cssUnit is not.

cssUnit is not a number of things - I thought I would just mention the ones I did not mean it to be

Not cross-browser tested - although in principle it should be easy to make cross browser - it is not - not yet anyway - if it proves to be useful, then perhaps there is a case for making it cross-browser

Not beta - cssUnit is not by any means complete, and is as expressed before is more of an experiment, I have only coded for the scenarios I thought of, I’m sure there are better ways of doing this, but this is at least a starting point to explore how it might be done.

Not a service - lots of testing platforms have become services recently, but this is not currently my aim. There are many directions this could take, at the moment I’m not sure making it a service is the correct path.

Not easy - like all unit testing, cssUnit takes time to set up and implement in the beginning. The hope is that in the long term it saves you more time than it costs. It is also dependant on you having a strict style guide - with out standardisation in your design, cssUnit becomes useless.

Tags: , , ,
Posted in agile, developement, development environements, javascript/css, testing | 2 Comments »

Is frontend & backend done the wrong way round?

Thursday, May 14th, 2009

I’ve been thinking about this post for a few weeks now, what stopped me from writing it initially was that I couldn’t decide which an analogy to use. I had too possible contenders, the car industry or the housing industry. The car industry is generally a better comparison tool as it is soo well regarded as being the pinnacle of efficiency, however the housing industry seems to make more sense in this context.

So I want to start thinking about how we build a room in a house. Lets say we have a shell of a house and want to create a bedroom to sleep in.

So firstly we’re going to need walls. Which will be made by our carpenters, who create a frame and screw it into place. They then add drywall, which is prefabricated plaster board to act as a base layer for the wall. Then the plasterer will come along and apply a layer of render to the wall, it will dry and then we can put wall paper on it.

Now in web development, what currently seems to be the practice is for front-end developers to be given a head start on templates, a lead in time of one interaction, say 2 weeks. Then on the next iteration the back end team pick up the stories where they develop the back end functionality and integrate the front end templates.

Now to me this seems like the house process in reverse, the plasterer has 2 weeks to create a render, the wall then gets built and the carpenters then have to try and apply the render to the walls themselves. This results in two outcomes, the carpenters getting frustrated as they don’t specialise in plastering and can’t see why it isn’t right, the second is that the wall looks shit and the plasters have to be called in again to do it again. This to me appears very wasteful and pointless.

We should be moving towards a process where, the the back end builds the bare bones of the site 2 weeks before the front end gets to it to apply the final render. This will enabled the front end coders to apply the superficial level and plaster over any cracks that may have appeared and the back end free to focus on providing a solid base layer.

Posted in Uncategorized, development environements, testing | No Comments »

Seeking the perfect build process.

Monday, May 4th, 2009

I am currently working on a project with a continuously integrating multi-stage build process.

Every time you commit a new build gets triggered and flows down the pipeline of testing stages. Problem is it doesn’t work very well.

One of the primary problems is that the tests getting run seem to be incredibly brittle and unstable which causes a lot of broken builds. Which in turn prevents people checking in. Which in turn creates a backlog of checkins. Which results in people forgetting what the code they need to  check in does. Which results in more broken builds. Which results in a myriad of other problems, including people losing code so massive wastage. Very un-lean.

So the process is broken. The solution is  that a broken build shouldn’t stop other developers working.

How do we implement this?

Well, one suggestion is that we have an automated revert process that every time a build fails, it automatically reverts to the last good build. This works in principle - a developer commits, it breaks the build, the build reverts, the next developer checks in, the broken build commit gets pushed to the back of the queue & creates an incentive to checkin working code.

However in reality, the build is not a instantaneous process. It takes around 15 minutes to get through all the various stages in the process. Arse.

This means the above scenario actually results in: Developer checks in broken code, build runs, meanwhile another developer checks in working code that requires some bit of broken code, build fails, build reverts, next checking code also fails as requires reverted code. Everyone stabs each other in the face.

So, how do we solve this?

Well, lets create an analogy, if we look at the build like it is a lego brick wall, and each commit is a new brick in the wall. So someone clicks on a brick which is unstructually sound. What are the other brick layers doing?

Well, some of them are working on the adjacent walls, and their brick work is unaffected by removing the broken brick, so their bricks should remain unchanged as long as they don’t sit upon the broken brick or any of the bricks that were installed at the same time as the broken brick.

Bricks that were built on top of the broken brick or any of the bricks installed at the same time as the broken brick, well, they’re fucked.  Because it’s Lego, you can’t just slide out the broken brick and replace it, you have to dismantle the wall and take out any bricks above it in order to replace the broken brick, which results in a large number of angry brickies f’ing and blindin’ and going off to read the sun and get a bacon sandwich.

So one way to fix this is to change the build process. Instead of letting people build on top of other bricks before they’re checked for stability, you take their bricks and hold them until the bricks they rely on have been checked, then if the parent bricks are safe, the dependant bricks are put on and checked themselves. If further bricks are put on then they are put in the dependant queue and checked in turn.

Voila, you have a smooth parallel build process.

Tags: , , ,
Posted in development environements | No Comments »