Not all developers are created equal.
In a perfect world, everyone would be super diligent and proficient at creating CSS, but in reality this is not the case. In some cases less experienced developers can make mistakes, create inconsistant code or not reuse exiting code. Even in other cases when more than one experienced developer works on a project, you can still end up with inconsistency and repeated code just by the dint of different working styles.
I thought I’d start experimenting with automated testing of frontend presentation code, focussing on regression testing. This topic is not really discussed a lot, as the standard response is that there is no replacement for an eyeball test, but humans are by nature unreliable beasts and I’d like to change that and make front-end more of an accepted science.
My hope is that by trying out some techniques and bringing them into the forum I can at least start a discussion that results in an advancement of this field. Allowing us to escape some of the common traps we see at the moment and mitigate some of the risks associated with our profession in the same way back end coding has with unit testing.
This is by no means the first time anyone has looked at this problem. There are other software solutions available like Hp’s WinRunner, but in my opinion they are generally unsuccessful or not fit for purpose. The existing solutions rely on doing algorithmic pixel-based comparisons of screens. Of which the process involves, a screen being designed & built. A “good version” of the screen captured and stored. Then every time the application goes through the build process, the screen is re-captured and compared for differences to the master. Any deviations are noted and the build process fails.
Now this works for static screens. But the problem is that most of the time our applications do not have static screens, the content dynamically changes and therefore, everytime it does so, the build process would fail, invalidating the automated nature of the process. Not only that, but these screens render differently in each browser, so you end up taking 4-8 captures, which exponentially increases the potential to fail incorrectly. In essence these tests are too brittle and so not actually very useful, as they break too easily.
Taking a step back
Given that doing very low level atomic checking seems to be unhelpful, let’s analyse what the actual process is, that a human developer uses to validate a page by eye.
When I look at a page against a design, I don’t compare pixel by pixel. I compare a higher level. First I look at the design. From the design I create a number of mental rules. In my head I create a list of all the different font variations, the different colors used and the rough layout. If the design has too many variations in these things, then it is inconsistent and hence bad user design, in which case I end up going back to the designers and asking for a higher level of standardisation. When we have this basic checklist of “design principles”, we can compare the font size, weight and face, compare colors, compare widths, heights and alignments to the implementation. This enables us to take a design and an implementation and quickly gauge at a high level whether it is likely to be correct.
Taking this principle changed the way I started thinking about unit testing CSS, what if we could formalise this set of design principles and turn them into a programmatic set of rules that we could test each page against?
For this, my experiment led me to create cssUnit, a framework for checking style consistency.
Where cssUnit might help.
Like all software projects, not all processes are suitable for every situation, this is no exception. You will have to evaluate whether css unit testing is right for your project, the key factors I would take into account for this are, project lifespan, number of developers and size of site.
The scenarios in which cssUnit testing will be helpful are:
- Large corporate web presence where there is a overall style guide but many different sites/microsites maintained by lots of developers
- where you have a very fluid team - may often be short term freelancers
- where you are training up a young/unfamiliar team of front-end developers,
- situations where there is overlap between front end and backend - but not necessarily the possibility to maintain quality
- co-located or remote teams
As well as being a tool to maintain quality - unit testing is also a way of communicating and distributing knowledge in a direct fashion, few developers will bother to spend the time reading the style guide, but many will learn the rules through failing tests.
What cssUnit is not.
cssUnit is not a number of things - I thought I would just mention the ones I did not mean it to be
Not cross-browser tested - although in principle it should be easy to make cross browser - it is not - not yet anyway - if it proves to be useful, then perhaps there is a case for making it cross-browser
Not beta - cssUnit is not by any means complete, and is as expressed before is more of an experiment, I have only coded for the scenarios I thought of, I’m sure there are better ways of doing this, but this is at least a starting point to explore how it might be done.
Not a service - lots of testing platforms have become services recently, but this is not currently my aim. There are many directions this could take, at the moment I’m not sure making it a service is the correct path.
Not easy - like all unit testing, cssUnit takes time to set up and implement in the beginning. The hope is that in the long term it saves you more time than it costs. It is also dependant on you having a strict style guide - with out standardisation in your design, cssUnit becomes useless.