Key Rules and Tenets of XP

<< XP's Audience

As described earlier in this article, XP is a mixed bag of common sense and, well, common non-sense. In the on-line XP forums and newsgroups, you will often see people arguing against the bad rules, and being quoted the good ones in return (and vice versa).

When you recognise that this is happening, you will also recognise that this constitutes a huge proportion of the debate that has been raging over XP.

In this section we analyse some of the XP practices, values and recommendations. For convenience, we collectively refer to these as "rules", although XPers might be quick to point out that XP does not have "rules" as such, just recommendations.

XP Rule: Have a Customer In The Office at All Times

Customers often don't have time for a 'love-in'. Often they will supply a specification and tell you to go away and make it happen, or ask you to beat it out of them in a two week span, get sign off, then make it happen. Do you refuse? Do you run of of the room when they produce the spec? If you come back several times a day to ask question after question they may well want to kill you, and think that your company is full of morons.

The original XP spec described the need to have a real customer (ideally an end-user of the system) in the same room as the programmers throughout the project.

Can the customer afford to simply give away a member of staff 100% to a project?

Would the staff member accept such a lame job (having to spend all his/her time answering a never-ending barrage of inane questions from the random mouths of the programmers)?

If the customer representative was junior then he would not be the right person to make project-related business decisions. If he was senior he wouldn't be there. He really isn't likely to be very high up the food chain in his own company. In fact, it's quite likely that he will be a techie (or techie-oriented) himself.

It's not surprising, then, that the Customer role in XP has since changed. For a while it was a single business analyst (rather than an actual customer). More recently, the Customer role has evolved into a team of analysts (possibly larger than the team of programmers). What is not clear at the moment, however, is whether all of these people need to be co-located in the same room for the entire project; or indeed whether it is now recommended that requirements be written down in more detail.

XP Rule: Integrate Often

What this rule really means is "integrate your code at least once a day, ideally more often".

Code divergence is a problem in any software project (XP or otherwise). If the programmers have different builds on each PC, getting them all to fit together can prove more time-consuming than writing the code in the first place. Doing a daily integration build in any project is definitely beneficial, because it prevents nasty integration problems from lurking, and suddenly emerging late in the project.

The problem is even more acute in XP because both the codebase and the design are evolving at a fast rate. A large number of the refactorings described in Martin Fowler's Refactoring book involve interface changes; so tests to cover constant refactoring must cover the entire system, not just the module being rewritten. If the design did not need to evolve quite so much - if some more time had been spent up-front working the design out, prototyping and so forth, then the design would be a lot more stable, hence the code would not be evolving and shifting at such a rapid pace.

XP Rule: Do a Little Design

XP doesn´t say "don´t do design", it says "do a little design every day and every week". Design up-front in XP usually involves CRC design sessions of 10-30 minutes. This approach is described by Ron Jeffries in Extreme Programming Installed:

“Get a few people together and spend a few minutes sketching out the design. Ten minutes is ideal—half an hour should be the most time you spend to do this. After that, the best thing to do is to let the code participate in the design session—move to the machine and start typing in code.”

The example given here is amazing. This wouldn't be so bad if they were talking about CRC cards metaphorically - i.e. they were advocating a software CASE tool that will persist this strange mix of requirements analysis and systems design work for later reference, requirements tracking, project metrics, automated impact analysis and so on.

But of course, with XP you don't need any of that "old-fashioned" stuff. Going by their example, the position of the cards appears to be a vital part of the analytical process. This is great until the office janitor helpfully shuffles the cards into a neat stack, or accidentally sucks one up into the vacuum cleaner. Then all that's left is peoples' memories of what they think is important - i.e. what direction they are planning to take the design in over the next few iterations. Everyone will remember slightly differently.

In reality, the overall system needs to be architected up front - at least to a certain level of abstraction. Sure, you can break it down into manageable modules and project milestones. Then the programmers can start work on one module whilst another module is undergoing a low-level design.

XP takes a similar approach, but - appropriately enough - takes it to extremes. Effectively, nothing gets designed until it's time to program it. Of course, this approach means that you're not going to know how the pieces will fit together until they have all been written. This is why XPers have to integrate so often, to make sure everything fits. It's why they are forced to keep testing the entire system, several times a day; and it's why they have to refactor (rewrite and redesign what was supposedly completed) virtually all the time.

If all the pieces were designed in advance, you would know in advance what will fit and what needs to change - before it's been coded. That's what design is all about. This approach has been proven to keep costs down, and to prevent the project scope from escalating like a mad thing. The XP approach makes this simple process impossible, so in any project they have to "make it up as they go along".

Programming does, of course, provide additional insight into the design (this is described in Agile Modeling as "proving it with code"). This is why the XP approach to design is so compelling for many people. There is a natural urge to "leap into the code" and see where the design takes you. This would appear to give XP the edge over more traditional up-front design approaches, except that up-front design approaches also tend to require a certain amount of early prototyping - writing "throwaway" code to validate the design first. This gives the required insight into the design before time is spent coding the production version, with unit tests, full error handling and so forth.

Your experiences may be different, but I have found that designing the system in detail first (including early prototyping) actually reduces time-to-market. Doing "a little bit of design every day" - leaping into the code and patching up the design later with aggressive refactoring - can only delay the project. Wrapping this hacky approach in a feel-good methodology just doesn't make it okay.

XP Rule: Refactor Mercilessly (If it ain't broke, fix it anyway)

The basic premise behind this rule is that if you keep going back to your code and rewriting it, then it will just keep getting better and better.

In XP, constant refactoring is necessary to support the emergent design practice, and to fight the rising "cost of change" curve by keeping the code simple. This sounds admirable, but turns out to be an immense effort hog - especially when combined with all the other XP practices that must be put in place to support this technique.

Outside the XP world, occasional refactoring turns out to be useful, when combined with an up-front design and short iterations. Spending some time initially getting the design right means that the design stabilises, therefore does not need to evolve as such. It will inevitably change to an extent as your understanding of the project grows, but this simply leads to an improved design, a bonus - whereas in XP the process relies on the design changing to harbour the waiting fleet of user stories.

With XP, constant refactoring (i.e. constant tinkering and redesigning of your code) is supposedly possible because the automated tests will catch out any problems. That is, the tests will catch out all the bugs that have been thought of in advance, but no others. Last time I checked, most bugs are due to things that the programmer had not thought of.

Sure, you can keep adding tests once the bug has been found by a diligent QA tester (or disparaging customer), but it's sort of too late then. Wouldn't it be so much better to concentrate on getting the code right first time round, recognising when that module is finished and moving on to some other part of the project that is more worthy of your attention?

XP Rule: Pair Programming

This is one of the more controversial aspects of XP.

A common argument amongst XPers is: "You don't know until you try it"; put another way, "Try it! You might like it!" So I think what they're saying is: until you've tried pair programming, you probably don't know whether you like it or not.

Luckily us humans have pre-cognitive dissonance. I have never tried injecting heroin; I have never tried sticking my head in a bucket of shit; I have never tried constant pair programming. But I know in advance that "NO sir I wouldn´t like it". I know in advance, having seriously considered it, that constant pair programming is a totally warped idea.

To put that in a slightly less contentious way: we don't have to try something out just to know whether it's a good idea or not. We have the power of thought and extrapolation based on prior experience.

For example, I have "pair programmed" in the sense that I have sat with other programmers, and together we have worked our way through a difficult area of code. Same goes for refactoring a badly written code module: sometimes some lateral thinking from a fresh mind is all that is needed to identify a better, clearer design.

However, it doesn't take a great leap of the imagination to understand that pair programming done all the time is a bad idea. Most of the time, a programmer needs to be isolated so that he or she can concentrate and work their magic.

Programming done properly is meditation - your mind transcends to some other place, of pure thought, with a thousand variables being juggled effortlessly. This is why programmers need peace and quiet to program. A couple of mediocre "programmers" fumbling their way clumsily through a problem together, bouncing lame ideas off each other, just isn´t the same. There is no direct, telepathic connection between the two minds. Ideas cannot flow. Logic cannot formulate in quite the same way that it does in one, totally focussed brain.

This idea is borne out by Mihaly Csikszentmihalyi, a professor and former chairman of the Department of Psychology at the University of Chicago who has devoted his life's work to the study of what makes people truly happy, satisfied and fulfilled.

To quote from the above-linked website:

Mr. Csikszentmihalyi (pronounced chick-sent-me-high-ee) is chiefly renowned as the architect of the notion of flow in creativity; people enter a flow state when they are fully absorbed in activity during which they lose their sense of time and have feelings of great satisfaction. Mr. Csikszentmihalyi describes flow as "being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz. Your whole being is involved, and you're using your skills to the utmost."

That's not to say that teamwork doesn't help. Teamwork is a vital part of software development: think-tanks; talking ideas through; weeding out the stupid stuff before you go ahead and program it. That's the vital point: team collaboration is much more beneficial when it takes place before you start programming, than when programming.

If you have to bounce ideas around whilst programming, use your subconscious. It's huge and rich with wisdom: make it work with you.

Another problem is that pair programming takes a "peer" concept: there is a danger that it will tie up one experienced programmer with a clueless sidekick. At least with proper teams (with clued-up team leaders) the "talent:wastage" ratio is more acceptable. In other words: in a team of four programmers, a talented, knowledgable team leader benefits four people, and in fact has enough time to do his own programming, design work, planning and testing as well.

With pair programmers, the man who would be team leader is tied to a single sidekick at a time, and must spend that time lowering himself to a level of cognizance and ability that the inexpert coder is comfortable with, correcting the junior coder's mistakes (at least until the pair rotates, and it's somebody else's turn to pair with the inexpert programmer). Sorry if that sounds offensive - it isn't intended to be. All programmers were just starting out at one stage. We all had to go through this process of learning the basics, making mistakes, correcting them (or having them corrected for us). One day, the junior coder will be King: experienced, confident, the teacher. It's a long and rocky road though.

XP's pair programming approach means that the junior programmer is guided all the way, rather than being left to figure things out for themselves at least some of the time. That's no good: the expert is just doing the junior coder's work for him. Neither the expert nor the junior coder benefits. The expert does not get a chance to really excel, to write superb and clearly constructed code in record time. The junior coder thinks he is doing brilliantly (even though he hasn't touched the keyboard, or is being told what to type). He will get a shock when he is suddenly asked to do something clever, and falls flat on his face.

XP Rule: No Requirements, Just User Stories

Detailed, documented user requirements are vital, even for small projects. Especially for small projects - as these tend to be approached in a much more casual manner than their larger cousins.

Small, vaguely specced projects are a big cause of political angst between customer and consultancy. "Could you knock together a Java applet for us to display on our website? Yeah, it just needs to do stuff." Months later, the applet has been re-written twenty times to try and please the customer; the customer, amazingly, will grow increasingly angry, as they feel that the consultant just doesn't know what it is they want. Which they don't, because nothing has been written down in any detail.

Now scale this example up to a larger project, where each module is roughly the size of that little Java applet - except now there are literally hundreds, or even thousands, of these modules. Imagine the chaos, and the misunderstandings, and the political head-chewing that will ensue.

The closest XPers come to documented requirements are small story-cards, on which notes and ideas ("user stories") are hand-scribbled. Sometimes (depending on the whim of the team) these cards are thrown away - dis-carded if you will. Then all that's left is the multitude of automated test classes ("acceptance tests").

Detailed user requirements aren't necessary? XP cheerily throws detailed specs out the window, and states a preference for having as little documentation as possible (even to the extent that some elements must be communicated only verbally).

To emphasise, the catch here is that earlier problems with software development, for which detailed requirements were postulated, have been neatly forgotten about: namely, that the software produced is rarely exactly what the customer had in mind. This result could be because the programmers simply got the wrong idea from the start; or it could be because they were given carte-blanche to analyse, plan and second-guess the customer's requirements.

Without something written down and signed off by the customer, this can (and frequently has) become a legal nightmare. Expect this problem to surface near the end of an XP project, when "That's not what I was expecting" starts to be heard around the office. This may not only come from the on-site customer representative (assuming that in actuality he/she is able to remain on-site for the entire duration of the project). More likely, the expressions of surprise will come from the real customer - the people who are paying the representative's salary - who are gobsmacked by the fact that the nice, straightforward e-commerce site they originally asked for has been steadily "refactored" into an ebay-style auction site with on-line community support and a groovy Flash user interface.

Never mind, they'll cheer up when they discover how cool all the refactored algorithms are behind the scenes.

XP Rule: Write The Tests First - "According to my calculations the problem doesn't exist"

The basic premise behind test-first design is that the test class is written before the real class: thus, the end purpose of the real class is not to fulfill a requirement, but simply to pass all the tests that are in the test class. Then - and this is the ingenious part - you can make changes to your code, with the confidence that if you've introduced a random unexpected bug, you'll have anticipated it in at least one of your unit tests.

What if there are bugs in the testing code? It's code, after all, so it is just as susceptible to bugs as real code. Who will know whether the real result should have been 4.1 or -7418.3 or "trout fishing"? No-one, because the requirement wasn't written down anywhere in a proper requirements spec. To make matters worse, XP says that by default there is no code-level documentation. Thus, it is even more difficult to track down the necessary changes.

Don't get me wrong: unit tests are wonderful things. You have to be intelligent about it though. You really don't need to write a test for every single one of your "important" methods. All that gives you is twice as much code to change, and fix, and trawl through.

Unit tests also tend to catch bugs at too low a level. They are not sufficient to validate or catch out a design, to prove a design's "wrongness". Meanwhile, acceptance tests are too high a level for this purpose. So test-first design leaves an area in-between that is fundamentally missing.

If the design is thought through up-front, validated by senior engineers and proven with prototyping, then it will not need to be refactored nearly as much during production coding. Unit tests are still useful in this situation, but the process does not rely as heavily on them.

XP Rule: The Acceptance Tests Are The Requirements

In the real world, there has to be a requirements spec - not written as an Excel macro! - that the QA team can refer to.

The QA team might not be programmers themselves: they represent the users. They'll be testing the system from a user's point of view: does this message dialog do what I expect? What happens if I press this button? Does the program prevent me from booking the same hotel room twice for the same night?

I'm also against dusty old requirements tomes that bore you to tears on the first page. No-one reads them. It is possible to write requirements that get read: just liven the text up a bit; avoid pretentiousness and repetition; use interesting examples; organise and index the documentation well enough that it's immediately obvious where the reader needs to go to get the information they need.

It's an acquired art, but being good at what you do is a lot less drastic than throwing away the rulebook and starting over.

>> Next: The Four Values of XP

Software Reality XP Forum

<< Back to XP Central

<< Back to Lifecycle