|~kiko||Resumé||Software||Diary||Papers||Outdoors||Travel||Book shelf||Unix help|
Async Free Software, Inc.
São Carlos, Brazil
Coding is the one most important task the process considers, and most of
the auxiliary subsystems a complete software process has are gone or
executed during, or together with, the coding phase. Gone, for example,
are the tasks of composing and maintaining requirements and
This basic drive of putting the system into production as soon as
possible and keeping software in `eternal evolutionary maintenance' echos
quite strongly Eric Raymond's ``Release early, release
often'' motto for Free Software projects.1
The author of the method and book, Kent Beck, makes liberal use of
informality in his first book on XP, but provides good background and
backup for his theory. One interesting point about XP is that it's
designed for small projects developed by up to 10 programmers, and most
of it is designed to help get products out the door as soon as possible
-- providing important feedback to customers and keeping developers
This contrasts heavily with common Software Engineering methodologies , which are usually quite cumbersome, focused on heavy amounts of documentation and that in general only adapt well to large-scale projects. Up to now, smaller companies had to rely on melting many different techniques into a software process that would scale well to their size and needs. XP is an attempt to provide a ready-to-use process for these companies.
My view of XP divides it into Exploration, Planning, and Development phases. The system goes from Pre-Release to Maintenance status after the first release, but these phases repeat themselves throughout with the same principles.
The Exploration phase starts off with business -- the customer --
writing stories that describe what the system needs to do, and
developers actively trying out all the different technologies and the
possible system configuration involved with the project.
The Exploration phase might be considered a parallel to Requirements Analysis, though very concise and incorporating a period of freedom for developers to experiment with the solutions they will be proposing and implementing.
As soon as stories have been collected, a Planning Game is conducted,
and development estimates how long it will take for each of these
stories to be produced. Business decides which of these are vital and
should make the cut for a first release, and which ones can come later.
The level of detail here is very coarse -- stories allow developers
just enough information to make a good estimate of how long it would
take to code the software.
I perceive these stories as being quite similar to use cases, where the
customer explains how they think they would like to use the software
produced, along with some basic background material to explain how
things are tied together.
This phase can be considered a Forecasting and Estimating phase. It is intended to be quite short -- usually less than three days.
Development is conducted as a sequence of iterative cycles that
concludes with a product release. There are usually a couple of cycles
before production. At the beginning of each cycle a there is an
Iteration Planning Game (IPG), which is finer-grained than the Planning
Game and focused on the software to be produced. During the IPG customer
stories are split into tasks and developers choose which tasks they
would like to implement. Then, estimates are given by developers on the
stories they've chosen. Each developer calculates his or her load, and
some reallocation can be done to balance out this load.
The next step is actually producing some of the software that will fit
to the stories. Every software piece is test-driven; before you write
the code you write a series of tests that should run if the software
does what it should do. One of the more controversial points to XP is
that software is to be programmed in pairs - two people to one computer.
Regression tests are run all the time to guarantee nothing is being
My understanding is that the pieces of software might not be directly
connected to a story during the initial cycles, primarily because the
infrastructure and communication architecture have to be created. In
later stages, however, most software will be directly linked to a story,
and the tests can run exactly as the story would specify a use case.
Once the stories targeted for the cycle end, the developers regroup and
another IPG is conducted. At this time the customer can sign the
completed stories and provide suggestions for the next cycle.
At the end of Development, a version of the product is released to the
customer and put into production. Only during the last weeks before
release should performance tuning be done. Once the software has entered
production, a new Exploration phase should start, and new features --
or features held off from the first release -- are to be considered.
Development concentrates Design, Coding, Testing and Release phases -- this is very much in accord to the `Coding is Everything' feeling XP has to it.
Integration is done periodically with very short intervals: all
components that make up the system should be assembled in a separate
server and integration tests are run on it. This tends to make releases
easier, since integration and packaging are being done all the time.
Integration tests that fail are to be serviced immediately and are an
overriding issue that preempts all work done by the team.
Since regression and integration testing are frequent, any developer
that wishes to alter any part of the software is free to do so; he or
she needs only to guarantee the software hasn't broken.
The philosophy that guides change and enhancement is `Do only what needs to be done to allow the task immediately at hand to be solved elegantly'. This means that no provisions for predicted need or features are encouraged -- problems should be solved as they come. Design is reduced to decisions made upon reaching a new task or fixing something proven wrong through testing. As soon as you reach a problem that can only be fixed by changing something, you can refactor it, making sure the tests still run.
I understand XP is targeted at projects where customers don't know
exactly how to proceed at the beginning and are likely to change their
mind -- software is produced quickly and development receives feedback
early in the development projects, so little planning is actually
wasted. If a project is done in-house and requirements changing is seen
as a constant reality, it seems very adequate.
This shortened release cycle is one of the primary strengths of XP.
This is indeed very positive: customers see what they paid for early,
even though incomplete, and developers stay more interested and on
track. More freedom for developers to choose tasks and to fix problems are
definitely additional important motivators.
Without becoming religious, I agree with the Worse is
Better2 mentality that Kent shows throughout
the book - simplicity should be paramount and keeping the system simple
but functional is the most important consideration to be kept when
scaling and changing. Since XP pushes to be early and always in
maintenance, you could say it is always scaling.
I see continuous integration as a very interesting idea for developing a
project. The idea that in short cycles everything is tested and packaged
makes progress comfortably visible and keeps developers enthusiastic
about the task at hand.
Worth noting is the fact that XP goes directly against Cleanroom
 development, where testing is always done by other
team members and developers are not allowed to execute their own code.
Cleanroom has been defended in several places , but has at times seemed awkward
to implement and enforce, specially with geographically disperse
Kent's arguments for extensive testing and integrating, and not separating developer's tasks, are quite reasonable. Keeping delivery simplified by always readying the product for release sounds good, as does letting developers see the whole product.
The one major implementation problem I see with XP is that it is a
difficult model to use when selling services to other companies, since
they either expect explicit requirements to be produced or at least want
a fixed price and time produced on day 0. Convincing customers to forego
these key pieces of information will be hard. I have chosen not
to discuss this in this document: the considerations that I list out
here are all technical, and have nothing to do with sales and
XP has a tendency to attempt to convince through shock. It denies many
practices that have been tried for decades, some of them in my
experience good ideas. Just because Software Development in general has
problems shouldn't mean that everything being done up to now isn't
Problem Analysis, for example, is never mentioned as something to be
done. Clients must have a good idea of the 'stories' they need to tell
at first. This could be easily countered with an Analyst's help up
front, before beginning XP, but it isn't mentioned as so during the
length of the book. The intent on iterating the process of composing
stories can be tiresome, since it occurs even before the first time
estimates are delivered.
Along the same lines are the problems the lack of a more specific
requirements engineering phase invites. As I see it, requirements aren't
a plot to plaster developers, but a phase during which the problem is
mulled over continuously and refined -- until a decent, simple solution
is decided upon. Thinking about the problem is necessary -- specially
because finding out what problem is to be solved and how to solve it
is hard , and should be hard. Furthermore, by not
spending a significant portion of time thinking before starting, a
chance to plan properly to avoid refactoring are wasted.
This is not to say excessive time should be spent on requirements, and
the subsequent design, specially since they will inevitably change;
however, a good idea of the overall problem to be solved and a good
agreed-upon path to the solution are always good head starts.
XP seems to be adequate for fast-paced projects during their
development. When the development process matures and less functionality
is requested than would be necessary to keep XP burning, there is no
provision for reduced maintenance. It seems projects that develop slowly
would not be properly tackled by XP.
The point described above may not be obvious, so I'll elaborate: little
or no documentation means the code and the people must be trusted with
knowledge of the project. Unfortunately, this isn't always a reliable
bet: it is common for some developers to quit a project (and this
is covered in XP), but occasionally all - or most
developers quit. With no one around to explain what the code means, the
barrier of entry to a new developer will be high. Documentation, if
properly maintained -- informal, if culturally possible, but
existent -- can help reduce this problem. Keeping it in sync is,
of course, a problem, one that XP claims to avoid.
XP is very trusting on the possibility of refactoring and retrofitting
existing code to extended functionality. This may be the case if all
changes are simple, but many of us have encountered significant problems
during code adaptation and extension, sometimes even with simple
changes. This is mainly because `simple' is a perceived value, and not
an objective one, when the matter is software development. Small changes
can impact seriously, and proper pre-coding planning could help
Regarding testing, if a large part of the project requires implementing
a user interface, it may be a problem for developers to create automated
tests. Studies show that usually 50% of all code is dedicated to user
interfaces . Obviously, tests can be handled
manually, albeit cumbersomely. This problem does exist in any User
Interface testing environment . However, XP
relies on testing to drive development; problematic testing can
mean slow (or problematic) development.
Finally, it seems that developers are trusted to be smart and willing. You could argue that anyone not smart or willing shouldn't be working at all, but every once in a while a developer will go through a bad phase or won't want to do a task at hand. In a group, this can occur quite commonly. A provision could be made -- voting, or a fallback rule -- for tasks identified that no one wants, or that are being executed poorly. This may come in conflict with the ideals of free choice and ownership that XP suggests.
As I've stated before, the early and frequent releases echoes the `Open
Source' development mentality. Since Free Software is in constant
evolution and maintenance (mainly because software isn't attached to a
strict schedule or requisite definition), many of the techniques used by
XP could be (and many times, are) adopted by projects.
As an example of constant integration, the Mozilla project comes to
mind, where nightly integration is done. Every night builds of the
current code base can be downloaded and tested by anyone who so desires.
They have an automated bug-tracking system where users can submit bugs
online, and tools for monitoring the build tree's status in real time.
An interesting part of XP -- test-driven development -- could be
exploited in a very interesting way in many Free Software projects. It
remains to be seen as to how successful one would be in enforcing
testing since the codebase is replicated and changed by all developers (and
Ownership of the code in Free Software projects is somewhat less
flexible, with the project lead decide what submissions go in or not. It
is important to note, however, that this has good analogy with XP tasks,
which are feature increments or enhancements. Tasks have an "owner",
which is who implements them and knows their impact best. Bug-fixes are
usually accepted without much discussion in Free Software; this is in
accord to the XP philosophy ``anyone may fix broken code''.
Ownership is a very motivating experience. Having a developer's name
attached to a piece of software and having it publicly visible is one
of the highly unperceived but very important characteristics that
interest and attract developers to Free Software. XP could explore this
to a better extent.
Since there is little formal, specific, planning, and most software
isn't produced for third parties' use, there isn't anything similar to
the Planning Game in the Free Software world. The Iteration Planning
Game (IPG) doesn't have a proper parallel either. The iterations through
major versions of Free Software projects3 in my opinion are too long to be considered
example of Development iterations.
Refactoring and design are quite similar to the process described
(`evangelized') in XP: refactor and design on demand. This is probably
due to the fact it is the only way to maintain systems that are already
in production and running: large changes that break everything wouldn't
be convenient at all. Not designing up-front, however, is -- in my
opinion -- a problem, both with XP and with current Free
The lack of real documentation (apart from the code, which could be
argued is the only `real' documentation) is quite similar, and it can be
pointed out that the real problems with it are the same: when a project
dies, and programmers stop working on it, it is difficult to work on it
again. Nobody knows or cares about how it works anymore. Documentation
that exists to provide an accelerated path to comprehension is a dire
necessity, in my opinion. This does not mean bureaucracy.
There is no parallel in the non-free world with the ``Right to
so it of course isn't covered in XP.
Finally, and obviously, programming in pairs couldn't be properly enforced with people developing constantly separated.
The book is quite interesting. It does exaggerate at times towards
breaking current practices, but it gives good explanations on why it
does so. In the Software Engineering field, in practice very bureaucratic
and systematic, it's a refreshing look into how simplicity and common
sense can bring advantages to small teams developing constantly changing
I find it very interesting that XP is focused on small teams; usually a
software engineering process invites so much overhead that any small
company doesn't even consider it.
The constant integration practices can be implemented easily and would
be to great avail to any business; free software projects with rapid
development speed should implement automated integration.
Some of the more controversial points -- programming in pairs comes to
mind -- can be hard to accept. For these, I'd suggest forgoing the
author's ``Anything but Everything is Not Extreme Programming''
attitude and adopting what feels right for you.
I would definitely not recommend avoiding Problem Analysis and Requisites Specification, no matter what XP would suggest. If your focus is on simplified and informal documents, fine; just make sure they exist. It is too easy to fall into the no-planning trap, and I trust not in pure refactoring and developer expertise to keep a system from becoming a mess.