A web developer’s workflow
A close friend of mine recently asked me for some advice in getting started with version control. He is a developer too and we have previously, in another life, worked at a web agency together, although this was not where we met and became friends. We had similar roles at this agency but have not worked together for a couple of years, but this conversation led to me realizing how our motivations had taken us down different paths and subsequently how our workflows had developed, and, how much my workflow had changed since we had worked together.
The email I sent my friend became quite lengthy and comprehensive so I thought it would be interesting to document these things and, for posterity have a record of the way I work at the moment to compare with the inevitable changes in my workflow that will come over the next few years. What I outline below is a workflow that has developed naturally for me and one that I’m sure will change in the future. Writing this, I quickly realized that this was the first time I had really analyzed the way I work and approach development.
Three long years ago
Starting with the way I used to work shows what led me to the way I work now. I can sum up my workflow from 3 years ago with one word – impatience! All I wanted to do was create stuff. This meant my workflow was geared around designing as soon as I could (a process I really enjoy), and then coding and developing that design into a website as soon as I could (a process I also really enjoy). So my process went like this:
- Open illustrator, select a web sized 72dpi canvas and start pushing pixels around.
- Send this to the client and moan a lot when I received a critique.
- Begrudgingly make the clients changes.
- Open Coda and start writing the code for all functionality and visual layers at the same time.
- FTP the entire site to test server
- After client feedback, make some changes, FTP back to the server.
- When ready FTP entire local copy to the final resting place of the site.
- Deal with permissions errors and server environment problems as the client found them.
To be honest much of this process is one that I have seen whilst working (and learning) in 2 different agencies and one I took for granted. It wasn’t until I went freelance that I started to think about better ways of approaching this – and of the drawbacks of the way I had been working.
My set up back then was on a Mac. I used MAMP to develop sites locally and then upload them to a “real” server later – mostly for client review, and then to the live server.
Problems and drawbacks
I find many problems with this approach but I feel there are 2 main problems. The first problem is fundamental to my attitude and about the way I approached the work. I was never approaching the work from a functional perspective. It was always focused on the way the site would look, instead of the way the end user would interact with it.
The second problem with my workflow was that it did not allow thorough browser, functional and code testing. It also did not allow for easy tracking of changes to code or allow multi-threaded development in the sense that I had to finish one piece of functionality before starting another. All this brought me to the conclusions that:
- Functionality should come first, the design second (without compromising design)
- I should be developing and testing in an environment that was as close to the live server a possible.
Changing the way I work
These are the 2 principles that drive my workflow now, however I did not just change my workflow over night. There were, and still are, a number of things that affect the way I introduce new techniques, software and processess into my workflow.
Firstly I like to learn, and I’m a geek. This means I try and push myself to learn new software, languages and techniques. Pure curiosity is often a motivator and after playing around with something I can see whether or not it will benefit me, or sit with the way I work or change the way I work for the better.
The biggest factor though is need, and the 3 largest changes to my workflow arose through a need to solve a problem or because I could feel there was a better way to do something I was doing.
This was the first thing to change. I knew that the way I was developing wasn’t allowing me to fully test things before deploying them to a test server or live server. I couldn’t test any scripts that sent email and I wasn’t using certain tools that I wanted to (such as ImageMagick), because they did not work in the development environment I was using.
So I got hold of a copy of VMWare and installed Debian. This was a hell of a learning curve but the pay-offs were massive. It allows me to run a fully functional web server, complete with mail server, apache, ftp server, imagmagick, mysql and absolutely anything else I wish. I have control over the php extensions I run and the versions of the software. I can create new virtual machines to match the specific specs of a server that a site will be deployed to, and I can map all the htdocs directories of the servers to local directories on my mac. I could (and may do oneday) even map these directories to DropBox directories.
This step gave me complete control over my development environment and allowed me to play (and learn) with the server in a totally safe environment. There is often a bit of a gap between a developer’s knowledge of the website he has created and his knowledge of the way the server where it lives works. I learnt so much about server management through doing this, and it has benefited me and my work enormously.
The other benefit was that I could now also natively run various operating systems on my Mac, I have Windows XP, Windows 98 Windows Vista, and obviously my native Mac OS X, running all major web browsers.
Prototyping and Sketching
To be blunt I’d become lazy with the way I designed. Maybe lazy is a bit harsh, I was too eager to start creating a fully polished design. My first few freelance jobs taught me a lesson here though. I always wanted to jump straight into Illustrator and turn out a beautiful finished design in my first attempt. I would spend a lot of time on this and inevitably the client would want to make changes which would frustrate me. The problem was that I wasn’t actually thinking about what the client wanted and how the user would use the site.
So instead I turned things on it’s head a little bit. Now I start by taking a pencil and notepad and I sketch layouts with notes about functionality. This allows me to burn through 5 – 10 ideas in almost as many minutes. I can quickly see which ideas have legs and which I should throw away. It is a more dynamic process that itself generates more ideas. The more I sketch the more refined the ideas become but I still only keep it to layout and functionality processes, sometimes expanding on specific pieces of functionality such as forms and sketching in more detail.
Once I have a functional design sketched out I then build a prototype. This allows me to test the functionality without any of the distraction of the design. It lets me get a better idea of how things will/can work and allows me to change the functionality easily as I work. It also means the functionality leads the way the page / element will look, which is an important point.
At the point of a functional prototype I let the client review it (with a large disclaimer that this is not the final version of the website and that no, the entire site is not going to be black and white). Once approved I then design the style of the site around this functionality. It makes the process much more intuitive and connected to the functionality instead of having the functionality merely bolted onto a design.
It’s important to note that not all of the site needs to undergo this process. I’m talking about particular pages and processes (shopping baskets, checkout flows, search functionality etc).
It may sound like a more long winded approach and one that takes longer, but in fact it doesn’t take much longer. You spend less time with amends from the client and it saves you time in the future when you are building the site as all the details have been thought about and ironed out earlier on. Prototyping itself is massive subject and there is loads written about it.
Implementing version control into my workflow has probably been the hardest and most forced change to the way I work. At first I wasn’t really sure why I was doing it. I couldn’t see the benefits and was only really learning it because it seemed like something that was a big deal and I was eager to understand. However it soon became apparent why it is so beneficial.
I was working on a large pre-existing site that already managed it’s code base with version control. I was not the only developer working on the site and there was both bug fixing and new features being implemented at the same time. In my old work flow if I was working on some new features and a bug was found with the existing code, the newly discovered bug would need to be fixed immediately, but I already had a codebase on my local machine that had incomplete code that couldn’t be uploaded to the live server. This means trying to manually track which files you have changed before you attempt to fix the bug and hope that the bug is not in one of the files that you have already changed. This also means that working with other people on the same project was difficult because it meant that multiple development copies of the same site would need to be kept up to date manually, with all developers communicating changes to each other and constantly downloading new files and updates. Version control literally solved all these problems and it is exactly what it is intended for.
There is one other major benefit to using version control that I discovered later as I read more and begun to understand some of the concepts around developing with version control. The benefit comes in not only maintaining the codebase in development copies but also in production and staging servers. I won’t go in to details here as I have decided to write more on this in another post, but essentially it comes down to the way you organise your repository and manage the production servers. You can ditch ftp completely and use a working copy of the code base on the production server.
This has been a fairly lengthy post and congratulations if you made it to the end! As I said at the start, this is as much for my own reference and posterity than anything else. However I have found it really interesting to analyse the way I work. There is lot’s more I could talk about such as unit testing, build scripts, and certainly more on version control, but I will leave that for another day.
- Pop-up Eiffel Tower, Bath Abbey and bi-lingual wedding invitation
- Using Selenium Server 2 and PHPUnit to automate browser acceptance testing
- A practical application of subversion for web development
- Sandie Harding, Beautiful Book Repair & Conservation
- WordPress 404′s when a custom form is submitted
- Brand New Website for Onemind.co.uk
- Windows Phone 7 Microsite