Here at BPM Logic we really like Agile principles and methodologies, proselytise about them and use them on our projects (not for absolutely everything but a lot). I plan to write a lot more about Agile in future, how we apply it here and some of the experiences we’ve had and learned from. First off though I thought I would say a little about how I came to Agile in the first place.

So I’ll skip all the David Copperfield stuff, but my first IT job was at Cap Gemini where I started out as a developer and consultant, before gravitating towards leading technical teams. This means I probably came to my later jobs in Project Management with a more pro-developer mindset than many PMs. My belief has always been that motivated professional people want to do a good job, you just need to remove the barriers that are stopping them from doing this, and let them do their job.  A command and control approach doesn’t sit well with me and generally this doesn’t seem to foster an effective and motivated workforce. CG was a very big and varied company with a lot of great and talented people doing great things, but clearly back in those days I could see there was something wrong when I encountered monolithic projects that moved slowly for months producing work which then got canned as not fit for purpose.

I became quite interested in the question of what made software projects work or not work. I was lucky enough to get involved in some of the very early work on Sharepoint with people from Microsoft and had great fun visiting Seattle.

There I came across MSF which whilst not strictly Agile in itself has principle that got me thinking along Agile lines.

1. Foster open communication

2. Work towards a shared vision

3. Empower team members

4. Establish clear accountability and shared responsibility

5. Focus on delivering business value

6. Stay agile, expect change

7. Invest in quality

8. Learn from all experiences

As well as engineering practices like regular build and testing cycles.

I tried to then take some of these principles to my later jobs as I thought about, and experimented with, what made effective software projects.

It wasn’t until a later job at uswitch.com I came across Agile, Scrum and XP properly. At the time they were still using a very waterfall model of development with heavy up front design. This could be made to work for them where they were doing projects that involved building new product calculator products which followed similar schedules to previous products. However without going into too much detail, I ended up being responsible for a very technically complex project different from those that they had done before.

Meanwhile there was a debate going on in the company about the adoption of Agile. Obviously this involved a massive cultural shift and what this means for companies, and how you might approach this with customers, is something I’ll come back to in a later post.

Whilst my project was only in the end a semi-Agile project, it was clear to me that without applying Agile and XP principles backed up by a technical lead with good experience and a strong belief in XP it would never have succeeded in anything close to the desired schedule. For me the proof was in the pudding and it convinced me of the power and benefits of Agile. I should also point out that getting to the start of development on this project had taken at least twice as long as the project itself took to deliver so a more Agile approach to requirements would clearly have been beneficial. Later the company then moved on to a Scrum methodology and I believe they are still using a version of this that they are all very happy with.

It’s taken us a long time at BPM Logic to get Agile working in a way that runs smoothly for us and we’re definitely still learning but we like to think we’re doing okay. I’ll write more about some of the things we do and what we’ve learned in the future.

It occurred to me the other morning that life would be a bit easier if SQL was a bit more like Latin. And I don’t mean that it should include the subjunctive or the ablative absolute because a) I can’t remember what those things are and b) having just looked them up on Wikipedia they wouldn’t really be of any use whilst querying a database.

Instead, I wish that SQL was a bit more permissive when it comes to phrase order.  Here’s a popular Juvenal line:

quis custodiet ipsos custodes

which, if we were to follow the translitteral word order when being trans-littoral, would read:

who guards themselves the guards?

Juvenal was writing at the end of the 1st century when Latin, whilst not dead, had certainly been around for a while so he could get away with an (appropriate) amount of poetic licence.  It would be more common in Latin to see the verb at the end of the sentence:

quis custodes custodiet

How the hell does this relate to relational databases?

Everyone knows that you should never have unrestricted access to a production database. In certain circumstances it might be acceptable to run SELECT queries but obviously no-one in the right mind would ever allow anyone console access to a production database with permission to run UPDATE or DELETE statements, let alone DDL commands. At BPM, our standard practice is to use an automated patching tool called AutoPatch which, in conjunction with CruiseControl for continuous integration, allows all such statements to be tested on both a seed database and a recent copy of the production database. That way we can apply changes to our persistent storage as part of the deployment process and be certain that it’s going to work beforehand.

FROM force SELECT * WHERE NOT type = 'darkside'

So, last week I was running an UPDATE statement on a live production database using the psql console tool (yes, bad Luke) and it occurred to me that it would be so much nicer if I could type in my WHERE clause first. That way I wouldn’t forget to enter it and accidentally update the ptfname field of every row in the table to “Chaz and Dave”*. It got me thinking that perhaps the clause order should really be irrelevant and if I really want to write SQL like Yoda then I damn well should be able to write SQL like Yoda:

WHERE age = 900 FROM jedi SELECT *;

Occasionally it would make sense to write your GROUP BY clause just after the SELECT clause as they’re usually very similar. When writing a long statement that involves multiple joins you will end up writing the SELECT clause last, because it’s only by then that you know the table aliases that you’ve defined.

What’s the catch? (or: why hasn’t anyone done this before?)

I can spot two obvious downsides to this, the first being that you will inevitably have a performance hit as the lexer will have to do more work. Logically the time it takes to parse the statement will increase if the statement is more difficult to parse. The second downside is that it will make your SQL statements more difficult to parse by other people. It’s bad enough having to put up with Java allowing people to put opening braces on new lines without having to cope with Jeffrey, the guy who insists on putting the OUTER JOINs at beginning and the INNER JOINs at the end. It won’t be long before I’m up the water tower with a sniper rifle.

It’s here that my Latin analogy breaks down somewhat. As PostgreSQL is an open source application I can change the way that it parses SQL and if I really wanted to I could even replace all the words it uses. Or I could add a pre-processor to reformat my garbled input into a sanitized SQL-compliant version. In fact, as with a lot of things in the programming world, someone’s probably already done this.

We were told at school that Latin is a dead language, so it won’t be changing any time soon. However it may not be as dead as I was led to believe:  Latin has influenced modern languages all over the globe and you still find Latin words and phrases in common usage in everyday English – all too often I find myself reading an agenda for yet another meeting. You can even browse a popular social networking site in Latin so I think we can assume it will be sticking around for some time.

quod erat demonstrandum

[ * My very own TheDailyWTF moment was when I did this on a production database but fortunately a particularly complex trigger kicked in and the statement timed-out and rolled-back before causing havoc, embarrassment and potentially my dismissal. ]