Here at BPM Logic we really like Agile principles and methodologies, proselytise about them and use them on our projects (not for absolutely everything but a lot). I plan to write a lot more about Agile in future, how we apply it here and some of the experiences we’ve had and learned from. First off though I thought I would say a little about how I came to Agile in the first place.

So I’ll skip all the David Copperfield stuff, but my first IT job was at Cap Gemini where I started out as a developer and consultant, before gravitating towards leading technical teams. This means I probably came to my later jobs in Project Management with a more pro-developer mindset than many PMs. My belief has always been that motivated professional people want to do a good job, you just need to remove the barriers that are stopping them from doing this, and let them do their job.  A command and control approach doesn’t sit well with me and generally this doesn’t seem to foster an effective and motivated workforce. CG was a very big and varied company with a lot of great and talented people doing great things, but clearly back in those days I could see there was something wrong when I encountered monolithic projects that moved slowly for months producing work which then got canned as not fit for purpose.

I became quite interested in the question of what made software projects work or not work. I was lucky enough to get involved in some of the very early work on Sharepoint with people from Microsoft and had great fun visiting Seattle.

There I came across MSF which whilst not strictly Agile in itself has principle that got me thinking along Agile lines.

1. Foster open communication

2. Work towards a shared vision

3. Empower team members

4. Establish clear accountability and shared responsibility

5. Focus on delivering business value

6. Stay agile, expect change

7. Invest in quality

8. Learn from all experiences

As well as engineering practices like regular build and testing cycles.

I tried to then take some of these principles to my later jobs as I thought about, and experimented with, what made effective software projects.

It wasn’t until a later job at uswitch.com I came across Agile, Scrum and XP properly. At the time they were still using a very waterfall model of development with heavy up front design. This could be made to work for them where they were doing projects that involved building new product calculator products which followed similar schedules to previous products. However without going into too much detail, I ended up being responsible for a very technically complex project different from those that they had done before.

Meanwhile there was a debate going on in the company about the adoption of Agile. Obviously this involved a massive cultural shift and what this means for companies, and how you might approach this with customers, is something I’ll come back to in a later post.

Whilst my project was only in the end a semi-Agile project, it was clear to me that without applying Agile and XP principles backed up by a technical lead with good experience and a strong belief in XP it would never have succeeded in anything close to the desired schedule. For me the proof was in the pudding and it convinced me of the power and benefits of Agile. I should also point out that getting to the start of development on this project had taken at least twice as long as the project itself took to deliver so a more Agile approach to requirements would clearly have been beneficial. Later the company then moved on to a Scrum methodology and I believe they are still using a version of this that they are all very happy with.

It’s taken us a long time at BPM Logic to get Agile working in a way that runs smoothly for us and we’re definitely still learning but we like to think we’re doing okay. I’ll write more about some of the things we do and what we’ve learned in the future.

It occurred to me the other morning that life would be a bit easier if SQL was a bit more like Latin. And I don’t mean that it should include the subjunctive or the ablative absolute because a) I can’t remember what those things are and b) having just looked them up on Wikipedia they wouldn’t really be of any use whilst querying a database.

Instead, I wish that SQL was a bit more permissive when it comes to phrase order.  Here’s a popular Juvenal line:

quis custodiet ipsos custodes

which, if we were to follow the translitteral word order when being trans-littoral, would read:

who guards themselves the guards?

Juvenal was writing at the end of the 1st century when Latin, whilst not dead, had certainly been around for a while so he could get away with an (appropriate) amount of poetic licence.  It would be more common in Latin to see the verb at the end of the sentence:

quis custodes custodiet

How the hell does this relate to relational databases?

Everyone knows that you should never have unrestricted access to a production database. In certain circumstances it might be acceptable to run SELECT queries but obviously no-one in the right mind would ever allow anyone console access to a production database with permission to run UPDATE or DELETE statements, let alone DDL commands. At BPM, our standard practice is to use an automated patching tool called AutoPatch which, in conjunction with CruiseControl for continuous integration, allows all such statements to be tested on both a seed database and a recent copy of the production database. That way we can apply changes to our persistent storage as part of the deployment process and be certain that it’s going to work beforehand.

FROM force SELECT * WHERE NOT type = 'darkside'

So, last week I was running an UPDATE statement on a live production database using the psql console tool (yes, bad Luke) and it occurred to me that it would be so much nicer if I could type in my WHERE clause first. That way I wouldn’t forget to enter it and accidentally update the ptfname field of every row in the table to “Chaz and Dave”*. It got me thinking that perhaps the clause order should really be irrelevant and if I really want to write SQL like Yoda then I damn well should be able to write SQL like Yoda:

WHERE age = 900 FROM jedi SELECT *;

Occasionally it would make sense to write your GROUP BY clause just after the SELECT clause as they’re usually very similar. When writing a long statement that involves multiple joins you will end up writing the SELECT clause last, because it’s only by then that you know the table aliases that you’ve defined.

What’s the catch? (or: why hasn’t anyone done this before?)

I can spot two obvious downsides to this, the first being that you will inevitably have a performance hit as the lexer will have to do more work. Logically the time it takes to parse the statement will increase if the statement is more difficult to parse. The second downside is that it will make your SQL statements more difficult to parse by other people. It’s bad enough having to put up with Java allowing people to put opening braces on new lines without having to cope with Jeffrey, the guy who insists on putting the OUTER JOINs at beginning and the INNER JOINs at the end. It won’t be long before I’m up the water tower with a sniper rifle.

It’s here that my Latin analogy breaks down somewhat. As PostgreSQL is an open source application I can change the way that it parses SQL and if I really wanted to I could even replace all the words it uses. Or I could add a pre-processor to reformat my garbled input into a sanitized SQL-compliant version. In fact, as with a lot of things in the programming world, someone’s probably already done this.

We were told at school that Latin is a dead language, so it won’t be changing any time soon. However it may not be as dead as I was led to believe:  Latin has influenced modern languages all over the globe and you still find Latin words and phrases in common usage in everyday English – all too often I find myself reading an agenda for yet another meeting. You can even browse a popular social networking site in Latin so I think we can assume it will be sticking around for some time.

quod erat demonstrandum

[ * My very own TheDailyWTF moment was when I did this on a production database but fortunately a particularly complex trigger kicked in and the statement timed-out and rolled-back before causing havoc, embarrassment and potentially my dismissal. ]

It occurred to me that there were 3 notable launches last week.

On Wednesday, Seth Godin announced the launch of Brands in Public which describes itself as a collection of interesting, accessible, public-facing dashboards for your favorite brands. Each dashboard is a Squidoo Lens, aggregating content and opinions from other sites such as Wikipedia, Amazon, Flickr, Yahoo, Twitter and the Blogospere in general. Squidoo doesn’t purport to have created any of the content itself, it’s merely a convenient way of viewing (viz. a lens) this information in one place. All the content would be easily discoverable by going to a few specific websites and spending a bit of time on Google.

The response was largely positive but a certain amount of backlash prompted a follow-up from Seth at the end of the week, the upshot being the some of the Lenses were removed from the site. In Seth’s own words:

“other people didn’t like elements of it. And they were direct in letting me know.”

Meanwhile, I was alerted to the relatively innocuous launch of Google Sidewiki by a twitter post from Jeremiah Owyang.  Having emerged from Google Labs, Sidewiki “allows you to contribute helpful information next to any webpage” if, that is, you use Firefox or IE with the Google Toolbar installed.

Internet users without the requisite toolbar can still see Sidewiki content as it is simply just HTML content hosted on the Google site. For example, Jeremiah followed up his post by contributing a Sidewiki entry of his own. I think that we can expect to see Sidewiki content appearing in Google search results at some point in the near future.

The most important consequence of this is stated by Jeremiah:

“recognize that you don’t own your corporate website –your customers do”

In essence, Sidewiki allows any other internet user to post content on top of your own website. For everything that you post online, Google has kindly provided a built-in comments facility whether you want it or not.

The cynic in me would argue that this came about in an attempt to usurp Twitter from its current top spot as the commenting (and moaning) medium of choice. Otherwise, why else would Google release an API so that third-party developers can leverage the content posted on Sidewiki? You can tell it’s helped Twitter along by the mryiad applications that connect with their platform.

This leads me on to the third launch of the week. OneForty is “A Better Way to Discover Twitter Apps” although I’m not quite sure why “to” is the only word not to warrant capitalization. What it boils down to is an App Store for Twitter – a place for application developers to promote their apps and for Twitter users to review them.

It’s remarkably easy to use, in part because it uses the OAuth protocol to seamlessly connect you to the Twitterverse, but also due to a very well put together site. I would like someone to explain to me exactly what their favicon is meant to be though.

The response to OneForty has been overwhelmingly positive. With over 1500 apps leveraging the Twitter platform it was both difficult for users to discover ways to get the most out of their Twitter account and even more difficult for app developers to promote their product to the right people.  OneForty allows app developers to say “we have a product” and then people can say “we love it” or “we hate it” or even write a mini essay critiquing its finer points.

You can see it’s not a million miles away from what Sidewiki set out to achieve so one wonders why the latter received so much negative publicity. My guess is that it’s the same reason why the objectors to Brands in Public were quick to have their pages removed:

No one wants to hear bad things said about them.

They don’t want to have a conversation that they can’t control so it leads to a one-sided adoption of social media, but leaving out the social part. A marketer that does this becomes the bore at the party – only talking about themselves. It’s no wonder that there is so little trust for corporate blogs. It’s a shame that they don’t take a leaf out of the other two sites. Seth Godin is an evangelist for permission marketing, and the point of Brands in Public is to provide a vehicle for corporate entities to see what the Internet is saying about them and then respond accordingly. Seth’s response to those that don’t want to be a part of it is understandable – we’ll just shut you down and make room for those that do want to be a part of it. At the other end of the spectrum in OneForty, where the brands themselves are embracing the opportunity to be talked about and to engage their customers in conversations. Hell, OneForty itelft tried to engage its customers by providing its very own Sidewiki in the form of a UserVoice forum, which allows users to directly comment on the site they are using. The best thing about this is that OneForty has and will listen to the feedback that they are given.

I don’t know if Google Sidewiki will be a good or a bad thing for the Internet, but I do know that it will move the goalposts because there isn’t a brand big enough to convince Google to take the facility of their individual website, and as a consequence corporate brands will have to follow Jeremiah’s advice:

“Don’t be reactive to negative content – embrace social content now.”

I hope they do, as all too often the standard corporate response to something that isn’t part of their programme is:

“Shut them down; or shut them up; at least shut the door and hide under the desk.”

and at the end of the day that benefits nobody.

Just round the corner from my flat in Belsize Park is an impressive former church which I occasionally see Japanese tourists taking photos of. I discovered a while ago that it had been the recording studio where Pink Floyd recorded “Dark Side of the Moon”.

This week I found out that it is apparently still in use as a studio, and that is where there was a meeting of such music industry and internet policy luminaries as Lily Allen and that bloke out of Radiohead.

Their conclusions were one step ahead of the government at least, in that they poo-pooed the idea of having the internet accounts of file sharing ‘perpetrators’ blocked, although the idea of ’squeezing’ them is perhaps sillier. However the nature of their and the government’s debate on the subject shows how flawed our approach is to policy and the internet.

The application of these restrictions is impractical and a burden which service providers are hardly keen on. This isn’t the government being supportive of companies which are supposed to provide the critical infrastructure for our new digital economy.

Inevitably any attempt to do this will result in accidentally targeting the wrong people (my brother in law’s work at the Institut d’Astrophysique de PARIS was nearly shut down by a similar scheme which detected his lab was sending large deep space recordings over the internet)

Anyway anyone who is more than a casual computer user and file sharer will find ways around whatever monitoring is put in place.

The more fundamental problem is the focus on attempting to prop up a failed business model and ignore the real challenges ahead. It’s a con that artists are the ones will suffer if we don’t do this, it’s the record company industry which has been ripping off consumers and artists for decades who will suffer, and if they have to cut down on ‘flowers for the model’ it means that money is actually more likely to go to artists or be recycled into other areas of the economy.

Instead we should be giving support to attempts to monetise these inevitable developments. The fact that it is major labels who hold the biggest stakes in Spotify shows that despite their protestations they at least can see which way the wind is blowing.

This hang up on the music industry and it’s prime place in the recent Goverment Digital strategy report also shows completely the wrong approach to supporting the digital industries that might actually get us out of the multi billion £ hole we are in the UK. All it does is introduce extra tax, quangos and legislation of no benefit whatsoever to industry in the broad sense or the individual. Nevermind the worrying continuing encroachment of state sponsored snooping.

There is no focus on doing things that might actually encourage innovative businesses in UK and make the most of the great intellectual capital we have in this country. If these do happen here rather than over the Atlantic then it will be inspite of all the structural barriers that exist here in comparison.