Category Archive

If you find that you are stepping outside of your product development process on a regular basis, you need to do one of two things:

1. You need to enforce the process more stringently and educate all team members in how to follow the process;

2. You need a different process, my friend.

Welcome to the world of impedance mismatch. Over the years, I have seen a *colossal* amount of effort poured into smoothing out the mismatch with #1. If you’re sitting there with a full Programme Management office and a three page Change Request document and are still having to subvert the process to deliver results, you’ve reached the end of the road for the current process. But, the problem comes after you take the plunge with #2 and find yourself unexpectedly back at #1 after a while. There is a point of diminishing marginal returns, a point when it’s no longer a matter of continuing to educate everyone in the new process (they get it) or locking down workflows (the workflows are tuned). Instead, it might be the case that the new process isn’t the right one.

Crucially, this doesn’t mean that the new process wasn’t the right one at the start. Analyse the reasons for the turbulence: at the beginning this was likely to be fear-of-the-new issues, however it’s entirely possible that the organisation has evolved new issues as it becomes used to working in a new way. Moving from Waterfall to Scrum might have solved 90% of your problems but it’s not the end of the road: Scrum may only be a staging post to something else.

In Agile at Westfield, I covered how Westfield employed the Scrum process, noting that a key component is the ability to inspect and change the development process from the vanilla ‘out-of-the-book’ Scrum implementation to better interlock with the organisation, commenting that ‘No battle plan survives contact with the enemy’. This has proved to be true.

Since writing about the second anniversary of the Scrum process at Westfield we entered a crunch period in the run-up to Christmas, fitting in, among other things, launching a mobile site, adding three new payment methods and the re-engineering of the checkout experience to allow payment to multiple retailers with several payment types all in the one page. This was a high-risk strategy and we were forced to make compromises on our product development process to hit the deadlines. Some bugs slipped through, but mostly it all held together, and we spent a couple of sprints patching up the holes. There was a conscious decision to pile up technical debt and dial back test coverage (for reasons covered below), effectively taking calculated risks to hit the deadlines.

Before settling back into a comfortable Scrum rhythm of 3-weekly iterations, in which the last four sprints were a bad dream, one question that needs to be asked is: was the recent period really an aberration or were we witnessing the evolution of the organisation to a new ‘normal’? If the latter, how do we cope? What could we learn and put in place before finding ourselves back there again? Here are my takeouts….

1. The piper must always be paid: you cannot run development at 150% and not expect to rack up tech debt. Especially in software development, there is no such thing as a free lunch; it was acknowledged across the business up-front that the short-term race to the finish would need following up by sprints with very little business benefit in them.

2. Test code dwarfs the production codebase by a 3:1 ratio. It is insane to not think that tech debt piling up in your test codebase isn’t real technical debt and it doesn’t need to be paid down. Thanks to hidden tech debt, our checkout refactor fell foul of brittle, badly coded tests, leaving us with no recourse but to drop test coverage by disabling tests.

3. It is possible to release without certainty that the code is production ready, in special circumstances. If you have tested the important pathways (e.g. payment) thoroughly you can be reasonably sure that any bugs that slip through will be in less critical areas. You can then watch the system in production and patch as necessary with low impact to customers.

The last point turned out to be Pandora’s box: traditionally, skimping on QA releases all the software evils on the world, but there is also some good. You can scale back from ‘perfect’ to ‘good enough’ in your QA process only if you still have rigour around key processes and are able to release changes quickly. This means embracing Continuous Deployment techniques within a Scrum framework, and the two are odd bedfellows.

Scrum vs Continuous Deployment

At the heart of the Agile development methodology is the idea that you need to release a little, and release often. For teams coming at Agile off the back of nine-month-phased Waterfall projects, cutting down to monthly iterations feels blisteringly fast, but it allows a lot of the features of Waterfall to remain in the background, such as long UAT and release processes; it may be the case that development can start and finish on a feature within the same month, but that feature may face a tortuous journey to production from then on. If you then have pressure from the business owner to deliver on a fortnightly basis, you soon realise that you have an Agile process with a cumbersome Waterfall tail, where the set-up and tear-down costs for a fortnightly sprint are a large percentage of the total costs of the sprint. The time has come to optimise the workload downstream of Dev so that each feature can flow efficiently and continuously into production.

Continuous deployment requires the following:

* Continuous integration of the codebase via a farm of build servers, plus alerting on red builds and a philosophy of whoever breaks the build fixes it, quickly

* A clean source main line, so that if the builds are all green the codebase can be tagged for a deployment at any time

* A source management system that allows cheap feature branching so that all non-trivial work is kept away from the main source line until it has passed feature QA and is ready to be merged into the main line

* Automated QA to allow the deployment to be stress and regression tested, releasing human QAs to range freely across the application looking for trouble spots

* Automated, reliable, auditable deployment scripts

It’s not strictly necessary that each merge into the main line triggers a release process that deposits the code into production (after all, would you want code going straight out that was checked in during Friday drinks?), but rather that there is a capability of starting a release process at any time and having it ready for production by (at most) close of business the same day. Using this capability, fortnightly iterations become low-cost, low-burden, which seems to solve the problem.

Or does it? Scrum requires that the stories are lined up from an ordered backlog at kick-off and committed to as an ensemble. Stories must fit within the sprint and allow an increment of business value to be delivered. There is a point where shortening the sprint cycle to increase delivery responsiveness runs into the software management equivalent of the Planck length: the minimum size of a unit of business value. If your average per-developer velocity is 10 points for a three week sprint, dropping to a weekly sprint cycle means that all stories must be three points or less. If you have an atomic five-point story, it won’t fit.

An ideal scenario would allow stories to be injected into the sprint while also allowing velocity to be calculated and tracking of team commitment. In this world, the team commitment is a set of empty story slots that may be filled as the sprint progresses by the top-prioritised stories from the backlog. The Scrum concept of being able to visualise the entire commitment before starting work is still mostly possible by looking at the top stories in the backlog, with the understanding that priorities may change over time. While this may be a little less efficient from the developer’s point of view (in wanting to line up similar stories to get pipelining efficiencies) it means that the team is never engaged in wasted or lower-priority work.

Trivial stories and bug fixes can be committed directly to the main line, more complex stories (and even entire features) sit on their own feature branches waiting to be pulled back into the main line by the QA team, who now act as quality gatekeepers to keep changes off the main line until deemed stable and accepted by the business owner. Developer, BA, UX and QA resources are responsible as a team for taking a feature from concept, through visual design, development, acceptance, and finally release in one single sweep. This also has the advantage of the team being able to follow a story from cradle to grave (and into production) while the knowledge is still fresh (as covered off by @mootpointer on Continuous Deployment).

This sounds closer to Kanban than Scrum. The pipeline of story production that sits above the story development process in Scrum is extended down into the iteration and all the way out the other end into a tight DevOps integration that flows the stories out into the production environment. Scrum process components such as velocity can still be calculated retroactively at the end of the iteration period, to allow capacity planning going forward. The all-important Retro is still held at the end of each iteration. Regular backlog grooming meetings still occur. However, if the business needs to react swiftly to a new opportunity, there is no longer the harsh grinding of gears as stories are pushed out of the commitment, new stories brought in, sprint cycles potentially lengthened or reduced.

Should Westfield move to full Kanban? Not in the short term, due to the massive education and process re-engineering overhead across the business. But the journey of a thousand miles begins with a single step. Continuous delivery is that first step.

Some things just go together, like gin and tonic, or public holidays and improbably inaccurate short range weather forecasts, or jam and cheese (try it, you might like it). Another is a social media strategy and a mobile strategy.

Over half of activity on Twitter (and one third on Facebook) is read on a non-desktop device. It’s a good bet that the Like or tweet about your product or company was either sent from a mobile or will be read on a mobile, rising to a near certainty if it’s viral sharing between individuals.

So what? Imagine if you were running a traditional marketing campaign and half your ads were accidentally written in a foreign language… the cut-through rate would be pretty poor. Worse, if everyone else in the market did the same thing then you would probably be performing at industry average, which makes the problem invisible. So it’s important that, after going to great lengths to get that social strategy off the ground, you deliver people to pages that can be read easily on their device… making them wait 20 seconds while you send them the full 1MB desktop web page that’s then scaled into illegibility is unlikely to boost conversion.

Sites like eBay understand this. The pages most likely to be emailed, Liked, tweeted are mobile optimised, making the journey from a friend’s recommendation to a product purchase pretty seamless. By sewing together social and mobile you can make every click count.

I was in the U.S. for the Black Friday sales, a traditional day of discounted sales and shopping frenzy in the American Thanskgiving holiday period. We bought some real bargains over the weekend and hit the restaurants to celebrate with a lavish meal on Monday. Walking into a swish eatery to find it deserted I was informed in hushed tones… “it’s Cyber Monday. Everyone’s at home looking for bargains on the internet”.

Cyber Monday this year broke all records. American shoppers have been conditioned to expect deep discounting online on the Monday after Black Friday and in a virtuous circle retailers have learned that if they provide discounts on the day they can expect large volumes of online purchases. But beyond all that, it has created an engine that is rapidly teaching consumers that shopping online is good… after the frenzy has died down we’re left with the base figures: according to comScore a 26% year-on-year increase in online sales revenue for the weekend (effortlessly breaking the $1 billion mark), and over 50 million Americans shopping online (up 35% on the previous year). There is no reason to expect that this is anything other than a fundamental and irreversible change in customer behaviour.

Three players in particular stand to make the most of this new world and are busy laying out their strategy in something akin to the heady days of the first internet boom. Each is a seasoned survivor of those early days and each is coming at the ecommerce marketplace from different angles.

Firstly, there’s Google. They understand the power of search and have built a business on the back of monetising eyeballs through their ad service. They need to be at the start of every purchasing path to ensure those eyeballs don’t end up somewhere else. Google Shopping is their key play here; its main aim is to acquire and aggregate rich product data from as many stores as possible to facilitate the widest search. It’s interesting that they have also just announced that they will be releasing a next-day delivery service for retailers to tap into, leveraging their relationships with courier and express shipping companies. A third plank is Google Wallet which, integrated with the Android mobile platform, has the potential to turn every mobile into both a shopping device and a wallet. However, it’s difficult to see quite how these threads weave together for retailers and shoppers, even leaving aside the question of which elements are U.S.-centric. The killer app here will be whether Google succeeds in getting a rich product feed with inventory accurate to store level across a wide enough range of retailers. Turning up in store with a Google Checkout coupon on your mobile phone to take home there and then the product you already pre-purchased would be compelling, certainly.

The next player is Amazon, who have signalled intentions to encroach ever further into the traditional realm of bricks and mortar retailing. The purchase of Zappos demonstrates a clear direction towards moving more of the high street spending online; their feed integration infrastructure and APIs are also heavily geared towards letting retailers come onboard the Amazon platform as fulfilment partners, integrated through the Amazon checkout process (this is also the model that Westfield, broadly speaking, employs). This model lives or dies on the quality of its retailer interfaces since there is a heavy reliance on getting good product feeds, accurate inventory and timely fulfilment. This can be a hard nut to crack.

Then there is EBay. In retrospect, it looks like they have been heading steadily in one direction for quite some time but it was only with the recent XCommerce launch that their pitch for the future face of ecommerce became readily apparent. They have an urgent need to be known as more than just an auction house for garage sale items, an image that will be holding many retailers (for whom brand positioning is their prime asset) back from integrating with EBay; without a feed of quality, new items from respected brands EBay will have a hard time shaking their current image and grabbing a share of the e-tailing pie.

But let’s look at what they now have under their roof. Firstly, Red Laser to scan barcodes and launch a product page on the shopper’s mobile phone right there in the store. Then there’s PayPal (and its mobile-optimised express checkout) to collect the money. But that’s not really enough… they have brought Magento (an open source ecommerce site engine, poised to do for shopping websites what MySQL did for databases a decade previously) and GSI into the fold. This offers the retailers the prospect of transferring onto ecommerce platforms that run shopping sites, support checkout including PayPal, handle inventory management and other back-of-house functions with a small cost footprint. This provides the kind of cradle to grave shopping path that has Google worried, but their approach goes further with the introduction of the Fabric, an internet-scale service bus that allows all the potential players in ecommerce to tie together to provide pluggable services to retailers such as fraud management modules, warehouse management systems, payment gateways, etc. It goes without saying that PayPal, Magento, etc. are part of the fabric from day one and thus already providing sufficient Lego blocks to build a viable ecommerce business upon. If they succeed in building up an interoperating ecosystem of ecommerce modules, EBay could quite possibly become to online shopping what Facebook’s app universe did for social networking.

So, we have the underlying barrier to be overcome: all the product feeds, smart searches, payment systems, slick checkouts and value added services in the world are useless without the retailers’ ability and willingness to fulfil. Amazon has the edge here, being an integrated retailer already with deep experience across the entire supply chain. But the model can only scale so far before it meets the likes of Walmart and Tesco coming the other way with their deep, world-scale logistics experience, all battling for top spot in global consumer sales.

The bigger (but tougher) pie is all about clipping the ticket on purchases made with other retailers rather than seeking to act as the retailer directly. Google has many of the required elements and a track record of bucking conventional wisdom to achieve the apparently impossible, but creating an integrated supply-chain-as-a-service could be a stretch even for them.

This leaves EBay. Their apparent strategy of supporting systems and integration points to help the various players in the ecommerce space come together to service the customer means less revenue for them per sale. It has potential to be a lot more sales though. Out of the three, they may be just nibbling around the edges of the pie.

But that particular pie is vast.

Back when I was a lad, good things were Ace. Later they came to be described as Wicked instead. Then Bad, then Sick, then (apparently to differentiate from merely sick) Fully Sick. I stopped tracking the phenomenon at that point, so I have no idea what the kids use now, though I suspect it’s constructed completely of consonants in capital letters.  It’s as if each new teen wave felt the need to re-imagine language again, and there are I’m sure reams of literature in the psychology press about tribal identities and group-inclusion.

The same is true of software development.  Following a tweet stream on physical architecture (you know, buildings and stuff) suddenly illustrated to me what software development most likely looks like from the outside: it’s a terrain of impenetrable jargon, presided over by tribal units with their own dialects.  No wonder your 10 o’clock product review meeting with the sales team went off the rails; they probably had a hard time understanding what you were talking about.

There’s no doubt to me that this is how some of the tribes like it, and it is definitely what makes Agile more difficult to adopt than it should be for business people.  Imagine you have been posted to a foreign country as their ambassador, just got to grips with their language, and then all of a sudden they change the dictionary on you.  Having mastered terms like project phases and use cases, you’re suddenly dealing with sprints and stories.  Add to this the various flavours of Agile (XP? Scrum? Kanban? Huh?) and it can feel like you’re back to square one, so you have to really want to do it to make it successful. This is also not helped by contingents who use Agile as a cover for another methodology: JFDI. Agile and cowboy programming are poles apart, but under the cover of the jargon the latter has passed for the former on more occasions than are healthy for the former’s reputation.

My first reaction to Agile was, “why have they made new names for things?”, and it felt like when I realised that “Ace” wasn’t cool anymore (dammit).  The reason for the new vocabulary is to avoid bringing in a set of old assumptions from Waterfall and other places – it’s an attempt at a clean slate with no preconceptions. But it also means there’s a steeper learning curve because it purposely veers away from the possibility of homely similarity to what you knew before.  I’m not about to suggest a new set of names for parts of the Agile process… people that are tempted to do that should first bask in the impromptu wisdom of XKCD’s view on standards, but I am saying we need to recognise the workload we impose on newcomers.

In the end, it’s in all our interests to bring everyone along on the journey and put some distance between Agile and JFDI. But the next time you’re explaining to Customer Support about their request needing to be estimated using fibonacci-based points poker to work out velocity for the backlog, just take a deep breath and maybe go over it again.

Scrum as a fully-fledged methodology has been around since the Nineties, as an idea since the Eighties, and has sired many offspring in that time. In some organisations these offspring are given a test straight out of ancient Sparta, where legend has it that newborns were left on a roof to fend for themselves overnight to ensure that only the strong survived. Fast forward two thousand years and a lot of Scrums are also left to fend for themselves and die within a few sprints.

The Happy Birthday goes to the Westfield scrum process. I note that with sprint 42 starting shortly we are exactly two years since sprint zero. It’s fair to say that our scrum process has had to evolve quite a bit since sprint zero, and our organisation with it, so I think it might be worth taking a snapshot at the end of its second year of life.

At the very top level we have a bunch of people coming up with good ideas all the time. Some of these are very large, such as rolling out the platform to the US, some are much smaller, like ways to boost product search indexing. These ideas can come from anyone: the head of the company, a developer, a help desk person, etc. Each is discussed on its merits and if enough people agree (we have a weekly review meeting) it finds its way to the start of the product funnel. An idea’s progress through the funnel can be best described as Darwinian: only the stronger ideas push through to the next stage where they form a cross-disciplinary feature team, where strength is measured in the number of people who care about the idea enough to see it progress.

Once a feature has been created, it is assigned to a cell and stacked in priority order to decide what that cell will work on next, with the possibility that a strong new feature can come to the front of the stack and displace other features. A cell is the group of developers, BAs and QAs who will turn the feature into a set of Epics and break these down into Stories, and then schedule these into the product backlog assigned tentatively to a future sprint.

The way we currently work is that we have four cells of usually around four people, which is looking like a good number: intra-cell communication is pretty efficient and the team is still large enough to deliver a substantial new increment of the feature every sprint. We have settled on a three week sprint since this has yielded higher throughput than two weeks (less time spent on sprint set up and tear down ceremonies, plus it’s long enough to tackle a big vertical stripe – which becomes more important if you have an application architecture of more than a few interconnected services).

The cell concept is also scalable. Individuals can and do move between cells to promote intermixing of ideas, which makes it easier to spawn a new cell if the development pipeline needs to be widened: the new cell can be picked from existing members of other cells, backfilling with new hires, or even outsourced. The crucial thing is that the new cell adheres to the same communications processes as the existing cells so everyone still has an idea of what’s going on across the entire team.

However, scaling cells leads to the standard problem of how to keep everyone informed: inter-cellular communication is a challenge. We came up with two ways of solving this, having discounted the traditional Scrum of Scrums concept since this only involves a subset of the team (we still find it useful to do a SoS that pulls in representatives from the development team alongside customer service, retailer management, etc. but by its nature the SoS only dives into the salient highlights). One idea was to have a single scrum with everyone saying their piece, which only works up until a certain size after which individuals tend to zone out and miss details. The approach we finally settled on was to form an inner circle of cell representatives (an honour that rotates through all cell members) to give their cell’s status, surrounded by an outer ring of everyone else in the team. This lets the entire team dig into particular issues if needed without the stand-up taking fifteen or twenty minutes.

The backbone of the process is JIRA, which has evolved from a bug management tool into a highly flexible story manager through the Greenhopper plug-in. Part of the problem in the early days was that story and sprint planning was entirely divorced from story management, the former being done in an Excel spreadsheet, the latter in a bug tracking tool, leading to the potential for endless confusion and miscommunication. We brought in JIRA and customised the workflows to add in the approval steps our organisation needed and in short time had the business people talking the same language as the development team, planning sprints as easily as dragging and dropping stories from one sprint to the next. Sure, the software costs money, but next time you’re in the middle of a crisis meeting on who committed to what, when, just jot down the collective cost of having all those people in the room for the hour. It breaks even pretty quickly.

As I said previously on Westfield’s Agile approach, it’s not necessarily the steps of Scrum that matter (though there are fundamentals that you need to keep), but it’s Scrum’s introspective ability to evolve itself to fit the organisation. I would now add its ability to radiate information effectively as a feature of successful Scrum. After all, what’s the point of being in a team and being in the dark?

Last week, I got into a robust discussion at work, some might even classify it as an argument, and looking back it was ultimately my fault.  I broke a cardinal rule of successful innovation: I was making a change without having justified it.  You tend to see this happen from time to time, and it goes by a variety of names such as skunkworks, 20% time, secret squirrel projects, etc. and looks to those outside the project that innovation is all about sneaking round processes and people to get an outcome that ultimately few people outside the project are bought into.

But… if you’re doing skunkworks in an organisation you’re throwing away one of the principal benefits of being in an organisation, which is the ability to draw on a wide range of resources to help you achieve your goal.  If you don’t utilise these resources, you’re effectively working as an n-person group (where n is typically a very small integer) in a virtual garage somewhere within the organisation.  It’s true to say x% of garage startups (where x is a large integer approaching 100) die because they aren’t properly supported, either financially or through specialised expertise (how do you expect to build the world’s next Great Widget when you have to do the accounting yourself?).  The same happens with most skunkworks projects, except the run rate is even worse because an outside influence (your management) can quite legitimately make you go and do something else with your day.

Skunkworks works when you get someone outside the team to care about it enough to resource it (going back to the garage analogy, the idea survives because it manages to attract seed funding).  This is what I mean by Doing It Wrong, since you’re hastily retrofitting the steps you should have taken at the start (e.g. securing backing) to allow the project to continue.   To do this right I think you need the following three skills in the team if you’re going to innovate successfully:

• talented people
• a good, well-defined idea or vision
• change management

The first two are self-explanatory, but the third is, I think, the key to being successful at innovation within the organisation: innovation is change therefore you need to understand how to make changes in the organisation.  You have to identify stakeholders who you will need to gather resources from (people, hardware, time, etc.) and bring them along for the ride, from the first blue-sky ideas session to the point where the project is accepted into the organisation’s mainstream activities.  You need to make allies of these people because they are the ones who are most likely to stall or kill the idea (they may still do that, in which case you need to be objective: are their reasons justified? Was there actually something flawed in your idea?).

There are a bunch of change management models out there on the interweb.  One particular model I’ve used successfully is ADKAR. The steps to successful change can be summarised in terms of innovation as:

1. Awareness: make stakeholders aware of the idea. Socialise it.
2. Desire: let everyone know why it’s a good idea. Sell it. Tell them why they should want to do it
3. Knowledge: let people know how to make the change. How do they get to There from Here?
4. Ability: tool up. Make sure you have the necessary people on the job to make it a success. Train them if necessary, recruit them if you have to
5. Reinforcement: embed the change into the organisation, make it part of the everyday operation

This is where I have seen innovation projects fall down.  You may be all over #3 but if you haven’t got #1 and #2 bedded down you don’t have #4 and certainly will find it very difficult to then achieve #5.  The gotcha is thinking you have #4 nailed because you are thinking in terms of the immediate team, whereas you’re nowhere near achieving #4 because you don’t have the other necessary people on the project to make it a success in the long term.

So, going back to to the work argument, what did I do wrong? I had an idea, socialised it less widely than I should have, and tried to kick off the project.  The result was a number of previously unidentified stakeholders scrambling to understand the idea in a compressed timeframe and therefore not buying in, and a rush to get through steps #1, #2 and #3 above in time to kick off the project on schedule.  This is not optimal… innovation is (by definition) not the business-as-usual work that people are used to seeing; it’s going to take longer to understand and needs more mental effort.  Innovation is hard enough without adding this extra pressure.

Innovation is all about a passion for new ideas.  Sometimes this manifests as a breakneck desire to drive change through regardless, but this is doing it wrong.  Similarly, all that you achieve in hiding innovation in special innovation groups with special dispensation is singling out innovation as a foreign body in the system.  In the end you’re highly likely to end up strangling your own idea and getting frustrated in the process.  On the other hand, if you lay the groundwork carefully the innovative idea looks just like a normal project and earns the right to the resources it needs.  If you can stop talking about seeding innovation and start talking instead about laying out a company strategy, you’re doing it right.

 

By the time you read this it will be out of date. Traffic to Westfield sites coming from mobile devices is sitting on 20% at the end of July 2011. If you trawl the net for similar stories, this figure isn’t remarkable. What is remarkable is that at the start of the year this figure was 10%, by Easter 15% and we have no mobile web offering. We are not serving properly 1 in 5 of our audience at this point in time, and while extrapolation is a tricky business the trend’s pretty clear… it looks like the mobile wave has finally broken.

This is one wave that’s been rolling in for a long time. I was building app stores for T-Mobile back in 2002, back when WAP was the markup standard (for any random definition of ‘standard’) and the main game was J2ME on Nokia. We were sure the wave was less than 18 months away.  By 2004 I was building mobile news sites for smh.com.au and theage.com.au at Fairfax, attempting to play nicely within the walled gardens of the four major telcos in Australia, and the wave was just a year away. I was back at Fairfax in 2007 building the mcommerce site for rsvp.com.au, attempting to navigate the maze of per-carrier premium SMS payment options. With the latter, at one point we reached 5% of all mobile internet traffic in Australia, and the wave was just about any day now.

But still the wave didn’t break. Why…? These are what I think the reasons were. In the early days it wasn’t because WAP was crap necessarily, it was more to do with other factors. Handsets were waiting for Moore’s Law to bring them up to speed, sure, and elements such as screen display weren’t great, but a news site or a Twitter client would have run fine. It wasn’t the god-awful rigmarole of setting your handset up for the interweb either… that was a UI and provisioning problem that could have been solved if enough people thought it mattered. It wasn’t really even about connection speeds that sometimes compared unfavourably to simply placing the handset directly on the server and waiting for the data to migrate via osmosis. For me it came down to two things: the blizzard of handset types and the fear of the cost of surfing. If every manufacturer and even every OS level has a different way to get onto the web, that’s quite a burden of education you’re imposing on your customers. But the root I think was the pricing of data. If it costs a dollar a kilobyte then the only defence against bill shock is to surf as little as possible. Why bother reworking the UI or building compelling content if no-one has a deep enough wallet to use it? I’d contend that the iPhone didn’t kickstart the mobile web so much as 3G plan prices kickstarted the iPhone. Breaking out of the costly data plans has also killed off the walled garden since there’s now little reason to stay in-garden to receive the luxury of unmetered downloads. We’re finally free of the telco standover man, and as the market grows a lot of the other problems with mobile are finally worth getting solved.

So the wave’s finally here… but what does looking back tell us looking forward? Here are my key learnings:

• Data still has to get cheaper; telling customers to stop using their mobiles through captive-market pricing is a massive brake on market growth. This has taken a decade to learn and it’s not over yet. As an example, international roaming data rates would make Al Capone blush and are killing a mobile heartland – you don’t take your desktop computer on a tour of Italy but you do take your phone. Who would surf the Lonely Planet guide at GPRS prices though?

• Handset proliferation will increase. You’d have to be a pretty hardcore Apple fan to argue that guys like Android aren’t going to dominate our future given the stats coming out of the US. Remember Nokia’s position in the 90’s? They brought mobile telecoms out of the dark ages, and pretty much transformed the planet in doing so. But they also gave a new breed of manufacturers a powerful incentive to get smart and try and cut in on the pie.  There’s nothing to indicate that Apple this time are any different; the mobile landscape will be heterogeneous. One size does not fit all.

• Related to the point above, apps may be the reality now but they are not the future. For a mobile product provider the number of app versions to test can be worked out via this formula: platforms * OS versions * countries = variants to test. When the iPhone launched, this was 1*1*1=1; for Westfield this would currently be 3*3*4=36 combinations (and this calculation doesn’t take into account Android platform differences between manufacturers). Add to this the logistics of deploying 36 app variants and keeping them up to date on customers’ handsets and you soon come to the conclusion that there are some combos you’re willing to skip testing for. At this point you have implicitly decided you are okay to move away from seeking the perfection of the app experience; so you may as well embrace the equivalent imperfections of a web experience but with a much smaller testing and deployment burden.
• Mobile is core. There is a shrinking distinction between web and mobile for consumers: checking a Twitter feed on the phone, browsing the news on a laptop, watching YouTube on the iPad, all these activities and devices are becoming increasingly interchangeable.

The last point is important for those of us who are seeking to provide a service online.  It’s time to stop talking about our mobile site and our website as if they are two separate things.  They are now two aspects of the same product and consumers increasingly expect a level of service on the mobile pages comparable to their experience on your website – the days of outsourcing the production of a 5-page mobile site to a 3rd party specialist dev shop are over.  It’s time to tool up and talk about mobile as a first-level part of your product offering.

Firstly, let’s get this out of the way: I like Waterfall. It’s definitely got a place in the toolbox. As a methodology, it’s been around for a very long time… I’m pretty sure they didn’t use Agile to build the Pyramids. It’s a great methodology if you have to make complex but atomic changes, such as cutting over a road traffic management system, or doing a PR launch: there are many situations where iteration isn’t possible, or the iteration length would be months (at which point, you don’t have an Agile process, you have a series of waterfall projects running in consecutive phases, regardless of what you may think is actually happening…). But here’s the thing: Waterfall predates the creation of the Corporation by thousands of years, so has become part of the invisible fabric permeating the enterprise, like Payroll services or Legal departments. It’s the default position and like any tool it shapes the wielder’s perspective: if you have a hammer, all problems tend to look like nails. You can tend to deliver a project or create a product using Waterfall with a known and understood success rate, which makes it a favoured option for the enterprise since there’s a lot of emphasis on reliability of outcome.

There’s a higher level question though. If you didn’t have Waterfall, would you find new ways to do things? Would they actually be more appropriate? This leads to the question of whether doing things differently increases the success rate enough to offset the risk and cost of retooling the enterprise to a new methodology.

Hardcore Agilistas tend to insist that if everything’s done to the letter of the manifesto, impediments will be overcome, management structures will melt away, everyone will be empowered and the people will get what they want. To me (and leaving aside the echoes of Karl Marx) this misses the point of Agile, in that it is a process that introspects itself. This is its superpower: where Waterfall has shaped the entire enterprise over time to look like it, Agile is capable of adapting to its environment as well as shaping it. You don’t have to be a Prussian Field Marshall to understand that no battle plan survives contact with the enemy; a methodology that is able to adapt will survive.

Westfield is a large organisation, and specialises in digging very big holes in the ground, pouring in concrete and earning rental income for the next two decades.  Fair to say that it does this very well in that they’re able to repeat the process successfully and have most of the processes down to a fine art. It’s Waterfall’s home ground.  But how does a property management company used to dropping a billion dollars to rebuild a sizeable chunk of the East End of London address a clear and present danger to their core business with the seismic changes due to the rise of retail ecommerce?

Let’s look at the ingredients of the problem: lack of deep knowledge of the domain (online retail), needing to create an ecommerce offering in a rapidly-evolving landscape, while building a new business from the ground up.  By the time that a traditional Waterfall project had gathered the business requirements, they would already be out of date.  This was why we went for an Agile approach, focussing intently on building and releasing in tight iterations, always delivering something new into the production environment, looking at our direction versus the landscape and modifying accordingly. It’s fair to say that the feature list we started with was not the one we ended up delivering; by deploying something to the market we were able to get an early insight into what was and what wasn’t working and act accordingly.  Another important factor is accommodating a rapid growth in team size and team functions; roles were defined and redefined several times as operational pain-points became clearer.  Often it seemed like whack-a-mole with new problems arising in one area as previous problems were resolved in another.  Here, choosing a particular Agile methodology such as Scrum is not as important as its underlying principle of continuous introspection and improvement through use of the end-of-sprint retrospective to air pain-points and a commitment to re-engineer processes to remove them.

So for us, was the cost of retooling for Agile outweighed by the benefits?  It’s certainly reshaped (and is still reshaping) the way we do things. In retrospect, there really wasn’t another option.

It seems like the best way to get someone to buy something is to get their friend to tell them they should buy something. On the face of it, this sounds like saying the best way to get from Sydney to Melbourne is via Perth: if getting someone to buy your product is hard, getting their friend to sell it for you must be nearly impossible, right?

Yet that’s just what some retailers are betting on. Maybe it’s the case of not knowing enough to know it’s impossible, but it’s looking like there’s money to be made in social commerce providing you can get it right – but it’s not easy money, by any stretch of the imagination because you have to get into the conversation, somehow.

But how? Here’s the four phases of buying a product:

* Consideration – realising that you might want to purchase something
* Research – how do you make sure it’s the right one for you?
* Purchase – getting the product at the right price and at the right time for you
* Post-sales – what do you do now you have the product?

I’d say that post-sales is distinctly a conversation between a customer and a retailer, but can be social through special offers and member-get-member campaigns. During purchase and even research, the opportunities here are in social ratings and recommendations: it’s not so important that a friend has recommended the product (hey, what do your friends know about buying lawnmowers anyway?), but that someone like you (and separate from the retailer) has given their view of it.

Consideration is where opinions are formed and the urge to acquire is kindled – traditionally the realm of glossy magazine spreads and opinion pieces. However, it’s also the realm of word-of-mouth – and there are lots of places that conversations are going on all the time on-line (Facebook, Twitter to name the obvious two).

A traditional way of getting into the conversation has been member-get-member, as mentioned above, which has evolved within Facebook to friend-get-friend campaigns – but since when did you enjoy a friend trying to flog you something? Take the typical scenario: you get a $10 discount for recommending a product to a friend (usually if they then go on to buy). The trouble is, you probably feel cheap selling out your friend for $10, and your friend is probably going to realise this at some point. A massive disincentive.

It turns out that people generally value their friendships more than they value a discount. But what if you turned this on its head? What if you got nothing but your friends got the discount? All of a sudden you don’t need to feel bad anymore since you’re helping your friends save money. They get to know about the product from someone they trust, and get an incentive to buy it.

And what do you get? Kudos, which is something money can’t buy.

Here’s the thing: you have something great to sell, you’ve built a fantastic site, you’ve opened your doors, but no-one’s coming in.  It’s the classic build-it-and-they-will-come scenario, which tends to work for ghostly baseball teams in cornfields but seldom in real life. There are ways to get the traffic…. You can painstakingly craft your pages and linking strategy to wring maximum SEO benefit in the hope of turning up on the first page of Google for specific searches. You could go for the sledgehammer and pay SEM dollars to appear on the first page. You can do deals to get other sites to refer in to you. Or you can produce a very cool viral video that tickles the zeitgeist and puts your site on the front page of every news site on the planet. (Some of these are harder to pull off than others, your individual mileage may vary).

Or you can fish where the fish are. After all, do you really think people will squeeze five minutes into their busy day *every day* to go to your site?

This is an age-old retail trick.  Think of everything from lunch carts to coffee vans to milkmen – by meeting the customer on the doorstep, they can ensure sales and also do some marketing at the same time.  The same applies to online retail, but where do customers live online?  Sure, there’s the inbox, but mass eDMs are like the door-to-door salesman: it’s highly unlikely you want to buy what he’s selling but he’s going to knock on thousands of doors to find a few punters.  And let’s face it, the in-box is already crowded enough!

So where else are they?

Well, Google.  It’s a guaranteed jumping off point for customers starting their shopping journey – over half of purchases in certain categories are researched online first, rising to 90% for consumer electronics. But, getting to the top of the pile in Google means you pay the bucks (to your SEO guy, to Google) to dangle your link in front of would-be customers.

Then there’s Facebook. They have also launched into the ad market, but with a twist: they offer much more targeted campaigns based on the rich demographic data they have on their members. For example, you could choose to reach everyone in Australia – 8.5M Facebook members, (well, today) – or just single females in Sydney aged between 18 and 24 (78,000). Again, your mileage may vary, but you may get much more bang for your buck if you know your market well.

But you could go further. An eCommerce website usually follows this sort of flow: landing page > search/browse results > product details > cart checkout. You can argue that as long as the customer clicks ‘checkout’ you don’t really mind how they got there. So, why not turn the website inside out? Instead of making customers come to you to shop, feed products out to them where they are, for example:

* Deals of the day to their inboxes
* Reading in their Facebook streams when friends comment and brag about their purchases
* Gathering together to demand group discount rates
* Sending out birthday gift lists
* Letting your customers do the selling

By sending your products out in affiliate feeds, emails, Facebook posts, tweets, how your own site works becomes less important. You could empower fashion bloggers to sell the products they mention from their own site (powered by your checkout systems). There is a control issue here, and there is also a requirement to know your customers well (after all spamming a Facebook stream is a good way to lose friends and alienate people!). But by letting shoppers buy your product without ever setting foot in your (virtual) store you increase your reach beyond your site.

After all, what you want are the sales, but running a website to get them can be expensive. If you love your products, perhaps you should set them free…?