Diana Larsen and James Shore presented "The Agile CTO", a session for executives (oooh get me) which had the aim of bringing out and discussing a few consistent concerns that they have about their position.

• Their research was based on interviews with 8 CTOs from a variety of companies - sized from 8 up to thousands;
• Many felt the shift in priority as they moved from manager to CTO, quite keenly;
• Priorities changed from "how this project is going" to "how the share price is doing";
• They had a need to retain credibility by meeting commitments, and by not being surprised by bad news (there was a sense that it's the surprise that causes problems);
• They spoke of trying to create an environment where mistakes are not punished, but are accepted and analysed;
• Another consistent theme was time spent dealing with emotional baggage of team members;
• They were also a consistent love of seeing people excel, or watching their teams come up with amazing stuff that completely surprised them;

We then broke into groups to discuss other consistent themes from our experiences. Our table came out with:

• The need to keep abreast of industry trends and have a position on them, and how they affect our business;
• Balancing new product development and "lights on" maintenance activities;
• Representing business issues to the technology side of the business, and technology issues to the rest of the business - effectively being a conduit for information both ways;

... and some others, which I never wrote down, so they'll be lost to civilisation forever.

"Take no prisoners, how a VC group does Scrum" was a chance to see Jeff Sutherland talk. Last time I saw him in London, he'd mentioned OpenView, the VC Group he was involved with who were using Scrum and insisting that all their investments used it, to up performance. This was a more detailed talk from Jeff and Igor Altman (who works at OpenView) on their experiences adopting.

Very interesting to hear that even a Scrum team so close to the originator of the method has gone through so many familiar stages and pitfalls...

• Organisations get refactored, just life software does. OpenView has "refactored its Scrum implementation" 5 times so far;
• VC groups have highly trained, silo'd specialists;
• The Maxwell Curve indicates that optimal working hours for a waterfall team are 60h/week. For a Scrum team it's 35h/week. VCs want to see their portfolio companies being optimal :)
• OpenView has a strategy of adding value to their investments by providing best practice, helping with execution in all areas of the business, running forums and workshops, doing on-site coaching and training. They and their portfolio companies were all full-Scrum by 2008;
• They do this to bring operational efficiencies to all their companies and maximise ROI. They've had 2 rounds of $100m investment, employ 22 staff and have 10 portfolio companies, including VersionOne;
• The VC ran 4-person teams on 1-week sprints, tracking in hours not story points (they couldn't get their heads around SPs); initially they adopted the trappings of Scrum but not much more;
• The team became self-managing, communications got better, they noticed themselves aggressively eliminating low-value work with stress reduced and impediments around clarity and communication;
• Quote: "If you're working long hours, you aren't doing scrum" - particularly surprising to hear that from a VC;
• Challenges they still had were overrunning meetings and some people uncomfortable with it all;
• In January 08 their planning took 1d (for a week sprint), with standups at 45m. But August planning was half a day and standups 15m;
• They noticed themselves oscillating between concentrating on quality and velocity; some team members were missing the big picture; and too many projects were dragging out whilst they were all being done in parallel;
• By November 08 they were running 2 teams of 4 staff, focusing on initiatives (epics) to give more "big picture", fewer simultaneous projects, and knowledge sharing through learning-lunch type things;
• They found they needed to balance direction, speed, quality and sustainability quite actively;
• Removing impediments without addressing the root causes usually led to those impediments returning later on - so they do RCA regularly now;

Performance without Appraisal by Esther Derby: Esther writes a lot on this topic and at FP we do the normal quarterly-review thang, which seems quite sluggish, so I was curious to hear about alternatives.

• How do you know your appraisal process is working? Most people say it doesn't work, but everyone does the same sort of thing anyway;
• People believe performance can be measured objectively; but measure any one thing and you encourage it to be gamed - plus the stuff that's easy to count, doesn't count (e.g. Lines of Code generated) and contributions aren't always visible;
• The data says it's virtually impossible to objectively measure performance;
• What do we really want: high team performance, or knowing who to blame?
• Most annual evaluations lead to an unhelpful label and in any case telling people they're below average triggers defensiveness and stops them listening;
• Annual feedback is far too slow - you need specific information, close to the event, for it to be useful;
• Improving individuals doesn't improve organisations! The law of crappy systems: "a crappy system will inhibit the ability of the most competent team to perform";
• Behaviour is a function of the person and the environment; how we work has both social and technical aspects;
• So make feedback "business as usual"; ensure it's specific, actionable information, and make sure it flows both ways (not just down the org chart);

Esther referenced an article on her site on how to create safe environments in which feedback can flow around, but I can't for the life of me determine which one of these pieces it is. Will just had to read them all, sigh... ;)

Herding Cats - Managing Large Tests Suites piqued my interest, as we've struggled with some aspects of automated testing recently.

• The presenter, David Kessler, has been running tens of thousands of tests on hundreds of builds of their product;
• There's a pull between fixing up tests and doing new work; keeping the build green is like herding cats;
• Many of the test failures are at their root caused by basic Java or design issues;
• They have a classic build monitor on a large screen; one refrain heard from the dev team is "it's not my build that broke!";
• Lessons they've learned include decoupling tests from one another; mocking out external services (they particularly referenced a third-party payment gateway); tracking percentages of tests which have been temporarily "turned off"; and simplifying test data;
• Complex hierarchies of tests cause problems; there's a maintenance cost here which can in part be addressed by simplifying their setup;
• Third party services need to be mocked out. A payment gateway they were working with said they were fine for automated testing, but when it got to them being hit per-build by a CI machine, they were less keen;
• Working with 3P services opens up issues of availability, network and firewall problems;
• They replaced this with a mocked out service, which also let them test edge cases easily, and backed this up with a single test of the remote service to check they could connect to it;
• Turning tests off breaks trust - "yeah we were testing for this thing, then we stopped doing it";

Joh's research presentation was fun, and a bit squirmy.