Agile global development - a conversation with Jeff Chang from Ariba

Many product development teams have adopted Agile methods, and many also highly distributed - with development spread around the world in remote labs, outsourcing partners, and people teleworking.

There's a lot written about Agile, but not a lot about how people have applied Agile with distributed teams, particularily large teams. So I set out to find out how product teams make Agile methods work with teams spread around the globe - how it works in practice. In my third interview, I talk with Jeff Chang, who leads the International Development team at Ariba. In this interview, Jeff talks about Ariba's long-term strategies for doing Agile with remote teams, how they deal with the high-cost of communications in distributed Agile, and how they've adapted Agile methods to their advantage.

Matt Hately: by way of introduction, it would be good to have you explain your role to the readers.

Jeff Chang: Sure. I'm the VP of International Development at Ariba. I wear two hats here at Ariba. One is the globalization function, which is internationalization and localization. The other one is as the executive sponsor of our India Development Center, which is our main offshore site.

Matt: Were you using Agile before you set up your global center, or is that something that came after?

Jeff: It actually came after. We were doing offshore before, and Agile came a couple of years later.

Matt: What did you find were the most challenging aspects of making the transition to agile, particularily with your global team?

Jeff: Actually, the way we cope with Agile - remote Agile or distributed Agile - in some sense is avoidance.

Matt: How so?

Jeff: Our theory about doing development offshore is it is a strategy for how you distribute work. And the way we look at it might be slightly different than how Macadamian thinks about it - we're not looking at necessarily just on a single project basis, but on the organizational lifecycle, over years’ time. We view offshore in three phases:

The first phase is the ramp up phase - typically a small project or design. Then, there's a co-development phase where you're going to have a lot of distributed Agile. Then, we get into what we call the encapsulation or ownership phase.

In our mind, where we're trying to get to is large enough ownership modules - ownership encapsulations - to reduce the communication cost, because we acknowledge that communication cost is high. So, that has been our long-term organizational strategy.

Matt: Yes. So, with your current team, What phase would you say they were at?

Jeff: It depends - it's a continuous process. Like most organizations, what we did is we started with a few small projects. There are two kinds of capabilities you are looking for with your remote team. One is obviously knowledge of the technology stack, and the second is domain knowledge. What is it that you're trying to build? That first stage is to try to get them knowledgeable about those two pieces.

Each organization is at a certain life cycle, so some will start with that phase one, then maybe after release two and release three, they're in a co-development mode. Then, you may get release four or release five, when they can take ownership for a smaller piece,then subsequently larger pieces. That's the evolution that we've seen.

There are certain teams that are mature and have a lot of ownership, and other teams are more in the co-development or just starting off mode, if we look across our organization at all the different product teams.

Matt: So, do you use Agile through all of those phases?

Jeff: Yes, we pretty much do. It's a modified Agile. Before we even brought Agile in, we looked at Agile, Scrum, XP. We took pieces of each. I would not say we're as pure, in terms of execution. For instance, we have milestones, but we also have a "back end" - a systems integration QA task at the back end. I would not say that we exit milestones as cleanly as we would like. (We call it milestones, as opposed to sprints). So, these are the aspects of where we may not be as purely agile as we would like.

Matt: Is that driven by the fact that you are distributed? Did you find some things that are very hard to do with a distributed global team in Agile?

Jeff: No, I actually think it's a little bit of our engineering culture. It's hard to get full attention to finishing everything in a milestone. We understand the agile principles, but in the actual execution we bring some of our own ideas in. So, I don't know if that's necessarily related to the fact that we're distributed that's caused the way that we've adopted the process.

Matt: So it's a legacy or a history that comes along with the way people are used to working, that makes it difficult to embrace a "pure" Agile process?

Jeff: I think so, yes. Again, this is an engineering culture aspect, but we're always trying to get better; a continuous improvement, and trying to use better tools, trying to improve process.

Matt: You know, I've been talking to a lot of people about this, and I find that most organizations "cherry pick" what they want from the Agile processes, some of which just do not work in their team.

Jeff: So, that's interesting. We've not done what you'd call a peered study or benchmarking in terms of our process adoption. There's probably something we'd learn from that.

Matt: That would be interesting. There are aspects that are common across most of the Agile processes, like self-organization, which can mean things like picking your own tasks. Is that something that you follow, or that you've adopted? Letting the team members choose their own tasks?

Jeff: I don't know if we do as much of that. Certainly, we have scrums where people talk about the burn-down chart and what has to be done.

Matt: How do you do things like stand-up meetings or daily scrums in a distributed model?

Jeff: That's where geography makes a big difference. And that's what I meant by communications cost. I thought a little bit about the Macadamian model. If I'm understanding it correctly, you have a live synchronization time, probably daily, that's sustainable.

Matt: That's right.

Jeff: And it's probably toward the tail end of the day and the beginning of the day for each team. With India, we don't think that's sustainable. We have tried that. So, there have been periods where there's been a particular project, which was a really tight deadline, highly collaborative, where you had people on both sides who had to contribute major components.

There's the classic 'follow the sun' that people like to talk a lot about, particularly in the context of India. I don't know if other organizations may do that well, but we don't think that's sustainable.

So, the worst case scenario, in terms of communication and synchronization, you might be at nine o'clock in the morning Pacific? You talk to India, that's their night. And then twelve hours later, you talk again. So, it's nine thirty at night U.S. time and it's the morning, India time.

In an extreme case, you can actually synchronize every twelve hours, and you are very productive. But, your team is likely to be pretty burnt out after a few weeks of that.

That's the most extreme form of synchronization. The reality is that we'll tend to have scrums on either side, and then, you'll have a synchronization point - typically the manager. That might be on a weekly basis, and everything else is more off-line.

We've tried many different models, but I think, sustainability is really the question.

Matt: That's a really good point. It's a very common model, but when your team is in India, you don't really have a lot of overlap at all, do you?

Jeff: There is really no convenient overlap in the most common scenario. One of the most common off-shoring models is California to Bangalore and there is no hour which is convenient ever. Someone has to sacrifice. And so, it going to be typically nine o'clock or 10 o'clock at night that someone has to take a call.

Matt: You mentioned some asynchronous communication or offline communication, what kind of form does that take?

Jeff: We have taken obviously email, and there is some IM'ing, but I don't know if we have as much of a pervasive culture around that.

And of course nightly conference calls - at minimum once or twice a week. And we use wikis, and are experimenting with Scrumworks. One of the big issues with Agile is "sticky" planning. The idea is that you want very lightweight planning. You write on the whiteboard saying, "Hey, this is what we are going to do." How is the team in India going to see that? The reality is that you need to have some additional level of more structured communications.

Matt: What do you use wikis for? Collaborative planning or for project information?

Jeff: Different teams have different levels of documentation. And so, some teams will actually write something more like a classic PRD. It is like a story, but it is not very detailed. That is going to be on the wiki. Other teams don't do that level of detail. So, it really varies by team, we are not so uniform in terms of that level of process.

Matt: Another thing that Agile promotes tightening the feedback loop between product management and developers. Can you talk a little bit about how you go about defining requirements and the interaction between PMs and Development?

Jeff: I will talk more generally about team encapsulation. The really good thing about Agile is that it really encourages cross-functional teams working together. We actually had that concept before we adopted Agile. We call them feature teams - we have a representative from product management, from documentation, from maybe usability, and QA and globalization on the team.

Where we struggled from a distributed standpoint is communication with product management. And what we slowly moved towards is essentially setting up full cross-functional teams in India, in the three phases: in phase one and phase two, you don't have that, but in phase three we do have certain teams where we have pretty good representation. But, the one piece that we are still missing is usability.

Matt: Why is that?

Jeff: That's a very good question. We were trying to set up usability in India. I do believe the usability industry is starting to mature.

Quite frankly, it depends a lot on geography and what level of maturity and availability of talent that you have. Product development is still relatively new in India. Trying to find all the talent at the right level of seniority is a challenge.Certain things we'll have to still do offline. We have usability in Sunnyvale. They will spec things and send it over to the teams in India, and then they will send their questions back. Again we can have collaborative discussion.

Matt: Another good point. In India, you don't have that heritage of design both in practice and in the schools that you have in some places in the U.S., canada, Germany, and son on. They have reknowned design schools.

Jeff: Exactly. But, in terms of documentation, product management, and QA we're getting there. There is obviously a lot of interaction between all those teams. The other piece is product management, which I think, is more problematic. And the reality is the technical leader will be the proxy to understand the requirements and translate them into features.

Matt: What about face-to-face communication. Does your team travel a lot?

Jeff: Definitely. That's necessary. Certainly, in the phase one when you've got to incubate a team from nothing and the team doesn't really have much background at all. We will typically bring the team out for some number of months - two to three months - and they'll be working under the wing of senior developers and QA . Then they will go back and they'll be leads for their teams in India. As the team is getting built out, typically the managers will go out and they will also do knowledge transfer.

The lot of the way we think about distributed development is around the knowledge transfer paradigm. How do you manage knowledge? How do you transfer knowledge efficiently? I think that in a single site you don't really have to worry about that.

Matt: Right. It just happens.

Jeff: It happens, right. It's very osmosis based. But something that India has done very well is create a core competency around knowledge transfer. Also face to face, people moving both directions, is pretty key to establishing the relationships.

Matt: Is there anything else we should cover before we wrap up?

Jeff: I will throw in one more process thing, around coping strategies for distributed Agile. One model that we worked on is the encapsulation model, where we believe that at the end of the day, co-location is good. We are trying to organize around fully functioning teams remotely. But the other dynamic behind remote development is that the reality is that you tend to over-invest in QA on the offshore site, because it's more cost-effective.

What that does though is that it puts a lot of development in California and lot of QA in India. And so, our coping strategy is a cascading QA model. The idea is that in an Agile milestone phase, one of the tasks of the QA person is to write high-level use cases. That will get sent to their counterpart, who then does a more detailed QA in a trailing milestone.

The key benefit is that it helps with knowledge transfer. Not everything is written down in Agile, so you're trying to figure out... what's the minimum level of artifact that I need to communicate effectively?

That's why the Agile QA person is writing these use cases. Very high level. And that would then be the guide for the person working on the next level of QA to drive from. So, that again, is breaking the model. The theory is that you've QA'ed everything by the end of that milestone.

Matt: To wrap up, what would you say, if you are going to pass on two or three pieces of advice, key best practices to making Agile work in a distributed model?

Jeff: Well, I will answer it the way I started which is, lots of times avoidance can be your best strategy - I don't know if that is the right answer - but we have looked at the cost of synchronization.

As soon as we started looking at it from a long-term perspective we took the approach of partitioning responsibility and growing capability on both sides, so that they can be fairly independent.

That being said, you actually have sort of the scrum of scrums, because now you have different teams which are still highly dependent on each other. And so maybe we are still distributed Agile but on a different level.

Matt: Right. So, to paraphrase, the long-term goal is to use Agile, but within local teams; rather than trying to do Agile across Sunnyvale and India, India will be using Agile and Sunnyvale will be using Agile, but within their local team.

Jeff: Yes, but the reality is that, at least the way we are set up as an engineering organization, is that we have multiple feature teams or Agile teams, which then have to integrate. So, we are still highly independent, but our dependencies are at a higher level.

Jeff: One other thing - I don't know if it is a best practice, but I think that the reality there has just got to be a little bit more documentation than some of the more pure models. And so, we tend to write down a little bit more upfront - not extraordinarily detailed PRDs or not extraordinarily detailed specs - but at least some minimum level of documentation. At the same time, with respect to planning and the burn downs, etc, and having very variable content and milestones, we use all those principles and we think that holds.

Matt: Jeff, I want to thank you for doing this interview with me. I set out to get a real-world view of how a large engineering team copes with doing Agile development with a distributed team. Thank you for giving us an inside look.

Jeff: You're welcome. I hope it was useful.