Should reviews weigh into your product design?

In software we pay close attention to product reviews. Let's face it, reviews influence sales, but to a degree. I think good design - good in the eyes of your target customer - matters more. Let me give you a couple of examples.

I have laptop envy for the new Macbook Air. I've always owned a Windows machine, but all the cool kids here in California have Macs, and they're growing on me. Especially when the guy sitting next to me on the plane and I opened our laptops at the same time. Me on WinXP and him with his Macbook. He was crunching sales figures instantly, while I waited another 7 minutes for Windows to boot and Outlook to start. But I digress. What held me back from buying a Mac was that they are so big (I spend a lot of time on the road), and the battery life (I know people who carry 3 or 4 batteries just to keep their Macbook alive for a four hour flight). The Airbook is light, the perfect size for travelling, has a great screen, is strong, and is just so sexy!

But the Air is getting lukewarm reviews from the experts, who say the reliance on WiFi was a bad idea. Where's the DVD-multi-CD-rewritable-superdisk-burner? What if I want to watch a movie in flight? Apple is betting on a wireless world, and their target customer is going to be more apt to download a movie from iTunes than watch it on a DVD, or pick the download option to install Photoshop rather than trek over to Best Buy, and I agree. The DVD drive, and even the Ethernet port, will be an anachronism sooner than we think. Apple had a vision for building the slimmest laptop in the world, not a pretty-darn-close-enough slim laptop with an Ethernet port.

I found this great quote in Tom Kelley's book, the Art of Innovation - "We learned that the public and even supposed experts are often not he best judge of which features need to disappear. Refining products is not a popularity contest. You have to take risks and you will alienate some people. For example, a number of reviewers criticized Steve Jobs for not including floppy drives in iMacs and iBooks. But they missed the point. The iMac was designed to catch the Internet wave, and a floppy drive would have demanded extra space, fattened the price point, and focused on soon to be obsolete technology. Jobs wisely decided that the machine couldn't look like something from the Jetsons yet be weighed down with eighties technology."

That was about the iBooks, which were released several years ago. How quickly we forget, but obviously Apple remembered the lesson well.

Here's another example. Last week I had the chance to rent a Prius for $1 more a day than a Kia. I've never driven a Prius and had to try it - I'll try anything once. You have to understand I'm a die-hard car guy, and the Prius is not exactly a car-guy-car. It's loathed in the hot-rod circles and dissed by the "serious" car magazines. Now, I do respect the planet, and I try to do my best to be conscious of the environment, but I have a serious car addiction. I have a mid-70s Pontiac with a 400 and a 4bbl carb, and no cats. You can actually see the gas gauge moving when you drive it. Every time I drive it, I'm sure I make a dent in the ozone layer and further the war. But I digress...

What did I think of the Prius? I loved it! It's a whole other kind of car geek fun. After I figured out how to start it (here's a crash course, so you don't have to look it up like I did: step on the brake, stick the key-fob into the toaster-looking slot in the dash, and press the On button. If the dash says "Ready", you're ready to go), I was having all kinds of fun trying to maximize my mpg - watching the little screen showing me when I was using the electric motor, and what my mileage was. I remember the auto journalists, like the guys at Car & Driver (admittedly one of my favorite magazines), were pretty down on the Prius as being full of gadgets. They much preferred the more mainstream-looking Honda Civic. Well guess what? In 2007, Toyota sold about 181,000 Priuses. Honda sold 32,000 Civic hybrids.

What did Toyota know that the reviewers didn't? 1) People wanted to feel like they were driving an electric car with a small gasoline motor, not the other way around - the On button and all the gadgetry makes you feel like the Prius is really an electric car, and the gas motor is just along for the ride. And more importantly, 2) People who drive Priuses want the world to know it. Dammit, you're saving the planet, and guys like me chewing up 10mpg should feel ashamed for not driving one also. It should look and feel like a Hybrid, not like a Camry with a little Hybrid badge.
So take what the reviewers tell you and put it in the mix, but don't rely on it as the gospel, and as a substitute for observing users, prototyping often, taking chances, and failing quickly.

Why Silicon Valley is still the hub of innovation

Great article in the LA Times today called "Come for Brunch. Bring Billions", about the Founders Brunch a regular, invite-only get-together of deal-makers and entrepreneurs that takes place in the SF Bay Area.

Lots has been written about innovation and entrepreneurship in Silicon Valley. This article really nails what I think are the top two reasons why the Bay Area is still the center of the tech universe - a culture and climate that rewards risk-taking, and the network effect. I've done business (in technology) now in Ottawa, Boston, Los Angeles, and Silicon Valley, and I've never seen such a strong and open network, and such openess to taking risk.

People think BIG in Silicon Valley. They aren't out to start a company to play it safe, they start a company to change the world, or at least the way people do something, and grow exponentially. And some do, and others fail spectacularily - but the big difference is that's ok. Failing means you're thinking big and you're trying. Many of the ideas that take flight in the Bay Area (and many that become huge successes) would be laughed out of the boardroom everywhere else - dismissed as crazy. How many people said "no-one will watch user-generated video content - what a waste of time"?

I've also never experienced such a strong network. Groups of engineers and entrepreneurs have been meeting since the 60s here. Go to a networking event in most areas of the country, and people are cliqu-ey - reluctant to talk and reluctant to share their network. In the Bay-Area, everyone is keen to chat, if even only for a few minutes, as everyone's on the lookout for the next Big Idea, and everyone seems to understand that even if you don't have business to do today together, you might know someone who might be able to help and visa versa.

To echo Peter Thiel's quote in the article, there are lots of cheaper places to set up a business - that's why people still set up in Silicon Valley.

Distributed Agile Development - an interview with Jason Mawdsley and Stephane Lussier

Typically, when people think of outsourcing development, especially offshore, they associate best practices with doing waterfall development – provide a big detailed specification up front that describes exactly what you want (otherwise known as BDUF - big design up front).

With proper controls in place, this may work for certain projects, especially more mature projects, but frankly it's hard to do BDUF for new product development. Many product development organizations are are embracing agile methods, because agile methods deal well with change, which is inevidable in NPD. They also provide a framework for making rapid forward progress while keeping focused on the highest value features.

I see a number of teams asking themselves, can we do agile development with a distributed team? Can I reap the benefits of agile when I have a team scattered around the globe?

So I decided to put together what I hope will become a series of interviews with people doing real-world product development, using agile methods with remote teams. Some say if you’re not in the same room, it’s nearly impossible to do agile development. Or if you can, you take a serious hit in productivity. I wanted to talk with people who were making it work.

For my first interview I started at home - with Jason Mawdsley, one of our Project Managers at Macadamian and a certified ScrumMaster, and Stephane Lussier, our VP of Engineering. At Macadamian, our team is spread out over three continents, but most of the work we do is new product development under intense deadlines, where BDUF isn't going to work.

Agile promotes a lot of communication, but normally in remote development you have to make do with “batch” communication. What are some strategies for closing the communication gap when you have teams scattered around the globe?

Jason: You have to set the tone for frequent communication, and have the tools in place - Skype and IM for example. We aim to have several live chats a week on a given project.
We have to be willing to adjust our times - someone from Ottawa may have to take the call at home first thing in the morning, or someone in Yerevan at the end fo the day. We also make use of a number of different technologies, like mailing lists and a Wiki (Confluence). We try to have a number of different tools to make communication easier, but everyone on the team needs to be on a common set of tools - the same IM, the same VoIP system

Stephane: You need to encourage people to ask questions. It's not easy, especially with newcomers. How do you do that? You tell people to ask questions. There are no stupid questions – this is not the time to shoot the messenger and tell them they should have known the answer.

How do you make the most of the one or two hour time overlap?
Jason: you need to mak sure that people in the west need to reserve some time their mornings for communication, and the in afternoon in the east

Stephane: But don’t wait to send questions until the overlapping hours – if something comes up in the afternoon and I need an answer, I'll send it, so that its at the top of the queue for when people come in in our Cluj and Yerevan labs. You only have a couple of hours of overlap, and somedays it may only be one – if it’s an urgent email, there’s good chance that people will act on it as soon as they come in

Tell me more about the mailing lists, what’s the benefit of having a mailing list for the project?

Stephane: It’s not the mailing list itself; it’s the philosophy of submitting code every day for review, in a place where everyone can see it. There are different ways of doing this – it doesn’t have to be a mailing list, but you want everyone on the team is showing activity every day, so that when people are going in the wrong direction you can realign them right away. It's the same concept as agile – everyone in the same room, talking, and aware of what everyone else is doing. Pushed to the extreme it’s similar to pair programming, but pair programming is a bit extreme for me, and not practical to do in a distributed team, but the benefit is the same - everyone reviewing everyone else's work.

Jason: I agree - it’s not the mailing list, the mailing list is just the tool. The culture is that we want everyone to be aware of all the questions and progress from the rest of the team

Stephane: We don’t want to do waterfall – big design up front by people close to the customer, and swing this offshore, and then hope for the best because you put enough detail in the spec. Some people make it work, but I think there are better ways

How important is it to have a local manager in each of your development labs? What qualities are you looking for?

Jason: No matter what kind of office you're opening, whether it’s a sales or development office, you need someone you can trust to oversee it. Someone who knows the culture, the rules, and the lay of the land.

Stephane: When there’s something that needs improving, you want to escalate it to someone close to the action. Solving probems remotely is more difficult than having a talk face to face with someone sitting in the same office. I get better results when say John, my manager in the Cluj lab, is sitting down with that person one-on-one to resolve an important issue.

You also want someone who’s persistent and patient. To develop an agile culture with new people takes time, especially if they’ve never worked in that mode. You want someone who will follow up on a weekly if not daily basis to reinforce the culture. -

How important is travel and getting face time?

Jason: It's important when you’re bootstrapping a project. When we sent one of our Project Leaders, Melissa, to our Armenia and Romania labs to kickoff a large project, it was the most effeicient way to get everyone up to speed. Also it helps build trust – you can get to know them on a social level.

Stephane: You build trust a lot faster. Once the trust is there, people are more open to discussing issues or even just tasks. They are more open and up front about things that need to be done. You want to have a strong trust relationship with the remote office manager- you rely on them to understand the culture, the process, and to coach people.

What types of people do you need to make distributed agile work? What qualities do you look for?

Stephane: that’s a good question… communication is important. Someone willing to communicate. You need people that understand what’s going on under the hood (of the OS, platform, etc). They have a logical brain, because in Agile development you don’t have a big spec with a lot of detail, so developers are asked to fill in the gaps; there are 2 ways to do this – be very bright, or communicate a lot and ask a lot of questions.

Jason, you’re a scrum master. Have we used all the Scrum concepts in distributed development? What pieces of Scrum work well and can be adapted, and what have you had to throw out?

Jason: We’re not using it full on. We’re using the most important tenets. We’re doing shortm 3-4 week sprints, we’re doing daily scrums but as a voice call, we have an infrastructure that lets people pick tasks, and a Wiki to track project info.

On many projects we let people pick their own tasks. We have Jira, which doesn't force you to assign tasks. You create them and can let people pick the ones they think they are best suited for … sometimes critical ones are assigned though, because you want them done by specific individuals.

Are there parts of Scrum that you say are impossible to do remotely?

Jason: No. It’s easier to do in person, but that doesn’t mean it’s impossible.

What can the open source community teach us about doing distributed agile development?

Stephane: A project can be successful and have high quality by people around the world, not just by people sitting in the same room. Open source projects don’t follow waterfall model – they are much closer to agile. It’s not true people need to be in the same room to develop using agile. Open source projects use mailing lists, or whatever forum, for sharing code. Typically the open-source projects that are working well have a very active mailing lists. There’s always someone to provide an answer and review work. They have a common forum to discuss issues. Also successful open-source projects have a few very good people running the show. They won’t let poor quality code slip by, they will set some rules about quality and enforce them.

Jason: I’d have to dig up the exact quote, but it goes something like “everyone is empowered to make their own decision”. In an open source project, everyone is volunteering – so everyone is passionate about it. So they pick the bugs and tasks they want to work on.

Stephane: Motivation is high and buy-in is really really high.

Jason: And because they contribute to the project by tasks, they don’t do big monolithic patches.
Stephane: Motivation and buy-in are at 100% - they do it in their free time. Who can say this in the business world?

Jason: Me? (laughs)

Stephane: So another thing we can learn from Open-Source is getting everyone's buy-in motiveates and gets good results.

How do you make distributed agile development predictable?

Stephane: Small patches. You need to be able track how you are doing in small doses. You also need to know your past performance. You need to have estimated tasks, then track how long they took, and adjust for next time. That’s the whole goal of doing Sprints – fixed time-boxes for iterations. One of the goals of this is to know how much you can do in one iteratrion. At one point, you know exactly what you can accomplish in 4 weeks. The same is true for a distributed team.

How do you do task allocation? Many agile methods advocate having team members pick the tasks they feel they are best suited for.

Stephane: We try to break things up in small 1-2 day tasks, but we'll also let the team do their own work breakdown. So when someone ends up with a task that is 3-5 days, they need to come back with a more granular plan. The developer has to do some breakdown . We eant people to think about how they wil tackle the problem, so that everyone knows the task is understood. This is a pattern that's hard to break. People are expecting, and this is everyone including North American developers, is that they will work a something for 2-3 weeks. I think it comes from school – you’re asked to go away and work on assignment for 3 weeks, and submit it when you're done. This doesn’t work, and you need to break this habit

Jason: In agile you want to favor communication over process and documentation, so you have to structure things to promote communication.

One of premises of agile is you have developers pick their own tasks. Does this work in a distributed team?

Jason: Yes but it's something you have to work on. You want to encourage them to pick their own tasks, but it's a culture that you have to instill. The advantage is they will pick a task that they are more motivated to work on.

Stephane: I find people that are just joining the project don’t want to just pick the task - they don't have the confidence yet. Normally you want to assign them something easy to get them some familiarity with the code base. Later, once they have confidence and experience they will be more apt to pick their own. This is true distributed or not.

Who plays the role of the user advocate in a distributed team?

Jason: The testers.

Stephane : I would kind of agree. Sometimes for us, it’s the real customer, who gets involved by being on the mailing lists, actively reviewing bugs and drops, and so on.

So how do you involve the actual customer?

Stephane: The customer has to want to be involved – and that’s the perfect scenario. You give them frequent drops, and they will give you feedback.

Jason: We don’t have the customer on-site, so we try to open everything up, securely, with Jira, Confulence, and SVN so they can see everything. We try to send frequent drops so they can provide constructive feedback. But you have to recognize that not every customer has this commitment – say it’s a non-core. feature. In this case it’s the Project Leader and Tester who have to take ownership of the product and act as the customer.

What kind of tools and systems can make distributed agile easier? What are the must haves?

Stephane: You need email (laughs). You need to be able to have phone conversations, so Skype is useful. You need live discussions. IM is a must. If you want to be able to distribute taks, you need some sort of task management system, or bug system that supports tasks like Jira. Video can be helpful sometimes.

One thing that is useful is working in a Virtual Machine– when you add a new developer, you give them a new VM – everyone has the same development environment.

Jason: it gives you a replicable environment – as the project gets bigger the setup is harder. It could take 2-3 days to set up a dev environment.

Stephane: you don’t want a situation where it stops working for one team member and no-one knows why, which happens frequently when you have a heterogeneous environment. You will lose time diagnosing, because troubleshooting will take more time. If the knowledge is with the team in India and the problem in Ottawa, you have to do multiple roundtrips – “send me your trace, etc.”

Jason: It can take four days to diagnose and solve in a situation like that .


What are some of the antipatterns that you need to avoid? Traps that you see managers who are new to distributed agile fall into?

Stephane: Not communicating expectations. In general, you need to over communicate. Just throwing a one-liner to someone in your lab halfway around th world, because you're in the habit of doing this with someone in the same office. You fire them an email - “hey I need this feature". What you forget is that when you do this, the guy next door to you comes and has a 30 min discussion about the feature. If you do this to someone remotely, and they don’t pick up the phone to talk about it, it won’t come back as you expected. So you need to provoke things – "hey, let’s have a 30 minute call tomorrow morning about this new feature you want to do."

The second is Monolithic patches – people going in the wrong direction for weeks at time, working on the same big feature, and you only learn it on a month later, and now it’s near the deadline and you don’t have time to recuperate.

Another is not having someone that plays the user-role – someone close the user of the project, You need to play it actively. Daily follow-up – are we going in the right direction? When I talk to people who say “it didn’t work out with my offshore team” – the common mistake I see is they write a big spec, swing it to the remote team, and we'll see you in two months; thinking that their involvement was done. If they had been involved throughout, I think they would have found the team ran into problems on week one.

But this is true for local teams as well – if the marketing teams specs out ome features and swings it to the dev team, and comes back in three months to see how it went, this won’t work.

Finally, not understanding that the cultures and work habits are a bit different. For example, sometimes with engineers in India, you'll find that things that for everything not mentioned it’s ok to make assumptions – you will be doing your job if you do everything that is said. It’s not customary to ask questions. In China, there’s a respect for authority to a degree you don't see in Eastern Europe or North America. Everytyhing that the manager says is true – you don't contest that. It’s not in the culuture to doubt, or double check that it is true. Most of the time the manager doesn’t have all the knowledge to make the low-level decisionsm, so if you say “use this library" - where with other cultures you assume you are making a suggestion, with a team in China you are making an order. The team will do what you say, even if it gets them very stuck. So not recognizing that each of your teams is different, and knowing how do work around those differences can make things difficult.

In closing, are there one or two things you'd like to leave people with - tips for new managers embarking on doing agile with a distributed team?

Stephane: To me, I would say be patient and persistent. In the beginning it will take a while – you won’t see the benefits right away, especially if you are changing the way you work.

Jason: Rome wasn’t built in a day. You need to take a mid-term view on this. You won't see immediate success. It will take training and culture changes to make it a success. You will see it manifest 2-3 months after it starts

Stephane: Encourage communication – both ways.

Jason: Open, transparent, quick, informal communication.

Stephane: And as a leader, you will need to enforce communication. But when you do Scrum, you enforce communication by default – there’s a Scrum meeting every day.

Trends in Web Product Development

One of the cool things about being at Macadamian, and working in Silicon Valley is we get to work on next-gen products, so we see the newest innovations coming from both the big technology leaders and startups alike. I also get to see trends as they develop.

A trend I'm noticing lately is that SaaS, and developing web-based products has hit mainstream. For the last five years or so, Web development was mainly for e-commerce and in-house apps. The bandwidth and development tools are finally here, which were probably the biggest impediment to the mass-adoption of web-based or network-apps (Anyone remember Network Computing? I spent some time at a NC startup in the late 90s, and we were going to put Microsoft out of business with thin-clients and network-enabled productivity apps. I think you can guess how that story ended...)

Here are some of the interesting trends and implications I'm seeing as a result.

• Adobe Flex and Flash for GUI development: The adoption rate of Flash as a front-end GUI technology is phenomenal. I hear constantly from ISVs that they are either using Flex for GUI development in their next-gen product or seriously considering it
• LAMP Stack in product development: "Serious Developers", i.e. people who have been developing in C or C++, often look down on PHP and Python as web scripting languages you use for developing small business web sites or fan-sites for your favorite RPG. Truth is, some serious products are being built on the LAMP stack, partly because of the tools available, partly because it's low cost when you're first getting started, and partly because there are more and more developers out there with PHP experience, so it makes it easier to find talent. (though I would warn that if you're developing a product, it's important to hire PHP developers who have product development experience - who have rigor and understand the product dev cycle, and that's a little harder to find)
• .NET very popular among Microsoft ISVs: This certainly makes sense, as the learning curve is probably faster for a Windows development team. A few years ago, people didn't take .NET seriously as a robust and scalable platform, and they were worried about building a product on .NET, especially if it was going to be sold to financial or Fortune 500, but I think this has changed. Microsoft has always won
• J2EE on the decline? J2EE was very popular with the early movers - ISVs who were among the first to move to a SaaS model, because it was the enterprise-class development platform. . I don't have the data to back this up. Anecdotally, I just see it less that previously.
• Ruby on Rails is no longer for prototyping: Again, some developers saw Rails as a way of doing rapid prototyping, and that's all, but I'm seeing more of Rails in full blown products
• And finally, my dream -the rise of the Software Enabled business: companies who traditionally haven't been ISVs are now selling software products, now that the barrier for building, selling, and distributing software is lower. I think we'll also see more Fabless software companies - companies being more market focused rather than technology focused, and who may even outsource or license most of their technology rather than build it themselves, much like chip companies or consumer product companies do today.

Macadamian joins ISV NXT

We sent out a press release today announcing that we've joined Microsoft's ISV NXT, a partner program at Microsoft that actively supports the ISV community and the ecosystem around ISVs.

As someone who's worked in product development outsourcing for nearly 10 years, ISV NXT is a revelation. I've bumped into many partner folks from many a platform vendor, including IBM, Oracle, Sun - you name it. They were all eager to get us signed up to their partner program, then their next question would be "so how many licenses of [insert server software here] do you think you can sell in a year"? Upon explaining that we provide engineering and design services to ISVs, not to IT departments, and that we're not a reseller, their next question is always "ummm, yeah... but how many licenses could you sell?" They simply don't get it.

Granted, the amount of commercial software for sale being created (i.e. software created by ISVs) vs. software for use (i.e. internal applications built by IT departments) is small - something like 10% of the software being written, so on one hand you can excuse them for ignoring it. But I think that's changing - more and more product companies are becoming software-enabled - meaning their product includes some component of software - think nav computers in cars, online banking, and so on.

So kudos to Microsoft for "getting it". They are the first to recognize that there is an ecosystem of product development outsourcing companies, and that they are influential in helping ISVs and SaaS companies choose the platform software on which to base their product.

I'm excited about our new partnership with Microsoft, because it gives us and our customers better access to Microsoft resources, which will mean fewer roadblocks when we're helping customers develop their product on the latest pre-release Microsoft platforms.

Do You Trust Your Partner? And other observations from CoDev

I mentioned in a past post that I attended CoDev last month, and the theme was Open Innovation.

Speakers touched on a number of topics, from where to find innovations, to different models like co-development or licensing, to building an open innovation culture at your company. But I was pleasantly surprised to hear one common theme - Trust. The fact that Trust was discussed was not surprising in itself, but rather the fact that nearly every speaker mentioned the importance of Trust and relationships in co-development. One speaker even devoted most of her talk on how to manage and nurture co-development and partner relationships.

I can't stress the importance of establishing trust in a co-development or product outsourcing relationship. While I'm not advocating that you gloss over contract terms, you will never think of all the possibilities and cover everything in the contract up-front, and things will change. What will get you through to a successful release is trust and open communication. I'll never forget a colleague telling me, if you are in the middle of a discussion with an outsourcing partner, and someone says, "Let's look and see what it says in the contract", the relationship is very likely too far gone.

Dr. Alison Lukacsko, VP of R&D at Church and Dwight, had a couple of great slides in her presentation. One showed the results of a study from Vantage Partners, that showed that 52% of co-development arrangements fail because of the relationship. Legal and Financial T&Cs accounted for 14%. But her best slide is one of the most valuable pieces of advice and ways of looking at co-development partnerships that I've ever heard. She says Partnering is about managing differences. You hear people talk about how co-development or outsourcing relationships are like marriages. She likens co-development to being more like raising a teenager. You are two people with different perspectives, and your own lives, trying to bring this product to market and have a successful life, and the project has a life and a mind of it's own.

Here are a few tips

1. Know thyself: What is your corporate culture - formal or informal? How do you make decisions, by consensus or command? What is your operating style - cowboy or process driven? Know these things, and then know them about your partner. It will help you understand your partner's point of view, and reach consensus faster when things get rocky.
2. Bad news has to travel as fast as good: this is one of the toughest things to put into practice, but when you have bad news you have to get it out in the open immediately. This has become a mantra at Macadamian.
3. Discuss the tough stuff up-front: What will happen if the hardware is late? How much time do we realistically need for approvals? One anti-pattern I see often is the tendancy to say "let's cross that bridge when we get to it". I think people worry that having those tough discussions on day-one will somehow hamper the relationship. Because it was in the contract or the SOW doesn't mean everyone read it, or thought it was important ("you mean you're delayed since I didn't get you that answer in 48 hours? I thought that was just a standard T&C). You are better off working out expectations up front, even if it's a tough conversation to have.
4. Communicate often, and speak plainly. 'Nuff said
5. You can't turn your partner into you: I stole this one from Alison Lukacsko. It sums it up nicely - your partner has different motivations, different business goals, and a different culture. You are in this relationship to release a product, not to make them more like you.

Macadamian Named to the Global Services 100

A few years ago, we set ourselves the 5-year goal of becoming known as one of the top companies in the world for software product design and development services.

Today we got a little closer to that goal. Macadamian was named one of the top 100 global service companies by Global Services Magazine and NeoIT, a well-respected analyst and consulting firm that specializes in global sourcing.

In choosing companies for the list, Global Services editors and NeoIT looked at size, capabilities, customer testimonials, and employee retention, to name a few. What's remarkable is that the list wasn't exclusive to product development outsourcing firms, or even IT outsourcing firms, so competition was tough, as there are thousands of BPO and ITO firms worldwide. It looked at all types of outsourcing firms globally, and had to pick 100, so being included on the list as one of a small handful of product design and engineering companies is an acheivement and a real honor.