Crowdtesting Success: Why Crowdsourcing User Testing Continues to Grow and Provide Value

By

Crowdtesting is a successful area of the crowsourcing landscape that provides great value.

Crowdtesting involves a distributed crowd carrying out  user testing for software development and IT projects, usually across a hosted platform. It has grown into an accepted model in mainstream business, used by both large enterprises and smaller companies.  It’s still a fast-growing area with the market leader uTest one of crowdsourcing’s undisputed success stories.

User testing focuses on identifying bugs, issues and potential improvements during the development and pre-launch phase of any software.  Within most software projects there are different types of testing, for example testing which focuses on usability, or “load testing” which tests the response from software experiencing high demand from multiple users.

The advantages of crowdtesting

Overall user testing is absolutely essential for a successful deployment.   Tapping the crowd for user testing has several advantages over more traditional models:

  • Volume:  The higher the number of testers, the more likely the number of bugs and issues are spotted, so this improves the quality of your software implementation.  For load testing, volume is a pre-requisite.
  • Flexibility:  IT projects are inherently unpredictable, and testing rounds are often required at short notice. Pre-booking resources usually means the testing you carry out has to be predictable, which is simply impractical. The crowd operates on an on-demand model, ready to action testing when it is actually needed.
  • Diversity:  Particularly for mobile apps, there are a proliferation of devices and browsers used, across a variety of different markets, groups, languages and scenarios. Using a crowd can identify particular issues across this diverse set of users and technologies.
  • Platform:  It is critical to have a robust platform to distribute what needs to be tested and record the bugs and issues found. Crowdtesting providers have already invested in high-performing and easy-to-use platforms.
  • Cost: Organising testing on-demand across a hosted platform greatly reduces costs, thanks to the flexibility and diversity of the crowd.
  • Availability: Internal resources are not always available, or willing, to give their full attention to user-testing. They often view it as a distraction from their day-to-day work.
  • Experience: Most of the crowd on uTest and its competitors are experienced testers so there is no training required and less risk of miscommunication.
  • Insight:  Because of their experience, testers will often provide additional insights.

From uTest to Applause

The market leader uTest is a good example of crowdsourcing success.

uTest was founded in Massachusetts in 2007, and attracted several rounds of funding, including a $17m D round of funding in late 2011.  In January 2014 the company announced a $43m E round of funding, led by Goldman Sachs.

The continuing investor confidence in the company has been sustained through consistent high growth, a quality client list including Microsoft and Google, and an expansion of its testing population to more than 100,000.  It has also won many awards and featured on lists of promising or fast-growing companies.

The company has always demonstrated its ambition to expand, and has added capabilities along its journey including services to help its testing community and a data insight tool which analyses feedback from user reviews of apps. In late 2013 it announced it would rebrand as Applause, expand its crowd population exponentially and focus on an integrated platform which combines testing and analytics.

Other players

There are a number of other players within the crowdtesting space.

Some of these seem to be on an upward trajectory. For example 99Tests.com, an Indian start-up which had just emerged when we were wrote the first edition of Getting Results from Crowds in 2011, now has an international client base and a testing population of over 7,000. Meanwhile German company Testbirds recently announced a 2.1m Euros Series A round of funding and also released a mobile app to manage crowds of testers.

There are also various specialist platforms which focus on a particular niche of crowdtesting. For example Bugcrowd.com focuses on testers finding security flaws in software, with testers rewarded with a “bug bounty” when they identify an issue.

Meanwhile Mob4hire focuses on mobile testing with 60,000 users with a diverse number of devices across various local markets. Usertesting.com concentrates on detailed crowdsourced and videoed user experiences of websites and mobile apps.

Sustained growth

We have every confidence that rapid expansion in the crowdtesting space will be sustained. This is through a combination of:

  • an unambiguous value proposition
  • demonstrable confidence from investors
  • the addition of bolt-on and complimentary services from major players
  • specialist services and niche providers to meet particular needs
  • a customer-base which includes larger enterprises
  • sustained demand for user testing for software projects, particularly around mobile apps.

Do you think our confidence in the sector is justified? Have you used a crowdtesting service with either positive or negative experiences?  Please let us know your thoughts and experiences!