The Challenge

8x8’s unified communication platform boasts a large number of essential enterprise features that differentiate their product in a competitive market. The complex nature of 8x8’s product left Mayu Patel, Director of Quality Assurance, with a several key challenges.

 

  • A dizzying number of tests to conduct 
    8x8’s platform required ongoing tests of hundreds of unique user flows (i.e. host and/or participant ability to join or leave PSTN calls, mobile-to-mobile calls, three-party calls and/or conferences, accessing voicemail, utilizing SMS services, accessing and managing contacts, etc.). Each of these use cases needed to be tested on different kinds of devices and/or different operating systems. As Mayu sums up, “We needed to test more things in less time.”

  • Understanding carrier-based performance 
    “Our Unified Communication platform is not WiFi-only — we heavily rely on local mobile carriers. So it was important that we could understand how our platform performed on these carrier networks,” says Mayu. “We needed real, SIM-equipped devices to conduct carrier-based testing.”

  • Testing in different Geos 
    Prior to HeadSpin, all of 8x8’s testing was conducted in house, which wasn’t efficient or scalable. “When you go to country x or y, then you're going to have a completely different experience than in the US — the network might go down or you have to deal with poor coverage. We needed the ability to get test devices in sensitive locations — or places where we might want to grow our business.”

 

 

 

The Solution

HeadSpin’s innovative platform gives Mayu an simple way to test the performance of 8x8’s products.

 

Automated testing

HeadSpin gives 8x8 a single interface for automated user flow testing. “We needed an automation drive, so that we could reduce the number of man hours needed to conduct testing,” says Mayu. “HeadSpin lets us proactively monitor and identify potential issues before the customer even starts to notice things like lag, downtime or degradation in our services. That helps us be better prepared.”

Given how HeadSpin has been performing, Mayu’s team’s use of HeadSpin is expanding rapidly. “At present, we have 250 unique test cases that we run during off hours — which sums up to 1000-plus total tests.” With in the next few months, Mayu predicts that his team will be scale their use of HeadSpin to more that 700 unique tests.

 

Cloud-based geo testing

8x8’s QA is no longer tied to any particular geographic location. With 22,000 real, SIM-card equipped devices in more than 150 locations globally, HeadSpin allows 8x8 to be responsive in t their testing. “Currently, we’re testing in seven different locations — all inside of the US. But now we have ability to go in the cloud and bring up devices in particular location. If we can quickly bring up a device on particular carrier in a specific country, then we are definitely proactively enabled. We see that as a huge value add from HeadSpin.”

 

Better understanding of performance issues

Insights provided by HeadSpin allow 8x8 to deeply understand the real-world performance of their platform. “If you have automated, programmatically measured times and performance data, then you can know a lot about specific issues you might see.”

“With HeadSpin, we were finally able to identify some of the difficult to reproduce problems. You sometimes get into situations where customers say, ‘Oh, I experience this.” But then you try and try and you can’t reproduce the problem. Having the ability to run a broad number of devices enabled us to go back, experience, and fix some of these hard-to-find problems.”

“Big picture, the fact that you can reproduce errors and test them, and test them and then potentially fix them, you're improving the user experience. And that means you're not having people call and opening tickets, and you’re just miles ahead.”

 

Get started with HeadSpin today

Contact Sales
Start Free Trial