How HeadSpin is helping Telstra improve performance across its world-class network

About Telstra

Telstra is Australia’s leading telecommunications and information services company. The company provides services to individual, enterprise, and government customers across 20 countries — including service to over 17.4 million mobile subscriptions.
2017 Revenue

28.2 billion

Number of Employees

36,165

Countries of Operation

20

2017 Revenue

17.4 million

Challenges

In order to provide good customer experiences, Telstra needs to be able to measure the performance of their network in a variety of use cases (e.g. time to play Netflix video, voice call quality, messaging service performance, etc.). Historically, real-world performance data has been extremely difficult to measure. Many telcos rely on simulations in lieu of network performance data which often don’t reflect the user experience.

The Solution

Telstra uses HeadSpin to continuously and accurately monitor real world performance measured using real devices running on real networks. These insights allow Telstra to easily identify potential issues, investigate them, and quickly take action to improve their network positioning the company for continued leadership.

Real app testing — anywhere in the world

HeadSpin devices are deployed in nine locations throughout Australia to perform OTT app testing using widely popular apps. Tests are conducted on LTE networks using real Apple, Samsung and Google Pixel devices.

1. Measure real-world OTT app performance on the Telstra network

Today’s customers rely on third party mobile apps for information, entertainment and to get things done. Telstra needs to be able to monitor how such top apps perform in the real world, on Telstra’s networks not just in simulations. However, real-world performance has historically been difficult to measure.

What HeadSpin does

HeadSpin allows Telstra to continuously monitor real-world network performance. Every 10 minutes, HeadSpin automatically collects KPI data for 25 popular apps measured using real devices placed in nine different locations throughout Australia. HeadSpin uses this data to provide Telstra with valuable insights about network performance and alerts engineers of potential outages, problems, or other issues.

2. Compare network performance with competitors

Network performance is the leading cause of customer attrition in the telecommunication industry. In order to protect its business, Telstra needs to perform at least as well as (if not better than) other networks in Australia.

What HeadSpin does

Using devices equipped with SIMs from rival networks, HeadSpin lets Telstra compare their network performance with KPIs from Optus and Vodafone. HeadSpin can automatically alert Telstra engineers if performance begins to suffer or fall behind against competing networks.

3. Measure and track app KPIs that affect the user's experience

Different apps demand that different KPIs be measured, especially for OTT applications that require more data (because of audio and video). This means there are several additional considerations that the network teams needs to understand. They need additional visibility on these core parameters.

What Headpin does

HeadSpin’s digital experience platform can be customized to measure and document different KPIs from different applications. For example, HeadSpin tells Telstra when video content on Netflix or Youtube was jittered, garbled, missing speech, etc.

4. Monitor and benchmark messaging apps infrastructure and performance

Modern users rely heavily on messaging (SMS, MMS, etc.) in their day-to-day lives, so performance must be quick and seamless. However, not all messaging services (iMessage, Android SMS, Samsung SMS,WhatsApp, etc.) are alike. Also, some communications are routed through Telstra’s Rich CommunicationServer.

What HeadSpin does

HeadSpin allows Telstra to continuously measure the core messaging KPIs like time it takes to send a message, time it takes to receive a message etc and compare this across OTT, in-house applications and competing networks. HeadSpin also benchmarks the performance of Telstra’s Rich Communication Servers and compare performance to default messaging infrastructures.

5. Test core voice services

Customers expect flawless voice performance on Telstra’s network. However, Telstra allows customers to route voice calls in multiple ways via Wi-Fi, LTE for voice, Skype, or analog voice which opens up
room for inconsistent performance.

What HeadSpin does

HeadSpin allows Telstra to measure the performance of analog voice and compare it with voice over LTE and voice over Wi-Fi, helping ensure that voice service is consistent — regardless of connection method.Voice use cases include peer to peer calls, 911, Skype, FaceTime, etc.

6. Monitor roaming performance internationally

Telstra has several partner networks throughout Asia and Europe. To ensure that customers experience the same performance abroad as they do at home, Telstra needs to be able to measure how apps perform while roaming.

What HeadSpin does

HeadSpin allows Telstra to monitor real-world roaming performance in international markets measured using devices with Telstra SIM cards running on partner networks around the world.

7. Get deep client-network-server insights that lead to action

Simply understanding the presence of a network problems is a good start, but addressing such problems require further insight. For example, was a problem caused by a bad API? The CDN? Native device code? Something else?

What HeadSpin does

HeadSpin does more that simply tell Telstra when they have a problem — it also shows engineers why and how the problem is happening. HeadSpin AI solution provides UX data (in session UI, burst UI, waterfall UI, etc) and can automatically capture important info from debug logs to assists teams troubleshooting problems.

8. Benchmark live TV and video on demand (VOD) performance

Today’s mobile users demand seamless live video performance. Mobile subscribers use Telstra’s network to stream live TV shows and events like the Australian Football League. However, the quality of live video has historically been difficult to measure.

What HeadSpin does

HeadSpin allows Telstra to measure and quantify the quality and performance of live video. HeadSpin provides video performance metrics using the following indicators: Commercial Black, Blockiness, Block Loss, Blur, Contrast, Exposure, Flickering, Freezing, Interlacing, Letter-boxing, Noise, Pillar-boxing, Slicing, Spatial Activity, Temporal Activity.

Blockiness Indicator

Contrast Indicator

Slicing Indicator

Spatial Activity Indicator

Improving Performance

Insights provided by HeadSpin’s digital experience platform help Telstra improve network performance in a variety of ways.

9. Conduct drive tests to understand network performance on the move

Today’s mobile users are constantly on the go — and they expect their network to perform equally well in all areas as they move through their day. However, real-world testing of networks and user flows in top mobile applications has historically been extremely difficult to perform on-the-go.

What HeadSpin does

HeadSpin’s mobile physical boxes allow Telstra to experience their network in a familiar real-world use case — while driving. Accurate geo locations, correlated video and audio feeds, and video and MOS quality scores allow Telstra to accurately identify areas in its network that offer poor user experience.

10. Test free WiFi hotspots throughout Telstra's network

Telstra Air is Australia's largest WiFi hotspot network. Over one million hotspots throughout Australia allow Telstra customers to access free WiFi data. However, due to Telstra Air’s increasing popularity, these hotspots are becoming congestion points for the network.

What HeadSpin does

Telstra uses HeadSpin to gain a detailed understanding of congestion across their entire network, which allows Telstra to plan and build a more robust network infrastructure. HeadSpin allows Telstra a wide range of mobile performance tests (voice over lte, voice over WiFi, video and audio streaming, video quality, etc.) while switching back and forth between network and WiFi hotspot connections.

11. Understand the full streaming experience for TV sticks on broadband

In addition to accessing videos on mobile, many customers access streaming video content using Telstra’s broadband service coupled with a TV stick (Google Chromecast, Amazon Fire, Roku Stick Media, etc.).

What HeadSpin does

HeadSpin allows Telstra to understand video performance on their residential and commercial broadband network. This insight provides Telstra with a complete understanding of their customer’s entire experience streaming video on their network — whether at home, in the office, or on the go.

Improving Performance

Insights provided by HeadSpin’s digital experience platform help Telstra improve network performance in a variety of ways.

12. Help Telstra build better in-house apps (Telstra Smart Home, innate, etc)

Telstra also builds its own suite of customer-facing apps — such as a home security systems app, an application that monitors use and performs electronic billing, Telstra data usage and an app for the company’s email service.

What HeadSpin does

In addition to supporting network performance, HeadSpin gives Telstra an innovative testing platform that lets the company improve their products’ user experience.

13. Ensure device-OS-carrier compatibility for pre-release mobile devices

Theoretically, device manufacturers are supposed to adhere to strict standards when creating new mobile devices. However, in practice sometimes new devices aren’t compatible with all mobile networks.

What HeadSpin does

HeadSpin allows Telstra to test each new device in real-world network conditions using real SIM cards — prior to the release of the device. This allows Telstra to troubleshoot potential network issues, notify OEM manufacturers of compatibility or performance problems and avoid negative customer experiences.

14. Prepare for the future with 5G networks and eMBMS testing

5G is coming, and it’s going to change everything. 5G will enable previously impossible connectivity and latency, while powering a massive web of connected devices — the internet of things. There aren’t that many 5G supported devices in the market. On a separate note, eMBMS or LTE Broadcast requires massive amounts of investment and infrastructure.

What HeadSpin does

Although none of Telstra’s handsets support 5G (yet), HeadSpin is helping Telstra prepare for the coming move to 5G — monitoring performance of test 5G network infrastructure, tracking and monitoring network KPIs, and providing insights from speed tests, packet loss, etc on Samsung S9 Plus phones(which provides LTE plus capabilities) across 200 devices in Australia. HeadSpin is also helping Telstra optimize for video delivery and network performance as part of their eMBMS experimentation.