HOW MICROSOFT DEFINES QUALITY
Microsoft defines quality as a combination of service metrics and user experience. This aligns with the famous quote by Peter Drucker: "You can't manage what you can't measure." Therefore, measuring both service health and user sentiment is crucial for delivering exceptional quality within Microsoft Teams.
This reinforces the importance of the metrics we'll be discussing throughout this document, as they provide the necessary data to objectively assess service performance and identify areas for improvement impacting user experience.
Service Metrics
Client-Based Metrics:
To proactively identify and address potential issues impacting user experience, Microsoft Teams offers a comprehensive set of client-based metrics collected for each call. These metrics provide valuable insights into the technical health of Teams sessions. Here are some key examples:
- Telemetry: This includes metrics like jitter, packet loss, and round-trip time (RTT) that measure network performance and its impact on audio, video, and screen sharing quality.
- Reliability: Metrics like dropped calls and multiple attempts to join calls indicate potential issues with call stability.
- Endpoint: Information about used headsets and the chosen transport protocol (UDP or TCP) can help diagnose endpoint-specific problems.
- Client: Monitoring client updates and VPN split tunneling functionality ensures optimal client performance and avoids connectivity issues.
User Experience
While service metrics provide objective data, user experience (UX) in Teams is subjective. Users might perceive issues even when the network and service function normally. This subjectivity can complicate troubleshooting. Here's where correlating service metrics with user experience becomes crucial:
- Meeting/Call Access: Monitoring successful joins helps identify access or permission-related problems.
- Audio Quality: User feedback on audio quality during calls can point to network issues or audio device problems.
- Video Quality: Blurry or choppy video can be caused by insufficient bandwidth, network congestion, or client limitations.
- Screen Sharing: Monitoring screen sharing functionality and user feedback ensures smooth collaboration during meetings.
1. Latency One-Way or Ping In Millisecond and Round-Trip Latency
Definition:
Latency measures the time it takes for a data packet to travel from a user's device (point A) to the Microsoft Teams service (point B) and back again. It's influenced by the physical distance between the user and the server, transmission speed, and processing delays within routers along the path.
Impact on user experience:
Latency directly affects the smoothness of conversations. Higher latency introduces unnatural pauses, making communication feel disjointed, similar to using a satellite phone. This can lead to accidental talking over each other, hindering communication flow.
2. Packet Loss Rate
Definition:
This metric indicates the percentage of data packets that fail to reach their destination within a specific timeframe (e.g., 15 seconds). For instance, if 1000 packets are sent in 15 seconds and 50 are lost, the packet loss rate would be 5%. Packet loss is crucial for Voice over IP (VoIP) applications.
Impact on user experience:
High packet loss results in dropped audio or periods of silence during calls. It can also degrade voice quality, creating a robotic or distorted sound. Microsoft recommends keeping packet loss below 1% for optimal call quality. Here's a general guideline for interpreting packet loss impact:
-
- Less than 3%: Offers decent call quality.
- Between 3% and 7%: Noticeable impact on call quality.
- Over 7%: Severe degradation in call quality.
3. Packet Reorder Ratio
Definition:
This metric reflects the percentage of data packets that arrive at their destination out of order compared to the sending sequence. High-traffic networks are more susceptible to packet reordering.
Impact on user experience:
Network protocols often misinterpret packet reordering as packet loss or a congestion signal, impacting the network MOS similarly to actual packet loss. Additionally, reordering can disrupt the packet sending rate, leading to increased round-trip time and further performance degradation. Consequently, calls may sound distorted and experience cutouts. A common alert threshold for packet reordering is 0.05%.
4. Jitter: Also Called Packet Inter-Arrival Time
Definition:
During a Teams call, audio packets are ideally sent at regular intervals. However, network latency can cause these packets to arrive out of sync. To compensate, a buffer is employed to temporarily store packets before reconstructing them in the correct order. Jitter essentially reflects the size of this buffer needed for smooth playback.
Impact on user experience:
A low jitter value signifies a good, stable connection. Conversely, a large jitter buffer indicates network congestion. As the buffer size fluctuates to accommodate varying packet arrival times, it introduces additional delays in the call. This can manifest as audio distortion, with callers' speech speeding up or slowing down.
5. Ratio Concealed Sample Average
Mitigating Packet Loss:
When network packets are lost, audio samples can be concealed to compensate. This is a technique to smoothen out the abrupt transitions caused by dropped packets. Hence, the term "Healed Percentage" is often used for this statistic.
Impact On User Experience:
A high "Healed Percentage" indicates that many audio samples had to be concealed due to packet loss, signifying poor audio quality for users, manifesting as distorted or lost audio. It's generally recommended to keep this ratio around or below 2%. Values exceeding 2% will gradually degrade audio quality, and users
6. Estimated Bandwidth:Bandwidth Minimum, Maximum and Average
While Microsoft Teams can adapt to varying bandwidths, audio quality is demonstrably impacted when bandwidth dips below 100 Kbps. Video calls are even more susceptible to bandwidth limitations and packet loss. Therefore, monitoring and maintaining adequate bandwidth within Microsoft's recommendations is crucial.
Here's how to address bandwidth considerations:
Microsoft's Bandwidth Recommendations:
Refer to Microsoft's official documentation for their recommended bandwidth requirements for optimal Teams performance, considering factors like call type (one-on-one vs. group calls) and desired video quality.
Monitoring Bandwidth:
Implementing a monitoring solution is essential to ensure consistent bandwidth availability.
Optimizing Networks for Quality and Reliability
Below are the eight key configuration settings to optimize Teams performance and ensure a consistent, reliable user experience:
- Quality of Service (QoS): Prioritize media traffic on congested networks to maintain packet integrity.
- M365 Traffic Bypassing Proxy: Bypass proxy and inspection services commonly used for web browsing to avoid audio delays.
- Split Tunneling for VPN: Facilitate direct connections to Microsoft cloud endpoints for VPN users through split tunneling.
- Open Ports and Protocols: Ensure subnets, ports, and protocols required for Teams signaling and media traffic are open.
- Microsoft Certified Devices: Utilize certified phones and devices for seamless operation without additional configuration.
- Local DNS Resolution: Leverage Microsoft's global deployment with geo-based DNS for efficient service allocation.
- Shortest Path to the Internet: Route traffic to the internet as close as possible to the endpoint for faster connection to Microsoft's network.
- Antivirus/DLP Exclusions: Exclude Teams processes from antivirus or data loss prevention (DLP) scans to avoid performance disruptions.
Bandwidth Considerations and User Experience
Bandwidth And Call Quality:
While Teams adapts to varying bandwidths, audio quality prioritizes over video. With unlimited bandwidth, Teams optimizes media quality up to 1080p resolution and 30 FPS.
Impact Of Variable Bandwidth:
Network limitations and high latency can cause lag, impacting the user experience.
Recommended Bandwidth Speeds:
Refer to Microsoft's recommendations for optimal performance based on factors like video resolution and FPS.
Assessing User Experience with Rate My Call
Microsoft Teams Feature:
Rate My Call is a built-in feature that prompts users to rate their call experience every 10 calls or 10% of total calls.
Subjective Feedback:
This voluntary survey provides user feedback on call quality, with a 5-star rating system (1-2 stars = poor, 3-4 stars = good, 5 stars = excellent).
Limitations:
-
- Not real-time feedback for every call.
- Users might ignore prompts after good calls, leading to skewed data towards negative experiences.
- Requires correlation with service metrics for a complete picture.