mopdf.com

xvAlarm ProcessingCorrelation178179Simple Correlation180Advanced Correlation180Complex Correlations181Recommendations for VoIP-Centric Network Management Framework 182Performance Analysis from a Transit Network Perspective 183Signaling Protocol Transport OptimizationEnterprise Networks184184Cisco IOS QoS Recommended SNMP Polling Guidelines 187Case Study of Link CongestionsSP Networks187194Performance Data in an Enterprise VoIP EnvironmentCPU Status197198Physical Memory 198Hard Disk Status199High Utilization of Disk Space199Virtual Memory 199Number of Active Phones200Gateway Registration (MGCP)200Gatekeeper Registration (H.323 RAS)Calls in ProgressCalls Active200201201Calls Attempted202Calls Completed202PRI Channels Active203Conferencing/Transcoding DSP’s DepletionAvailable Bandwidth of a Location (CAC)203204Recommendations for Categorizing Performance MeasurementsEnterprise Case Study—Analyzing Network Performance206CPU Rate and Critical Processes 206Rate of Active Calls207Tracking Trunk Utilization for PSTN AccessTrend Analysis Best Practices208211Performance Analysis from Call Agent Perspective 211Performance Analysis for VoIP Call Traffic 211Performance Analysis for a PSTN Network (PSTN Trunk and SS7Signaling) 215204

xviVoIP Performance Management and OptimizationPerformance Analysis for an SIP Network217Performance Tracking for a Session Border Controller (SBC)218Performance Information Through the Call Detail Records (CDR)219Performance Enhancement Schemes and Their Effect on VoIP NetworkMonitoring 220Effect of DNS Caching220Server Load Balancing220Firewall220Optimizing the SBC 221Performance Analysis from a DOCSIS NetworkVoIP Endpoints222DOCSIS/DQoS224CPU Impact/Link Utilization221226Trace Log Monitoring on Softswitch and Network Devices 229Analyzing and Correlating Syslog Messages 230Log Files ManagementSecurity231231Storage Location (Local Versus Remote) and Archiving LogsTools and Scripts233234Tools for Monitoring an Enterprise VoIP NetworkCisco Unified Operations Manager (CUOM)234234Cisco Unified Service Manager 236Cisco Unified Service Statistics Manager 237Tools for Monitoring Service Provider VoIP Networks 239IXIA’s IxRave Solution239IxRave Case Study—Voice Assurance for Cable NetworksTools for Monitoring DOCSIS Networks—VoIP Dashboard240242Tools for Monitoring VoIP Network Health Through ProtocolsTools for Analyzing Call Detail RecordsSP CDR Report Scenario246246Customizing CDR Reporting for Effective MonitoringDashboard Views for the VoIP NetworkSoftware Maintenance247248Software Release ManagementSoftware Lifecycle Management249249Software Resiliency 251Periodic Auditing of a VoIP NetworkSummary 254References 254251247244

xviiChapter 8Trend Analysis and Optimization257Trend Analysis Through Key Metrics 258Dashboard as a Profiling Tool 259Network Utilization and Efficiency 260Safeguarding Network Resources from Potential Security ThreatsDashboard for Trunk Resources UtilizationFeedback for Change Control265266Profiling in an SP VoIP Network271Profiling in an Enterprise VoIP Network277Balancing the Device Load on CUCM Cluster Nodes278Maximizing Trunk Capacity and Avoiding Call BlockingCall Detail Record–Based Trend AnalysisBenchmarking261280283283Verifying VoIP Network Resources CapacitySLA Conformance284286Monitoring for Service Availability286Normal Versus Abnormal Termination Profiling: Categorizing andCorrelating the Call Termination Code 288Monitoring for Service Quality289Verifying Toll Savings (On-net Versus Off-net Profiling)Detecting Toll Frauds289291Resource Optimization and Capacity Planning291Network Resource Utilization and Optimization291Capacity Planning and Upgrade Strategies 296Managing Subscriber Growth Impact by Using Trend AnalysisUC Manager Cluster Capacity298298Network Bandwidth and Transcoding DSPs299Considerations for Adding Trunk Capacity302Summary 302References 302Part IVAppendixesAScripts and Tools for Monitoring and Troubleshooting VoIP Networks 305BDetailed Call FlowsCVoIP Dashboard 367DDebugs, Traces, and LogsIndex409331373

xviiiVoIP Performance Management and OptimizationIcons Used in This tchCommunicationServerPCATMSwitchGatewayPC CSUISDN/Frame RelaySwitchAccess Cisco WorksWorkstationFront EndProcessorLine: EthernetToken RingLine: SerialFDDILine: Switched SerialNetwork CloudFrame Relay Virtual CircuitClusterController

xixCommand Syntax ConventionsThe conventions used to present command syntax in this book are the same conventionsused in the IOS Command Reference. The Command Reference describes these conventions as follows: Boldface indicates commands and keywords that are entered literally as shown.In actual configuration examples and output (not general command syntax),boldface indicates commands that are manually input by the user (such as ashow command). Italic indicates arguments for which you supply actual values. Vertical bars ( ) separate alternative, mutually exclusive elements. Square brackets ([ ]) indicate an optional element. Braces ({ }) indicate a required choice. Braces within brackets () indicate a required choice within an optional element.

xxVoIP Performance Management and OptimizationForewordAccording to a Gartner market share analysis done for Enterprise Unified Communicationson June 23, 2009, the total vendor revenue from the entire enterprise unified communications segment in 2008 was 5.1 billion. FierceVoIP quoted ISP-Planet study in its onlinenewsletter on July 28, 2008, saying that the total subscribers in just the United States forthe top 10 VoIP service providers are around 18 million and growing in double digits.Comcast came in on top at 5.2 subscribers followed by Time Warner (3.2 million) andVonage (2.6 million), based on their first-quarter reporting in 2008. This puts great emphasison managing VoIP for both enterprises and service providers.Management of a VoIP network is a cyclic process that starts even before VoIP isdeployed. The first stage is planning, which includes forming a team, defining the scopeof deployment, requirements validation, and assessment of the IP network to determinewhether the infrastructure is adequate to support media traffic. The design phaseincludes comprehensive design-based traffic engineering and validated requirements. Itnot only covers call-processing servers, remote gateways, and features implementationbut also covers changes to the IP network in the form of quality of service design andprovisions for network resiliency. It is followed by the implementation phase, which isgoverned by project management principles and ensures that best practices for deployment are followed for on-time completion.Implementation also includes a test plan execution and transfer of information to network operations prior to commissioning. The correct execution of these phases ensuresminimum problems and decreases the total cost of deployment. Implementation is followed by the operations phase, with continuous optimization to close the loop. Thisbook briefly mentions planning, design, and implementation stages and emphasizes theoperations and optimization phases.First, the hand-off to operations needs to be complete, including remediation of any issuesdiscovered when the postdeployment test plan was executed. All the deployed devicesmust be discovered by the network management systems. But most important, VoIP canno longer be managed in a silo that is separate from the data network management subteam.This book emphasizes correlating network problems with VoIP-related key performanceindicators for faster problem resolution by isolating it and fixing the root cause.Operational data provides critical feedback for continuous optimization of the network,including its performance and capacity. Optimization is not limited to fine-tuning thetraffic engineering process for future growth but also for extending VoIP for the nextevolution to collaboration-enabled business transformation.What is presented in this book is the authors’ collective experience and knowledge, workingwith several other colleagues from Advanced Services, Cisco Remote Operations Service,the product development teams, and most important, Cisco customers, whose feedbackwas critical in developing best practices for VoIP management and optimization.Regards,Talal Siddiqui, Senior Manager, Unified Communication/Collaboration Practice CiscoAdvanced Services

xxiIntroductionWith the exponential growth of the Internet and an increasing number of VoIP deployments, customers are looking for new ways to manage and scale their networks to meetthe growing needs of end users. Customers not only need to fix problems in a timelymanner with minimal downtime, but they also need to proactively monitor their networksto fix potential problems before they become service and revenue impacting.The complexity of an IP network increases with the addition of new services, and asthese networks start to scale, managing them becomes a challenge. Customers are lookingfor new ways to manage their networks and effectively scale these services.Customers are looking for new techniques and efficient ways to monitor multivendorproducts in the network and use tools/applications that can scale with the growth of theirnetworks. We got feedback from our customers and VoIP SPs through forums such asSANOG, NANOG, APRICOT, and Cisco Live (formerly known as Networkers) aboutwhat they would like to see in a VoIP management book. This feedback can be boileddown to “We want a practical guide with specific details and examples that we can useright away.something that is a desk reference for NOC (Network Operations Center)staff and the network architects.”This book addresses some of the challenges associated with deploying and managingVoIP networks and also provides guidelines on how to optimize these networks.Goals and MethodsThe most important goal of this book is to help define a methodology and frameworkof collecting, analyzing, and correlating VoIP performance data from various networkelements. When correlated in a meaningful way, this data can help network operatorsidentify problematic trends in their VoIP networks, and isolate and fix problems beforethey become service impacting.One key methodology in this book is to use a layered approach when troubleshootingVoIP network problems. This helps narrow the scope of the problem in an efficient mannerand also helps find the root cause. By quickly identifying the root cause of the problem,the network operator can resolve issues in a timely manner and minimize customer impact.This book also provides guidelines for optimizing VoIP networks by defining the following: What VoIP performance data should be collected from various network elements? How to collect VOIP performance data? How to use dashboards to analyze and correlate VoIP metrics? How to use the VoIP dashboard for trend analysis and capacity planning?

xxiiVoIP Performance Management and OptimizationWho Should Read This BookThis book is meant to be used as a guide by network engineers, architects, and operationspersonnel in managing and optimizing their VoIP networks.This book also helps network operators troubleshoot VoIP-related issues efficiently andidentify root causes to fix problems in a timely manner. However, it does not focus ontraces, logs, and debug messages but rather on analyzing trends and correlating networkissues to address core issues. This book compliments other Cisco Press publications: Kaza, Ramesh and Asadullah, Salman. Cisco IP Telephony: Planning, Design,Implementation, Operation, and Optimization. Indianapolis, IN: Cisco Press,February 23, 2005. Halmmark, Addis, Giralt, Paul and Smith, Anne. Troubleshooting Cisco IPTelephony. Indianapolis, IN: Cisco Press, December 11, 2002. Clemm, Alexander. Network Management Fundamentals. Indianapolis, IN:Cisco Press, Nov 21, 2006.How This Book Is OrganizedThis book discusses some of the challenges faced by service providers and enterprisecustomers in deploying, managing, and optimizing VoIP in their networks. It providesguidance on how to address voice quality issues and proactively monitor key performance indicators (KPI) to help gauge the health of the VoIP network.The first part of the book provides an overview of VoIP and key network managementconcepts. It also discusses a metrics-based approach of managing and optimizing VoIPnetworks.The second part of the book concentrates on different VoIP deployment models in SPand enterprise networks, and reviews the common VoIP-related problems in eachdeployment approach.Note The first and second parts of the book set the stage for how VoIP is deployed inenterprise and SP networks and discusses the challenges associated with such deployments. You might feel that both these parts of the book are brief and high-level; they donot cover in-depth technology and protocol details. For example, what is DOCSIS and howdoes it work? How does the Session Initiation Protocol (SIP) work, and what are the various SIP messages? This is by design; it is assumed that you already understand these basicsas this information has already been covered in various other texts. The main focus of thisbook is on managing and optimizing VoIP networks; these concepts are covered in detail inthe third part of the book. That is why chapters in the third part of the book are longerand more detailed than the chapters in the first and second parts of the book.

xxiiiThe third part of the book focuses on a proactive approach to diagnosing problems inVoIP networks and fixing these problems before they become service impacting. Thispart of the book also talks about what tools can be used by customers in gauging thehealth of their VoIP network and improve network performance. Using performancecounters, Call Detail Records (CDR), and Call Agent trace logs, customers can utilize realtime data to gauge the health of their voice network and make capacity-planning decisions before network resources get congested.Chapters 1 through 8 cover the following topics: Chapter 1, “Voice over IP (VoIP) and Network Management”: This chapter talksabout VoIP media transport and signaling protocols, some common voice qualityissues, and their causes. The second half of the chapter discusses network management methodologies such as Telecommunications Management Network(TMN); Fault, Configuration, Accounting, Performance, and Security (FCAPS);and Information Technology Infrastructure Library (ITIL). It also talks about thestrategic importance of managing VoIP networks. Chapter 2, “A Metrics-Based Approach for Managing the VoIP Network”: Thischapter highlights the key performance indicators that can be utilized to effectively manage a VoIP network. It encourages the use of a layered approach forisolating localized and systemic issues. It explains how performance data fromvarious network segments and service flows can be used to manage SLAs in aVoIP network. Chapter 3, “VoIP Deployment Models in Service Provider Networks”: This chapter discusses various VoIP solutions in an SP environment. The deploymentmodels cover scenarios in which broadband SPs provide VoIP service to residential and business customers. These providers own the last-mile connection toend users; they use their infrastructure to not only provide Internet connectivitybut also to offer VoIP services using the same infrastructure. Because they ownthe last-mile connection and the VoIP infrastructure, they can provide betterQoS to VoIP traffic and offer high-quality VoIP services. Chapter 4, “Internet Telephony”: This chapter describes how VoIP is deployedover a publicly shared infrastructure such as the Internet. In such deploymentmodels, the company providing VoIP services might not own the entire networkinfrastructure, such as the last-mile connection to the end users, which is usedfor deploying this service. They might use infrastructure, owned by other entities, to provide VoIP as an overlay service by deploying some of their ownnetwork components that are required for offering the VoIP service. Thisdeployment model is different from the models discussed in Chapter 3. TheVoIP SP is faced with several challenges with providing QoS to VoIP traffic;these issues are also discussed in this chapter. Chapter 5, “VoIP Deployment Models in Enterprise Networks”: This chapterexplains various deployment models that are commonly used in typical enterprise networks, including the fundamental models: central call processing anddistributed call processing. It also discusses large-campus deployment schemes.

xxivVoIP Performance Management and OptimizationThis chapter discusses the differences in hosted and managed services aroundUnified Communications solutions. It also presents a brief overview of IPContact Centers, which are essentially an extended functionality of a UnifiedCommunications solution. Chapter 6, “Managing VoIP Networks”: This chapter discusses the best practicesfor planning media deployment over IP networks starting from how to assess thereadiness of the network, traffic engineering, high availability, and managing theIP network and its integrated components that process voice and other mediatransmissions. This chapter also covers the monitoring mechanism available tonetwork administrators and their scope and effectiveness in managing VoIPnetworks. Chapter 7, “Performance Analysis and Fault Isolation”: This chapter discusses anapproach for proactive monitoring of the VoIP network for performance analysisand fault isolation of problems caused by anomalies in the network. It startswith explaining the VoIP network monitoring aspects including collection, categorization, and correlation of performance counters for both enterprise andservice provider networks. It also discusses different ways of gauging the performance of a large-scale VoIP network by looking at various key performanceindicators (KPIs). Chapter 8, “Trend Analysis and Optimization”: This chapter explains the use ofVoIP dashboards to monitor and trend performance data from different components in the VoIP network. This trend analysis can help network operators notonly establish a baseline but also help with resource optimization and capacityplanning by looking at problematic trends in the network, such as resourceoverutilization and changes in traffic patterns.

Chapter 3VoIP Deployment Models inService Provider NetworksThis chapter gives you an understanding of how Voice over IP (VoIP) is deployed in service provider (SP) networks. This chapter focuses on describing a use case in which theVoIP infrastructure and the transport and the access are managed by an SP. Chapter 4,“Internet Telephony,” focuses on VoIP networks in which only the VoIP infrastructure ismanaged. Different network components and their functions are described to illustratehow various call functions are implemented to provide voice services to residential andbusiness customers. Figure 3-1 depicts a block architecture of the SP scenarios discussedin this chapter. Here, the service provider also owns the last-mile network access. Laterchapters cover scenarios where the SP does not own the access network.Bundled Services Provider(Network Access VoIP SP)Call ControlBroadbandRouter ModemEdge RouterNetwork Access in of Control BoundaryCustomer PremisesNetwork Access ProviderVoIP NetworkPSTNFigure 3-1 Service Provider Architecture Overview

54VoIP Performance Management and OptimizationThis chapter provides a high-level view

ATA-Based Voice Applications 74 Traffic Profiling 74 Potential Bottlenecks 75 . Cisco IOS QoS Recommended SNMP Polling Guidelines 187 Case Study of Link Congestions 187 SP Networks 194 Performance