Titlepage How To Manage Your Network with SPECTRUM Document 1909 Network Management Copyright Notice Document 1909. Copyright © 2000-present Aprisma Management Technologies, Inc., 273 Corporate Drive, Portsmouth, NH 03801 USA. All rights reserved worldwide. Use, duplication, or disclosure by the United States government is subject to the restrictions set forth in DFARS 252.227-7013(c)(1)(ii) and FAR 52.227-19. Liability Disclaimer Aprisma Management Technologies, Inc. ("Aprisma") reserves the right to make changes in specifications and other information contained in this document without prior notice. In all cases, the reader should contact Aprisma to inquire if any changes have been made. The hardware, firmware, or software described in this manual is subject to change without notice. IN NO EVENT SHALL APRISMA, ITS EMPLOYEES, OFFICERS, DIRECTORS, AGENTS, OR AFFILIATES BE LIABLE FOR ANY INCIDENTAL, INDIRECT, SPECIAL, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING BUT NOT LIMITED TO LOST PROFITS) ARISING OUT OF OR RELATED TO THIS MANUAL OR THE INFORMATION CONTAINED IN IT, EVEN IF APRISMA HAS BEEN ADVISED OF, HAS KNOWN, OR SHOULD HAVE KNOWN, THE POSSIBILITY OF SUCH DAMAGES. Trademark, Service Mark, and Logo Information SPECTRUM, IMT, and the SPECTRUM IMT/VNM logo are registered trademarks of Aprisma Management Technologies, Inc., or its affiliates. APRISMA, APRISMA MANAGEMENT TECHNOLOGIES, the APRISMA MANAGEMENT TECHNOLOGIES logo, MANAGE WHAT MATTERS, DCM, VNM, SpectroGRAPH, SpectroSERVER, Inductive Modeling Technology, Device Communications Manager, SPECTRUM Security Manager, and Virtual Network Machine are unregistered trademarks of Aprisma Management Technologies, Inc., or its affiliates. For a complete list of Aprisma trademarks, service marks, and trade names, go to http://www.aprisma.com/manuals/trademark-list.htm. All referenced trademarks, service marks, and trade names identified in this document, whether registered or unregistered, are the intellectual property of their respective owners. No rights are granted by Aprisma Management Technologies, Inc., to use such marks, whether by implication, estoppel, or otherwise. If you have comments or concerns about trademark or copyright references, please send an e-mail to [email protected]; we will do our best to help. How To Manage Your Network with SPECTRUM Page 2 Restricted Rights Notice (Applicable to licenses to the United States government only.) This software and/or user documentation is/are provided with RESTRICTED AND LIMITED RIGHTS. Use, duplication, or disclosure by the government is subject to restrictions as set forth in FAR 52.227-14 (June 1987) Alternate III (g)(3) (June 1987), FAR 52.227-19 (June 1987), or DFARS 52.227-7013 (c)(1)(ii) (June 1988), and/or in similar or successor clauses in the FAR or DFARS, or in the DOD or NASA FAR Supplement, as applicable. Contractor/manufacturer is Aprisma Management Technologies, Inc., 121 Technology Drive, Durham, NH 03824. In the event the government seeks to obtain the software pursuant to standard commercial practice, this software agreement, instead of the noted regulatory clauses, shall control the terms of the government's license. Virus Disclaimer Aprisma makes no representations or warranties to the effect that the licensed software is virusfree. Aprisma has tested its software with current virus-checking technologies. However, because no anti-virus system is 100 percent effective, we strongly recommend that you write-protect the licensed software and verify (with an anti-virus system in which you have confidence) that the licensed software, prior to installation, is virus-free. Contact Information Aprisma Management Technologies, Inc. 273 Corporate Drive Portsmouth, NH 03801 Phone: 603.334.2100 U.S. toll-free: 877.468.1448 Web site: http://www.aprisma.com How To Manage Your Network with SPECTRUM Page 3 Contents Introduction 9 SPECTRUM Core and Network Management ...........................................................10 Additional SPECTRUM Network Management ..........................................................10 Your Network Model 11 Before You Begin .......................................................................................................12 Optimizing Your Network Model .................................................................................13 Understanding Your Network Model .......................................................................13 Port Connection Representation .........................................................................15 Completing Your Network Model ............................................................................17 Generating an Inventory Report of Your Modeled Devices .................................18 Comparing the Inventory Report to Your Inventory List ......................................23 Modeling Devices Manually ................................................................................24 Customizing Interface Model Names ..................................................................25 Example: Configuring Interface Model Name Suffix ........................................26 Resolving Port Connections ................................................................................27 The Device Topology View ..............................................................................27 Using the Device Topology View to Identify and Resolve Port Connections ...29 Rearranging Your Network Model ..........................................................................31 Adjusting Management Capacity ............................................................................35 Staggering Polling Intervals ................................................................................35 How To Change the Polling Interval ................................................................36 Disabling Polling for a Model or a Model Type ....................................................38 How To Change the Polling Status ..................................................................38 Polling Devices that are Down ............................................................................39 Customizing Your Network Model ..............................................................................40 Using the Annotation Toolbox ................................................................................41 Using the Change Background Option ...................................................................42 Changing Backgrounds ..........................................................................................42 Background Color ...............................................................................................43 Using Organizational Views ....................................................................................45 Creating a Network Model that Shows the Business Impact of Services on Administrative Structure ...............................................46 How To Manage Your Network with SPECTRUM Page 4 Creating a Network Model that Shows Administrative Responsibilities ..............48 Maintaining Your Network Model ...............................................................................51 Adding and Removing Device Models ....................................................................51 To add a device model to your network model: ..................................................51 To remove a device model from your network model: ........................................53 Saving and Restoring the Database ......................................................................53 Creating a Backup ...............................................................................................53 Restoring a Database .........................................................................................54 Scheduling Backups ............................................................................................54 Configuring for Fault Management 55 Setting Thresholds .....................................................................................................55 Alarm Thresholds ...................................................................................................56 Rollup Condition Thresholds and Significance Levels ............................................56 Configuring Traps .......................................................................................................58 Creating New Traps ................................................................................................58 Configuring Alarms .....................................................................................................59 Clearing Old Alarms Automatically .........................................................................59 Alarm Policies for Specific Interfaces .....................................................................60 Alarm Management on DLCI Ports .....................................................................61 False Management Lost or Contact Lost Alarms ...................................................62 Using SpectroWATCH ............................................................................................63 Configuring for Fault Isolation ....................................................................................65 Configuring General Fault Isolation Parameters ....................................................65 Configuring Management Neighbors ......................................................................69 Configuring Maintenance Mode ..............................................................................73 Maintenance Mode for Devices ...........................................................................73 Maintenance Mode for Ports ...............................................................................74 Configuring Cross-Landscape Fault Correlation ....................................................76 Cross-Landscape Fault Correlation Example ..................................................76 Configuring Port Fault Correlation ..........................................................................78 Port Fault Correlation Options .............................................................................79 Port Fault Correlation Criteria .............................................................................80 Port Fault Correlation Caveats ............................................................................80 Port Fault Correlation Examples .........................................................................81 Fault Scenario 1 ..............................................................................................81 Fault Scenario 2 ..............................................................................................84 How To Manage Your Network with SPECTRUM Page 5 Fault Scenario 3 ..............................................................................................87 Known Port Fault Correlation Anomalies ............................................................89 Configuring Port Status Monitoring .........................................................................90 Port Status Polling Criteria ..................................................................................90 Port Status Events and Alarms ...........................................................................92 Link Trap Handling ..............................................................................................93 Interface Trap Configuration View .......................................................................95 PollPortStatus Feature ........................................................................................96 Utilizing PollPortStatus to Watch a Connected Port’s Status ..........................96 Enabling Port Status Polling ............................................................................97 Wide Area Link Monitoring .....................................................................................98 Link Fault Disposition ..........................................................................................99 Wide Area Link Monitoring Scenarios ...............................................................100 Port Layer Alarm Suppression ..............................................................................101 Port Criticality .......................................................................................................101 Live Pipes .............................................................................................................102 Enabling or Disabling Live Pipes System-Wide ................................................103 Enabling or Disabling Live Pipes on Individual Links ........................................103 Receiving Port Alarms .......................................................................................104 Automatic Port Status Alarm Clearing ..................................................................106 Suggested Port Fault Settings for Optimal Fault Notification ...............................106 Fault Management View Settings ............................................................................107 Device Model Settings ......................................................................................107 Port Model Settings ...........................................................................................109 Configuring Fault Management for Pingables ..........................................................109 Connecting Two Pingable Models ........................................................................110 Connecting a Pingable Model to a Device’s Port Model .......................................110 Fault Management 111 Fault Detection .........................................................................................................114 Condition Colors ...................................................................................................114 Blue (Initial) .......................................................................................................115 Green (Normal) .................................................................................................115 Yellow (Minor Alarm) .........................................................................................115 Orange (Major Alarm) .......................................................................................116 Red (Lost Contact) ............................................................................................116 Gray (Unknown) ................................................................................................116 How To Manage Your Network with SPECTRUM Page 6 Rollup Condition Colors ........................................................................................116 How Is a Rollup Condition Detected? ...................................................................117 Audible Alarms .....................................................................................................119 Fault Isolation ...........................................................................................................120 Enterprise Alarm Manager ....................................................................................120 Accessing the Enterprise Alarm Manager .........................................................120 Using the Enterprise Alarm Manager ................................................................122 Navigate In and Point and Click ...........................................................................123 Chassis Device View ............................................................................................125 Accessing the Chassis Device View .................................................................126 Device Topology View ..........................................................................................127 Accessing the Device Topology View ...............................................................127 Using the Device Topology View ......................................................................127 Performance View ................................................................................................128 Accessing the Performance View .....................................................................129 Using the Performance View .............................................................................129 What is in the Performance View? ................................................................129 Pie Charts ......................................................................................................130 Troubleshooting with the Performance View .................................................132 Beyond the Performance View .............................................................................136 SPECTRUM Intelligence 137 Static Configuration of Device Models .....................................................................137 Dynamic Configuration of Device Models ................................................................137 The Pulled Board List ...........................................................................................138 Router Reconfiguration Events .............................................................................139 Condition vs. Rollup Condition .................................................................................140 Attributes Determining Condition and Rollup Condition .......................................142 Condition and Rollup Condition Sensitivity ...........................................................146 Rollup Condition Flow ...........................................................................................148 Example of Rollup Condition Propagation ............................................................149 Organization Models .............................................................................................153 Fault Isolation ...........................................................................................................153 How Model Category Affects Contact Status .......................................................154 Examples ..............................................................................................................160 Duplicate Addresses ................................................................................................165 Automatic Naming and Addressing in SPECTRUM .................................................167 How To Manage Your Network with SPECTRUM Page 7 Monitor Points ..........................................................................................................169 Monitor Point Statistics .........................................................................................170 Composite Monitor Point Statistics ...................................................................171 Discrete Monitor Point Statistics .......................................................................172 Monitor Point Calculations in SpectroWATCH ..................................................178 Detection of Firmware Problems ..............................................................................179 Device Threshold Events .........................................................................................179 Redundancy .............................................................................................................180 Redundant SpectroSERVERs (Fault Tolerance) ..................................................180 Redundant Paths ..................................................................................................180 Configuring Device Redundancy .......................................................................181 Editing the Redundancy Preferred Addresses List ........................................181 Alternate Path Repeater Management .....................................................................184 Device Lost and Found ............................................................................................185 Device Type Verification ..........................................................................................185 Device Type Mismatch .........................................................................................186 In-Band Configuration of Device Alarms ..................................................................187 Interface Intelligence ................................................................................................187 Interface Alarms ...................................................................................................189 Interface Events ....................................................................................................190 Index How To Manage Your Network with SPECTRUM 192 Page 8 Introduction SPECTRUM is a powerful and flexible network management software program that provides a network administrator with effective and reliable network management tools. Reliable network management has become a necessity to maintain a competitive edge in today’s market place. SPECTRUM’s diversity of features and functionality can seem intimidating to an administrator just beginning to use the product. Not only does the SPECTRUM core* product provide a wide variety of features and functionality, but an additional array of separately purchasable Aprisma or third-party add-on components offers the means for further customizing or extending SPECTRUM. Given these extensive management capabilities, the need for a book describing the use of these network management tools has become increasingly important. This document is an attempt to address the need for a “how to” approach in using SPECTRUM to effectively manage a network. The focus will be on the management features included with the core product and how to use them. * Core product refers to those components included in the basic SPECTRUM package. How To Manage Your Network with SPECTRUM Page 9 SPECTRUM Core and Network Management Using SPECTRUM to successfully manage your network involves reducing network downtime, improving network performance, and maintaining a clear and up-to-date model of your network. SPECTRUM functionality allows you to optimize your network model, supports proactive fault management and fault isolation, and provides network performance enhancement capabilities. The flexibility of SPECTRUM allows an administrator to: • Customize the network model to optimize ease-of-use, performance, and fault management. • Keep the network model up-to-date as devices or subnets are added or removed. • Identify and isolate network faults down to a port level. This document describes these management tasks and the management techniques used to achieve them. Additional SPECTRUM Network Management Ethernet IEEE 802.3 is the network management standard that is used in the SPECTRUM management examples described in this document. Additional separately purchasable SPECTRUM network management products provide management of other networking standards. Some of these products are: • Frame Relay Manager • ATM Circuit Manager • VLAN Fault Isolation • Non-Persistent Connections Manager • RMON Contact Aprisma for more information about these products. How To Manage Your Network with SPECTRUM Page 10 Your Network Model This section describes how to refine an autodiscovered network model to improve management capabilities. This process includes: • Optimizing your network model - Understanding the network model - Completing your network model - Rearranging your network model for optimal monitoring capabilities - Optimizing management capacity • Customizing your network model - Using annotations and backgrounds to locate and identify your network - Creating SPECTRUM views for locating or grouping devices and sub-nets based on your network needs • Maintaining your network model - Adding and/or changing devices in your network - Saving, restoring, and automatically scheduling database backups How To Manage Your Network with SPECTRUM Page 11 Before You Begin In order to properly model your network and facilitate the management techniques described in this document, it is important that the following prerequisites are met: • Your network design meets all established Ethernet specifications (as outlined by the IEEE 802.3 group). • You have an accurate network diagram mapping the physical placement of your devices, nodes, and cable. • You have an inventory list of all the devices in your network. (This network diagram and list will be used to verify that SPECTRUM’s Autodiscovery has modeled all the devices and properly resolved all port connections.) • Your network has been autodiscovered initially to allow SPECTRUM to identify and communicate with all the devices on your network. When AutoDiscovery is run, SPECTRUM may not be able to identify devices that are temporarily off the network or not allowing management communication. Completing your network model will require some subsequent manual modeling. How To Manage Your Network with SPECTRUM Page 12 Optimizing Your Network Model Once your network has been autodiscovered, you may want to perform the tasks described in this section to complete your network model as well as optimize management capabilities. Understanding Your Network Model SPECTRUM’s representation is based on logical relationships and rules and may not look exactly like your network diagram. AutoDiscovery uses address tables and ICMP pings to identify subnet address ranges and devices within those ranges. Once discovered, those devices and subnets are modeled by SPECTRUM. Figure 1 and Figure 2 illustrate the relationship between the actual devices, the network diagram, and SPECTRUM’s network model. These figures show how your network is represented by SPECTRUM. How To Manage Your Network with SPECTRUM Page 13 Figure 1: Your Actual Network WanLink Router IPClassBLAN EMM-E6 EMM-E6 LAN 132.177.117.0 Fanout A LAN 132.177.118.0 Your Actual Network How To Manage Your Network with SPECTRUM Page 14 Figure 2: SPECTRUM Representation of Your Network SpectroGRAPH: Topology: LAN SpectroGRAPH: Topology: Universe ew Tools File Help Bookmarks View Tools Bookmarks 132. 132.177.117.0 EMM-E6 LAN_802_3 132.177.118.0 VNM EMM-E6 132.177.119.0 LAN_802_3 LAN_802_3 SpectroGRAPH: Topology: 132.177.117.0 ew Tools Bookmarks SpectroGRAPH: Topology: 132.177.11 File View Tools Bookmarks Workstation Workstation Workstation Workstatio CSIRptr Bridge Workstation CSiRptr CSI Repeater CSIRepeater Workstation Works Fanout A Port Connection Representation SPECTRUM represents port level connections between devices in the Device Topology views. The Device Topology view shows you the devices that are connected to each port for the selected device. Figure 3 provides an example of a Device Topology view for the EMM-E6 device. How To Manage Your Network with SPECTRUM Page 15 Figure 3: Device Topology View for the EMME/EMM-E6 SpectroGRAPH: Device Topology: EMM_E6 File View Tools Bookmarks Help M I I M M I 1 M 0 F u n k n o F O R M I M E M M l E 6 BRtrCSIEMM-E6 LAN 802.3 LAN 802.3 LAN 802.3 Network A Network B Network C CSIRptr CSIRptr CSIRptr 1 FWD ETHERNET 2 FWD ETHERNET 3 FWD ETHERNET 0.0.2.1.0.5.1.A:BC 0.0.2.1.0.5.1.A:BC 0.0.2.1.0.5.1.A:BC 0 0 0 If SPECTRUM does not understand which port a device is connected to, the device model will appear in the Off-Page Reference panel. AutoDiscovery should correctly place all of these connections; if it does not, you will need to refer to Resolving Port Connections on Page 27 to How To Manage Your Network with SPECTRUM Page 16 resolve the connections and allow SPECTRUM to properly monitor the device. Completing Your Network Model When AutoDiscovery is initially run, SPECTRUM may not be able to identify devices that are temporarily off the network or not allowing management communication. Although you may have run AutoDiscovery twice during a 48-hour period as recommended, some devices may not have been identified by SPECTRUM. To identify and properly locate any undiscovered devices or unresolved port connections, you must analyze your network diagram. The process of completing your network is as follows: 1 Generate an Inventory Report to identify all devices that have been modeled with AutoDiscovery. 2 Compare the Inventory Report to your inventory list of network devices to determine if any devices were not modeled. 3 Manually model any devices that were not modeled through AutoDiscovery. 4 Use the Device Topology view to resolve any unresolved port connections. How To Manage Your Network with SPECTRUM Page 17 Generating an Inventory Report of Your Modeled Devices Devices that were not contacted during the Autodiscovery process need to be identified and discovered individually or within a range. To determine what devices have been modeled, generate an Inventory Report as follows: 1 Select Reports then Generate from the Tools menu. The SPECTRUM Report Generator view will appear Figure 4. Figure 4: SPECTRUM Report Generator View SPECTRUM Report Generator File Applications Help Report Type Alarm Inventory Statistical Landscapes... Event Up/Down Time Models banshee Event Filters Report Format... Output File... Date Range Day Post Generate Script Report Depth Site Name General Output Format Graphical Tabular Display (.GRF) ASCII (.TAB) Postscript (.ps) pace/Spectrum/report.config/default.rptrc GIF (.gif) pace/Spectrum/report.config/default.rptrc How To Manage Your Network with SPECTRUM Postscript (.ps) Page 18 2 In the Report Type panel of the Report Generator view, select Inventory. 3 Select ASCII (.TAB) under Tabular for the output format. 4 If applicable, deselect any Graphical selections. 5 To select the models to be included in the inventory list, click on the Models button to open the Model Selection dialog box. Landscapes... banshee Report Format... Output File... Models Types... Event Filters Date Range Day Post Generate Script Site Name How To Manage Your Network with SPECTRUM Report Depth General Page 19 The Model Selection dialog box appears Figure 5. Figure 5: Model Selection Dialog Box SRG: Model Types Scope: Under Landscape Under Model Landscapes catapult Model Types 2E253_49R 2E253_49R_Mdul 2E42_27 2E42_27_Module 2E43_47 2E43_47_Module 2E43_51 2E43_51_Module 2E48_27 2E48_27_Module 2E49_27 2E49_27_Module 2E50_28 2E50_28_Module Select All Case Sensitive Deselect All Search OK Cancel 6 For the purpose of identifying undiscovered devices, highlight your landscape in the left-hand panel, and click on Select All at the bottom of the right-hand panel to include all models within that landscape. Click OK. 7 Select Detailed from the Report Depth button in the Report Generator view. Output File... Date Range Day Post Generate Script Site Name Default Site How To Manage Your Network with SPECTRUM Report Depth Detailed Page 20 8 Click on the Report Format button. Landscapes... Models Types... banshee Report Format... Event Filters Output File... Date Range Day Post Generate Script Report Depth Site Name General The SRG:Report Format dialog box appears (Figure 6). Figure 6: Report Format Dialog Box Srg: Report Format Filter Spectrum/sg-support/csrib/*.rib* Directories Files .. 3comgenbdgapp 3omnetbld 3comnetbld2 3comsrcrteapp Atmifport Atm_network Selection Ace/spectrum/sg-support/csrib/* Ok Filter How To Manage Your Network with SPECTRUM Cancel Page 21 9 Select Invt_Det.rib file as the format. This report information block file, located in the <SPECTRUM Installation Directory>/SG Support/CsRib/Inventory directory, provides a standard format for a detailed inventory report. (You can customize these reports with the Format option. See the SPECTRUM Report Generator User Guide.) Click OK. 10 Click on the Output File button to select an output file directory (the default directory is reports.output). Enter your output file name (e.g., inventory) and click OK. Landscapes... Models Types... banshee Report Format... Event Filters Output File... Date Range Day Post Generate Script 11 Select Generate from the File menu of the Report Generator view. 12 The following dialog box will appear. Click OK. SRG: Information Report Generation Invoked OK 13 Another dialog box appears when the report process has completed successfully. Click OK. How To Manage Your Network with SPECTRUM Page 22 14 Print the file located in the <SPECTRUM Installation Directory>output.report/<your file name.TAB>. Figure 7 provides an example of an inventory report. Figure 7: Example Inventory Report SPECTRUM Network Inventory Report Date: LandscapeID: User Name: Site Name: Page: 1 06/10/96 8:52:00 0x400000 arlo_admin Headquarters Model Type Model Name IP Address BdgCSINB30 Bridge #1 132.177.2.28 GnSNMPDev tutor 132.177.1.7 GnSNMPDev Workstation #2 132.177.1.5 GnSNMPDev Workstation #3 132.177.1.6 GnSNMPDev Workstation #4 132.177.2.18 GnSNMPDev Workstation #6 132.177.2.24 GnSNMPDev Workstation #8 132.177.2.42 IPClassB Automapped LAN LAN Automapped LAN LAN_802_3 Automapped LAN Hub_CSI_IRM3 Hub #1 132.177.1.52 Hub_CSI_IRM3 Hub #2 132.177.2.14 Hub_CSI_MRXi Hub #3 132.177.2.36 Pingable PsPrinter #1 132.177.1.44 Rtr_Cisco Router #1 132.177.1.1 WS_SGI Workstation #5 132.177.2.23 WS_SGI Workstation #7 132.177.2.47 MAC Address Create Date 0.0.1D.3.4A.74 06/09/94 8.0.20.B.8B.56 06/09/94 8.0.20.45.89.4D 06/09/94 8.0.20.77.8.5 06/09/94 8.0.20.0.43.AC 06/09/94 8.0.20.CC.AA.BB 06/09/94 8.0.20.12.B2.32 06/09/94 06/09/94 06/09/94 06/09/94 0.0.1D.1B.3.6 06/09/94 0.0.1D.3.34.5A 06/09/94 0.0.1D.B.8B.56 06/09/94 0 06/09/94 0.0.C.80.1.26 06/09/94 8.0.69.B.8B.56 06/09/94 8.0.69.B.8B.56 06/09/94 Contact ESTB ESTB ESTB ESTB ESTB LOST ESTB ESTB ESTB ESTB ESTB ESTB ESTB ESTB ESTB ESTB ESTB Last Contacted Vendor Object ID 06/10/94 08:51 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 Undefined Undefined Undefined 08/20/93 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:51 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 06/10/94 08:50 1.3.6.1.4.1.34 Comparing the Inventory Report to Your Inventory List The Inventory Report lists all models that have been discovered. This includes LANs, applications, and device models. This list will be used in conjunction with your inventory list to determine if any devices within your network were not discovered. The applications that are listed in the report are not represented in a Topology view. For the purpose of completing your network, the applications are not needed for the comparison process. However, you may need the inventory of application models if you wish to ensure they will be polled by SPECTRUM (see the Note on Page 37). Note the IP address or range of any missing device(s). This information will be used in the following section to discover additional devices if necessary. How To Manage Your Network with SPECTRUM Page 23 If the initial Inventory Report contains LANs or Fanout models, you may have to run inventory reports for each LAN or fanout to determine which models are contained within them. The inventory report provides the information shown in Figure 8. Using the Inventory Report Figure 8: The IP address for that device or LAN. The unique name of the device or LAN. SPECTRUM Network Inventory Report Date: LandscapeID: User Name: Site Name: Page: 1 06/10/96 8:52:00 0x400000 arlo_admin Headquarters Model Type Model Name BdgCSINB30 GnSNMPDev GnSNMPDev GnSNMPDev GnSNMPDev GnSNMPDev GnSNMPDev Bridge #1 tutor Workstation #2 Workstation #3 Workstation #4 Workstation #6 Workstation #8 IP Address 132.177.2.28 132.177.1.7 132.177.1.5 132.177.1.6 132.177.2.18 132.177.2.24 132.177.2.42 MAC Address Create Date Contact 0.0.1D.3.4A.74 8.0.20.B.8B.56 8.0.20.45.89.4D 8.0.20.77.8.5 8.0.20.0.43.AC 8.0.20.CC.AA.BB 8.0.20.12.B2.32 06/09/94 06/09/94 06/09/94 06/09/94 06/09/94 06/09/94 06/09/94 ESTB ESTB ESTB ESTB ESTB LOST ESTB Last Contacted Vendor Object ID 06/10/94 08:51 06/10/94 08:50 06/10/94 08:50 06/10/94 08:50 06/10/94 08:50 06/10/94 08:50 06/10/94 08:50 1.3.6.1.4.1.34 1.3.6.1.4.1.34 1.3.6.1.4.1.34 1.3.6.1.4.1.34 1.3.6.1.4.1.34 1.3.6.1.4.1.34 1.3.6.1.4.1.34 The Model Type Name for that device or LAN. Modeling Devices Manually If you have identified any devices that need to be modeled to complete your network model, they should be modeled manually. The following are examples of devices that require some manual modeling to complete your network: • MMAC-Plus and its associated modules • FDM Module • ForeRunner ATM Switch Module Refer to the specific management module guide for modeling instructions. How To Manage Your Network with SPECTRUM Page 24 Customizing Interface Model Names Note: Note: All changes to the interface naming suffix configuration are done at the device model level rather than at the interface model level. By default, when an interface is modeled, its model name is created using the device model name with the interface’s value for ifIndex as a suffix. For example, a device model named 192.168.9.17 might contain interface models named 192.168.9.17_1, 192.168.9.17_2, and so on. Using the Interface Model Name Suffix option available in the Device tab of SPECTRUM’s Global Attribute Editor for a given device model, you can select the attribute value to be used as an interface model name suffix. The following attributes can be chosen as suffixes: ifIndex, ifDescr, ifAlias, or ifName. These alternative interface model naming schemes can provide valuable information about a given interface. For example, some devices store board and port information in the value of ifName, which can be useful to have on hand when viewing an interface alarm in an alarm console. Configuration of the Interface Model Name Suffix option can be done either on a device model by device model basis, or by device model type. For example, you may want to set all Enterasys device model types to use ifDescr for an interface naming suffix, but for Rtr_Cisco device model types, you may want to set one subset of models to use ifName and another subset to use ifAlias (see Example: Configuring Interface Model Name Suffix (Page 26)). This will depend on what the particular device settings for these attributes are in your network environment. In Search Manager, the interface name suffix can be set for a device model or models by selecting the model(s) from Search Manager search results and choosing Management > Set Attribute Values... and setting the Interface Model Name Suffix option on the Device tab. How To Manage Your Network with SPECTRUM Page 25 Note: Note: Selecting Management > Rename Interface Models in Search Manager recalculates interface model names. This only needs to be used to update SPECTRUM when new values for interface suffix attributes have been set on the device. For more information, see Example: Configuring Interface Model Name Suffix and the Search Manager User’s Guide (2383). Example: Configuring Interface Model Name Suffix Follow these steps to configure the Interface Model Name Suffix option for two subsets of Rtr_Cisco device models setting one subset to ifName and the other subset to ifAlias: 1 In Search Manager, search for and select the subset of Rtr_Cisco device models you want to set to have interface models named with a suffix of ifName. 2 Select Management > Set Attribute Values... 3 Select the Device tab in the Global Attribute Editor and set the Interface Model Name Suffix option to ifName. 4 Click Apply to name this subset with a suffix of ifName. 5 Click Close in the Operation Results dialog box and the Global Attribute Editor window. 6 In Search Manager, search for and select the subset of Rtr_Cisco device models you want to set to have interface models named with a suffix of ifAlias. 7 Select Management > Set Attribute Values... 8 Select the Device tab in the Global Attribute Editor and set the Interface Model Name Suffix option to ifAlias. 9 Click Apply to name this subset with a suffix of ifAlias. 10 Click Close in the Operation Results dialog box and the Global Attribute Editor window. How To Manage Your Network with SPECTRUM Page 26 Resolving Port Connections AutoDiscovery automatically identifies and resolves port connections between devices. It may be necessary to resolve some port connections manually to allow SPECTRUM to properly manage all devices on your network. To resolve port connections the following must be done: • Systematically go through all the Device Topology views to determine if you have any unresolved port connections. • Resolve each unresolved port connection. The Device Topology View The Device Topology view represents a device in terms of its ports and port connections. You can use the Device Topology view to: • Examine existing connections to a device. • Make new connections to other devices. • Navigate to other views via the icons displayed in the Device Topology view. • Access other views (Performance, Application, etc.) through the Device Icon panel and the menu. • Select which board’s ports are shown in the Connections Panel for a multi-slot device, such as a hub (Figure 9). • Resolve any undefined device connections represented by Off-Page Reference icons shown in the Unresolved Off-Page Reference Icon Panel. • Add, copy, erase, cut, paste, and destroy icons. You can access the Device Topology view using one of the following methods: • Highlight the icon and select DevTop from the Icon Subviews menu. • Double-click on the Device Topology down arrow on the icon. • Place the mouse pointer over the icon, click and hold the right mouse button and select DevTop from the pop-up Icon Subviews menu. How To Manage Your Network with SPECTRUM Page 27 Figure 9 provides an example of a Device Topology view. Figure 9: Example Device Topology View SpectroGRAPH: Device Topolog File View Tools Bookmarks Unresolved Off-Page Reference Icon Network Icon that contains the device connected to this port Icons for models that are connected to a specific port LAN 802.3 LAN 802.3 Network A Network B Port Connections CSIRptr Interface Icons CSIRptr 1 FWD ETHERNET 2 FWD ETHERNET 0.0.2.1.0.5.1.A:BC 0.0.2.1.0.5.1.A:BC 0 How To Manage Your Network with SPECTRUM 0 Page 28 Using the Device Topology View to Identify and Resolve Port Connections The Device Topology view indicates unresolved port connections between devices as Off-Page Reference icons in the Off-Page Reference Panel. Resolve the port connections as follows: 1 Open the Device Topology view for the device to check for Off-Page Reference icons in the Off-Page Reference Panel. If no Off-Page Reference icons appear, continue to the next device. 2 If an Off-Page Reference icon appears in the Device Topology view, use your network map to verify the physical connections between devices to ensure proper port connections. 3 Select Edit from the File menu in the Device Topology view. 4 Highlight the Off-Page Reference icon (Figure 10) and drag it to the proper port. You may have to scroll the Port Connections Panel to view all the port connections. Release the mouse button when the icon is placed over the correct port. How To Manage Your Network with SPECTRUM Page 29 Figure 10: Resolving a Port Connection SpectroGRAPH : Device Topology : 111.222.333.4 File View Tools Help Bookmarks File > Edit 111.222.333.4 Highlight C A Y M A N C R M 2 R E I M I M R 1 0 B T I M I M C M I M E M M - E - EMM-E6 Drag Release A ON ETHERNET B OFF ETHERNET C OFF ETHERNET D ETHE 0:0:1D:19:AC:52 0:0:1D:19:AC:53 0:0:1D:19:AC:54 0:0:1D:1 0:0:0: 111.222.333.4 0:0:0:0 0:0:0:0 5 Repeat step 4 for each Off-Page Reference icon in this view. 6 Select Close Edit from the File menu to exit the Edit mode. 7 Repeat steps 2 through 6 for each device with unresolved port connections. How To Manage Your Network with SPECTRUM Page 30 Rearranging Your Network Model SPECTRUM flexibility allows you to group devices at any level within a conceptual model to reduce the complexity of your topology views. This is done by creating conceptual models, such as LAN, Fanout, Network, FDDI, etc., and placing the groups of modeled devices within them. SPECTRUM intelligence functions, such as rollup conditions, will still apply and enable you to effectively manage those devices. In our example, a LAN_802_3 Topology view displays a hub and its connected workstations as shown in Figure 11. In this example, we create a Fanout icon to represent the devices at the LAN_802_3 Topology level and cut and paste the workstation icons into the Fanout model. The procedure below can be used for all conceptual models. Figure 11: Example Network Topology View SpectroGRAPH : Primary Landscape File View Tools Bookmarks Help Router #1 Hub # 2 Hub_CSI_IRM3 1 Review your network and identify groups of devices that could logically be placed in a group model. Note the Hub and connected workstations in this example. 2 Select Edit from the File menu of the LAN_802_3 Topology view. How To Manage Your Network with SPECTRUM Page 31 3 Select New Model... from the Edit menu. 4 Select Fanout from the Select Model Type dialog box and click OK. 5 Enter the name of the Fanout (TechPubs in Figure 12) and click OK in the Creation dialog box. 6 Select Close Edit from the File menu to exit Edit mode. 7 Open the Fanout Cablewalk view by double-clicking on the down arrow shown in Figure 12. Figure 12: Creating a Fanout Model SpectroGRAPH : Primary Landscape File View Tools Bookmarks Help Router #1 Hub # 2 Hub_CSI_IRM3 TechPubs Down arrow 8 Select Edit from the File menu for the Fanout Cablewalk view (Figure 13) and for the LAN_802_3 Topology view (Figure 11). How To Manage Your Network with SPECTRUM Page 32 Figure 13: Fanout Cablewalk SpectroGRAPH : Cable Walk : Fanout File View Tools Bookmarks Help TechPubs 9 In the LAN_802_3 Topology view, hold down the shift key and use the mouse pointer to highlight each workstation icon connected to the hub (Figure 14). Figure 14: Selecting the Models to be Grouped SpectroGRAPH : Primary Landscape File View Tools Bookmarks Help Router #1 Hub # 2 Hub_CSI_IRM3 TechPubs How To Manage Your Network with SPECTRUM Page 33 10 Select Cut from the Edit menu in the LAN_802_3 Topology view. 11 Click OK in the Confirm dialog box. 12 Select Paste from the Edit menu in the Fanout Cablewalk view. 13 Select Close Edit from the File menu in the Fanout Cablewalk view and LAN_802_3 Topology views. These two views will appear as shown in Figure 15. Figure 15: SpectroGRAPH : Cable Walk : Fanout File View Tools Bookmarks SpectroGRAPH : File View Tools New Grouped Views Bookmarks Help Help TechPubs Router #1 Hub # 2 TechPubs Hub_CSI_IRM3 How To Manage Your Network with SPECTRUM Page 34 Adjusting Management Capacity SPECTRUM polls devices at regular intervals to access the management information that SPECTRUM’s intelligence uses to perform fault isolation. However, this polling activity consumes considerable bandwidth and host system resources. Therefore, a key part of optimizing your network model is to selectively adjust this trade-off between bandwidth use and management capacity so that resources are deployed where they are needed most. SPECTRUM provides two main ways of doing this: • Staggering Polling Intervals • Disabling Polling for a Model or a Model Type (Page 38) Staggering Polling Intervals Most SPECTRUM device models are polled every 60 seconds by default. You can change the default polling interval for an individual device model or for all models of particular model type. But keep in mind the abovementioned trade-off between bandwidth use and management capacity, which applies as follows: • If you increase the time between polls, you will use less bandwidth for management traffic, but you will receive device status updates less frequently. • If you decrease the time between polls, you will have more frequent updates of device status. However, you will use more bandwidth for management traffic. One way to achieve both goals (i.e., reduce network management traffic and enhance fault management) is to stagger polling intervals as shown in Figure 16. If all the devices in this example were using the default polling interval of 60 seconds, they would all be using bandwidth every 60 seconds. However, by leaving the polling interval for the router at the default of 60 seconds and changing the polling interval of all the other devices to 600 seconds, the SpectroSERVER resource utilization is reduced. Moreover, you will not lose management capabilities because if anything happens to the devices downstream from the router, polling will be interrupted and an alarm will be generated. How To Manage Your Network with SPECTRUM Page 35 Figure 16: Staggering Polling Intervals Polling Interval=60 WanLink Router Polling Intervals=600 Polling Interval=600 Polling Interval=600 How To Change the Polling Interval You can change the polling interval for any device by entering a new value in the Poll Interval field in the device model’s Model Information view (be sure to select File > Save All Changes after you enter the new value), or you can use a Command Line Interface (CLI) script called update_mtype to change the polling interval either for a specific model or for all models of a specific model type. The script requires you to supply three arguments: the model type name, the polling interval attribute ID (which is 0x10071), and the polling interval value in seconds. To run this script, do the following: 1 Navigate to the following location in your top-level SPECTRUM directory: /vnmsh/sample_scripts 2 Enter the following command: ./update_mtype <model type name> 0x10071 <new polling interval in seconds> How To Manage Your Network with SPECTRUM Page 36 (For example, to set the polling interval to 600 seconds for one or more models of type Host_Sun, you would enter: ./update_mtype Host_Sun 0x10071 600). This will display a list of all models of the specified type, showing for each one the model handle, the model name, the model type handle, and the model type name, as in the following example: 3 0x100001 workstation1 0x60000 Host_Sun 0x100002 workstation2 0x60000 Host_Sun 0x100003 workstation3 0x60000 Host_Sun To apply the new polling interval to a specific model in the list, enter the model handle (e.g., 0x100001) or the model name (e.g., workstation1). To apply the new polling interval to all models of the specified type, enter the model type handle (e.g., 0x60000) or the model type name (e.g., Host_Sun). 4 The system will indicate success or failure and disconnect you from CLI. Note: Note: Polling intervals also apply to application models, many of which have an initial setting of zero, which in effect disables polling (although the preferred method for disabling polling for any model is to set the Polling Status attribute to “False” as explained in the following section: Disabling Polling for a Model or a Model Type). You can reset the polling interval for application models using the same methods described above for device models. Indeed, it is recommended that you check your inventory report (see Page 18) for application models and run the update_mtype script on each global application model type to set a polling interval of 60 seconds. How To Manage Your Network with SPECTRUM Page 37 Disabling Polling for a Model or a Model Type In addition to the strategy of increasing default polling intervals for selected models to conserve bandwidth, you may decide that the status of certain devices is not worth the bandwidth that polling them requires, even at longer intervals. For example, some network administrators may choose not to model endpoint devices such as workstations so that they do not have to deal with the alarms that occur each time these devices are powered down. If you wish to have the endpoints modeled, but do not wish to expend bandwidth with network polling traffic, you can disable polling to these models (or to any models) by changing the value of the Polling Status attribute to “False” as described below. How To Change the Polling Status You can change the polling status for any model by toggling the Polling Status selector button in the model’s Model Information view (be sure to select File > Save All Changes after you change the setting), or you can use the Command Line Interface (CLI) script called update_mtype to change the polling status either for a specific model or for all models of a specific model type. The script requires you to supply three arguments: the model type name, the polling status attribute ID (which is 0x1154f), and the polling status value (“true” or “false”). To run this script, do the following: 1 Navigate to the following location in your top-level SPECTRUM directory: /vnmsh/sample_scripts 2 Enter the following command: update_mtype <model type name> 0x1154f <new polling status value> (For example, to disable polling for one or more models of type Host_Sun, you would enter: update_mtype Host_Sun 0x1154f false). This will display a list of all models of the specified type, showing for How To Manage Your Network with SPECTRUM Page 38 each one the model handle, the model name, the model type handle, and the model type name, as in the following example: 3 0x100001 workstation1 0x60000 Host_Sun 0x100002 workstation2 0x60000 Host_Sun 0x100003 workstation3 0x60000 Host_Sun To apply the new polling status setting to a specific model in the list, enter the model handle (e.g., 0x100001) or the model name (e.g., workstation1). To apply the new polling interval to all models of the specified type, enter the model type handle (e.g., 0x60000) or the model type name (e.g., Host_Sun). 4 The system will indicate success or failure and disconnect you from CLI. Note: Note: The Polling Status value takes precedence over the Polling Interval value in terms of enabling/disabling all kinds of periodic external requests to a model. Even though setting the Polling Interval to zero will automatically change the Polling Status to “False,” if you then reset the Polling Status to “True,” requests generated by SPECTRUM inference handlers could still occur, regardless of the fact that the Polling Interval is still set to zero. However, to enable normal SPECTRUM polling for fault isolation purposes, the Polling Interval would have to be manually reset to a non-zero value. Polling Devices that are Down When contact with a device has been lost, SPECTRUM will continue to poll the device to see if the contact status has changed. By default, SPECTRUM will poll devices that are down once every third polling interval. For example, if the device’s polling interval is set to 60 seconds, when the device is down, SPECTRUM will poll the device once every 180 seconds. How To Manage Your Network with SPECTRUM Page 39 As of SPECTRUM version 6.6 SP5, you can change the interval at which SPECTRUM will poll a device that is down. To do this, insert the following syntax into the <$SPECROOT>/SS/.vnmrc file: down_device_poll_interval_multiplier=<user_defined_multiplie r> For example, if the <user_defined_multiplier>=2 and the device’s polling interval is 60 seconds, then the down device will be polled once every 120 seconds (2*60=120). Customizing Your Network Model SPECTRUM functionality provides such features as the Annotation Toolbox, the Change Background option, and Organizational and Location Views to customize your network model to enhance management capabilities. • Annotation Toolbox - provides a set of graphic tools that allow you to annotate your SPECTRUM views such as: - Labeling - Drawing graphical images - Changing background colors • Change Background option - allows you to select an image to be used as the background for the entire view such as: - World map - Aerial photographs - Floor plans • Organizational and Location Views - allow you to copy device models into views that are created to group models based on location or administrative responsibility of devices for ease of management. How To Manage Your Network with SPECTRUM Page 40 Using the Annotation Toolbox The Annotation toolbox gives you access to the color palette and drawing tools (line, circle, box, and text). There are also a variety of options for changing fonts, line width, and colors. The Annotation Options window is described in more detail in Annotation Toolbox. Figure 17 provides an example of how you might annotate a Topology view to help understand SPECTRUM’s representation of your network. Figure 17: Annotation of a Topology View SpectroGRAPH: Topology File View Tools 132.177.117.0 Bookmarks Wiring Closet 45 - 2nd Help 132.177.118.0 EMM-E6 LAN_802_3 LAN_802_3 Technical Documentation Building 45 2nd Manufacturing Building 45 1st 132.177.118.0 LAN_802_3 How To Manage Your Network with SPECTRUM Page 41 Using the Change Background Option This option allows you to change the background color and raster image, and to set the maximum size of the window for the current GIB View. The Background Color label button accesses the Color Selection palette, from which you can select a color and corresponding color number. The Background Raster button allows you to open the Select File dialog box and choose a background raster image file. The image you select becomes the background for the view. The window size will change to accommodate the size of the raster image you select. Changing Backgrounds Use this menu option to change the dimensions and background color and/or the background raster image for a GIB View. Select the Change Background… option from the Edit menu. The Change Background dialog box appears as in Figure 18. Figure 18: Change Background Dialog Box Change Background 220 Background Min Width = 1507 Min Height = 707 Height Width = 1544 Screen Width = 1258 757 Screen Height = 985 Background Raster OK Cancel How To Manage Your Network with SPECTRUM Help Page 42 Background Color To change the Background Color: 1 Enter the desired window dimensions, in pixels, into the Height and Width fields of the Change Background dialog box. These entries define the maximum window size; you cannot resize the window beyond the selected size dimensions. 2 If you know the index number for the desired background color from the color palette, you can enter the index number directly into the Background Color field. Otherwise, click on the Background Color button to open the Select Color Index dialog box (Figure 19). SPECTRUM displays color index numbers 77 through 255 within the individual color blocks of the SPECTRUM Color Palette. (Only these colors are used by SPECTRUM). Figure 19: Select Color Index Dialog Box 77 78 91 92 Select Color Index SPECTRUM Color Palette with Associated Index Numbers Click the left mouse button on the desired color and click OK to apply a color to the view. or Double-click on any color in the palette to select that color and close the palette dialog box. Selected Color: How To Manage Your Network with SPECTRUM OK 247 Cancel Page 43 To change the background graphics, click the Background Raster button on the Change Background dialog box. Raster is a term used to describe the background graphics image in a view. The Select File dialog box lists image files available in the CsImage/Background directory. Some of these are solid color backgrounds and others are graphic images. The file select dialog box permits selecting image files written in Aprisma graphics file format (.csi) or Tagged Image File Format (TIFF). SPECTRUM supports the most commonly used (non-compressed and PackBitscompressed) TIFF file formats. Image files used as a background image must be placed in the SPECTRUM Installation Directory/SG-Support/ CsImage/Background directory. Note: Note: The maximum size of the view is the size of the background raster image. The window view may not be enlarged to be larger than the size of the raster image selected. To change the background to a raster image from the Change Background dialog box: 1 Select the Background Raster button. The Select File dialog box appears as in Figure 20. Figure 20: Select File Dialog Box Select File Filter /space/Spectrum/SG-Support/CsImage/Background/* Directories /Spectrum/SG-Support/CsImage/Background/. /Spectrum/SG-Support/CsImage/Background/. /Spectrum/SG-Support/CsImage/Background/. Files Bd_Genbd1.csi Bd_Genbd2.csi Bd_Genbd3.csi Bk_DkBlue.csi Bk_Oat.csi Bk_LtOat.csi Bk_MdBlue.csi Default.csi Selection /space/Spectrum/SG-Support/CsImage/Background/ OK Filter How To Manage Your Network with SPECTRUM Cancel Help Page 44 2 Select the directory for the desired graphic file in the left-hand panel. Apply a filter to file selections if necessary. 3 Select an image file by clicking on the file name in the right-hand panel. 4 Click on OK to confirm your selection, or Cancel to leave the background unchanged. 5 Click on OK in the Change Background dialog box, or Cancel to leave the background unchanged. If these are the only changes being made to the view at this time, use the Close Edit option in the File menu to exit edit mode. Using Organizational Views The Organizational view (aka Org-Chart/Services view) allows you to group subnets and device models based on organizational considerations such as: • Corporate structure; for example, the finance department network device models are grouped in their own organizational view. • Services; for example, a set of devices are essential for supporting the Email service which provides Email for several different departments. • Administrative responsibilities; for example, an operator has been assigned responsibility for all routers in your network. A variety of Org-Chart/Services views are available to help you manage your network based on your specific needs and network design. The first example below shows how to use organizational views to track the business impact of network problems by modeling how corporate services are linked to corporate structure. The second example shows how to use organizational views to track network problems by administrative responsibility. How To Manage Your Network with SPECTRUM Page 45 Creating a Network Model that Shows the Business Impact of Services on Administrative Structure The procedure described below is an example of how to create an OrgChart/Services view for the management of a service that one or more administrative groups depend on. 1 From the View menu of any Topology view, select New View > OrgChart/Services. 2 Select Edit from the File menu in the Org-Chart/Services view. 3 Select New Model from the Edit menu. 4 Select Department from the New Model dialog box and click OK. 5 Enter a unique name to best represent your Department model. In this example, the model name is Engineering because it will contain the devices and services that the Engineering department depends on. The Department model icon will appear in the Org-Chart/Services view. 6 Select Close Edit from the File menu. 7 Double-click on the view button on the Engineering Department model to navigate into the Engineering Department container. 8 Select Edit from the File menu. 9 Select New Model from the Edit menu. 10 Select Service_Owns from the New Model dialog box and click OK. 11 Enter a unique name to best describe the service you want to represent. In this example, the model name is Email because it will How To Manage Your Network with SPECTRUM Page 46 contain the devices that make up the Email service. The Service_Owns model icon will appear. 12 Select Close Edit from the File menu. 13 Double-click on the view button on the Email Service_Owns model to navigate into the Email Service_Owns container. 14 Select Edit from the File menu. 15 You will need to place all device models that represent components of the Email service into this container. If these devices are already modeled in another view (or views), go to that view and use the Copy command to copy those models. Return to this view and paste the models into the container. If these devices have not yet been modeled, you can create models for them using the Edit > New Model command (see Adding and Removing Device Models (Page 51) for further instructions). 16 Select Close Edit from the File menu. 17 You have now created a model to manage the Email service and have shown that the Engineering department depends on this service. Other Department container models can be created to represent additional departments. (Enterprise, Division, Landscape, or Subsidiary container model types can also be used.) The Email Service_Owns container can be copied and pasted into any of those Department containers that represent departments which depend on the Email service. The condition of a service is displayed on the Service_Owns container model. For example, if two routers contained in the Email Service_Owns container have gone down and a red alarm is generated on each of them, How To Manage Your Network with SPECTRUM Page 47 the Email Service_Owns container model will show a red condition. If you open the Alarm Manager from the Service_Owns container model, SPECTRUM shows all alarms for devices contained within the Service_Owns container. In addition, the rollup condition (see Rollup Condition Colors (Page 116)) of the department is displayed on the Department container model. Thus, if network problems exist which affect services to a department, this will be reflected in the rollup condition shown on the Department container model. Once you have created a network model that represents your enterprise’s departments and services, the business impact of network problems can be easily determined. Network operators can correlate an alarm on a single device to the service(s) this device is critical to, and the departments that depend on that service. Creating a Network Model that Shows Administrative Responsibilities The procedure described below is an example of how to create an OrgChart/Services view for the management of routers that a given administrator is responsible for. 1 From the View menu of any Topology view, select New View > OrgChart/Services. 2 Select Edit from the File menu in the Org-Chart/Services view. 3 Select New Model from the Edit menu. 4 Select Org_Owns from the New Model dialog box and click OK. 5 Enter a unique name to best represent your Org_Owns model. In this example, the model name is ITClevelandRouters because it will How To Manage Your Network with SPECTRUM Page 48 contain all of the routers that the IT department in Cleveland is responsible for. The Org_Owns model icon will appear in the OrgChart/Services view. 6 Select Close Edit from the File menu. 7 Double-click on the Org_Owns view button to open the view. SpectroGRAPH: Org-Chart/Services: TopOrg File View Tools Bookmarks Help ITClevelandRouters Org_Owns 0 8 Select Edit from the File menu in the Org_Owns view. 9 Copy all routers that the IT department in Cleveland is responsible for into this view as follows: a Open the applicable Topology view and select Edit from the File menu. b Highlight all routers in the topology view to be copied into the Org_Owns view. c Select Copy from the Edit menu in the applicable Topology view. Click OK in the Confirm dialog box. How To Manage Your Network with SPECTRUM Page 49 d Select Paste from the File menu in the Org_Owns view. All routers copied will appear in the view. e You can arrange routers in the view as shown below. SpectroGRAPH: OWN: Routers File View Router 1 CiscoRtr Building 35 f Tools Bookmarks Router 2 CiscoRtr Building 37 Help Router 3 CiscoRtr Building 38 Router 4 CiscoRtr Manufacturing Select Close Edit from the Topology views and the Org_Owns view to return to view mode. 10 Using the Annotation Tool Box, label each router as to its location or significance. This will allow identification of the router for manageability. How To Manage Your Network with SPECTRUM Page 50 Maintaining Your Network Model Once you have completed optimizing your network model, you will need to make sure it accurately reflects the current configuration as new devices are added or existing ones taken out of service. You will also want to be able to create a backup copy of your database so that you can restore it to a known previous condition in the event of data corruption or equipment failure. These issues are discussed in the following two sections: • Adding and Removing Device Models • Saving and Restoring the Database (Page 53) Adding and Removing Device Models Whenever you add or remove a device on your network, you will need to alter your network model to keep it up-to-date. You must be sure to destroy the model of the device being removed or the model will be put into Lost and Found. To add a device model to your network model: 1 Navigate to the highest level topology view that should contain this device model. 2 Select File > Edit to put the view in the edit mode. 3 Select Edit > New Model (if you know the SPECTRUM model type you want to use) or New Model By IP (if you know the device’s IP address but want SPECTRUM to automatically select the appropriate model type). 4 If you are using the New Model option, in the Select Model Type dialog box, select a model type and click OK; then enter a name and address in the Creation View dialog box. If you are using the New Model By IP option, in the Create Model By IP Address dialog box enter the device’s IP address in the Network Address text box (see Note below). How To Manage Your Network with SPECTRUM Page 51 5 When you use either the New Model or the New Model By IP option, you can have SPECTRUM automatically discover and model network elements that are connected to the device you are modeling. To do this, click the Discover Connections check box available in the Creation view dialog box when you are using the New Model option, and in the Create Model By IP Address dialog box when you are using the New Model By IP option. SPECTRUM will use the settings in the AutoDiscovery Model Information view for discovering the connections. The IP Table (Always) option is checked automatically if you select Discover Connections. This option indicates that SPECTRUM will use the device’s IP Address Table for the discovery operation. You can also select the IP Route Table option. If selected, this option indicates that SPECTRUM will use the device’s IP Route Table for the discovery option. Whatever setting is chosen for the IP Route Table option in this dialog box will override whatever is set for the IP Route Table option in the AutoDiscovery Model Information view. Since the IP Route tables can be very large, the discovery may take longer with this option selected. However, this option will accommodate the modeling and mapping of unnumbered IP interfaces, which cannot be done using the IP Table option alone. 6 Click OK. 7 Select Close Edit from the File menu. Note: Note: You cannot use the New Model by IP option to create a model if a model with that IP address already exists in SPECTRUM. Attempts to do so will produce an error message that identifies the existing model by name and model type and asks you whether you would like to open its default view (usually its Device Topology view). However, if you know a modeled device’s IP address, this same functionality can be used as a shortcut for locating the model within the network map without having to open Search Manager. If you click OK in the error message, the model’s default view will be displayed, and you can then navigate from that view as necessary to determine the model’s location. How To Manage Your Network with SPECTRUM Page 52 To remove a device model from your network model: 1 Navigate to the highest level Topology view that contains the device model. 2 Select File > Edit to put the view in the edit mode. 3 Select the model to be removed by clicking on it to highlight it. 4 Select Edit > Destroy. 5 Select File > Close Edit to exit the edit mode and save your changes. Saving and Restoring the Database Once you have organized your SpectroGRAPH network model the way you want, you should save your database and schedule a regular database backup using the OnLine Backup feature, which is accessed from the SPECTRUM Control Panel. A backup database will enable you to quickly return to your optimized network model if something happens to corrupt your database. This can save hours of work and should not be overlooked. Creating a Backup To backup your database from the Control Panel: 1 In the Control Panel’s Database Administration block, click the Save button. If SpectroGRAPH is not running, a dialog box will ask you what host you want to save the database against. Select the default host to save the current database. The Online Database Backup Configuration View opens. 2 Enable compression of files to save space. 3 Assign a prefix for the backup file name if desired. The default is db_. 4 Assign a backup directory if one other than the default is desired. The default is <SPECTRUM Installation Directory>/SS-DB-Backup. 5 The minimum disk space required is shown. Check to make sure you have enough space to save the file. 6 Click the Begin Backup Now! button. How To Manage Your Network with SPECTRUM Page 53 Restoring a Database To restore your database from the Control Panel: 1 Before restoring a database, it is always a good idea to backup the current database (unless this is a corrupt file). Refer to the instructions above on how to backup a database. 2 Click on the Restore button. A dialog box will ask if you want to initialize the database. If you are recovering from a corrupt file, you should choose the initialize option to add a further cleaning with the restoration of the database. If you do not choose to initialize, it will simply reload the backup database file. In either event, a dialog box will prompt you for the directory and name of the database file to load. 3 Click on the directory and file name of the database file you wish to load. 4 Click OK. Scheduling Backups To Schedule Regular Database Backups From the Control Panel: 1 Click on the Save button. The Online Database Backup Configuration View opens. 2 In the bottom half of the view, there is a section on Automatic Backup Setup. Click on the Enable button to enable automatic backups. 3 Enter the backup interval in hours and minutes. 4 Check the next backup date and time to make sure it is correct. 5 Check the top section of the view for the backup directory name and whether compression is selected. Set these variables as desired. (refer to the section on how to backup a database). 6 Click the Begin Backup Now! button. How To Manage Your Network with SPECTRUM Page 54 Configuring for Fault Management SPECTRUM proactive fault management features provide you with the ability to set thresholds and configure traps, events and alarms to alert you before an actual fault occurs within your network. The following proactive fault management features are described in this section. • • • • • • Setting Thresholds Configuring Traps (Page 58) Configuring Alarms (Page 59) Configuring for Fault Isolation (Page 65) Fault Management View Settings (Page 107) Configuring Fault Management for Pingables (Page 109) Setting Thresholds SPECTRUM icons use color to indicate two types of status for the modeled entities they represent. All models have a condition color that reflects the current contact and/or alarm status of the model itself. Additionally, “container” models, such as Networks and LANs, have a rollup condition color that reflects the composite status of all the other models they contain (i.e., their “children”). In other words, the status of a modeled device “rolls up” to the “parent” container model. Both types of status rely on threshold values to determine when and how the associated color will change. Thresholds for condition and rollup condition are discussed separately in the following two sections. How To Manage Your Network with SPECTRUM Page 55 Alarm Thresholds Alarm thresholds apply to individual devices. Based on your experience with your network, you may want to set threshold critical values within some or all devices that will generate an alarm condition if these values are exceeded. For example, the alarm thresholds that can be set for Cabletron’s EMM-E6 are defined as follows: • Traffic - You can set a threshold value for the load per unit of time for the repeaters and ports managed by the EMM-E6. • Collisions- You can set a threshold value for the number of collisions per unit of time for the repeaters and ports managed by the EMM-E6. • Broadcasts- You can set a threshold value for the number of broadcasts received within a period of time by the EMM-E6. • Errors- You can set a threshold value for the number of errors per unit of time for the repeaters and ports managed by the EMM-E6. Alarm thresholds are generally set through one of the Configuration views available for the modeled device. For specific information, consult the documentation for the SPECTRUM management module used to model the device for which you wish to set alarm thresholds. Rollup Condition Thresholds and Significance Levels Rollup condition thresholds are set to display a rollup condition color based on the sum of the significance levels for the alarms generated by devices within the models that “Contain” them (Figure 1). If two devices have a combined significance level of 7, and the rollup condition threshold for the device that contains them is 3, then the containing model will be orange. Set these thresholds and significance levels in the Model Information view for each model in your network. The Significance Level attributes define the numeric value for Yellow, Orange, and Red Conditions and Rollup Conditions. Like Rollup Thresholds, Significance Level values are administrator-defined values entered on a model-bymodel basis. Significance Level field labels begin with the words “value How To Manage Your Network with SPECTRUM Page 56 when.” The default values for Significance Level are as follows: Value_When_Yellow=1 Value_When_Orange=3 Value_When_Red=7 Typically, models (devices) are divided into two classes, “significant” or “insignificant.” A significant device is any device that requires an administrator’s attention for proper network operation. Insignificant devices are typically endpoint devices, such as a PC or workstation. Insignificant devices usually toggle between Green and Blue (Condition Value = 0). Significant models can be made insignificant by changing their Value_When_Red attribute value to 0 (zero). Many users do not change the rollup thresholds from their default values because of the complexity of options available. Figure 21: Rollup Conditions Information Alarm Rollup Condition Color is Yellow Model Name 4 1 7 Landscape With a yellow rollup threshold set to 1 and the sum of the significance levels for the “Contained” models equal to 3, the rollup condition color will be yellow. Minor Alarm Rollup Condition Color is Orange LAN 802.3 Device Failure Device Condition Color is Red With an orange rollup threshold set to 3 and the sum of the significance levels for the “Contained” devices equal to 7, the rollup condition color will be orange. A noncritical device with a significance level of 7. HubCSIEMME How To Manage Your Network with SPECTRUM Page 57 Configuring Traps Traps are used to generate rollup conditions, alarms, and event messages. Some traps are automatically set by SPECTRUM. See the Event Configuration Files User Guide (5070) and Modeling with the GnSNMPDev Toolkit (1316) for information on defining how SNMP traps (unsolicited SNMP messages) are handled. You can specify how trap codes and their associated parameters are mapped into SPECTRUM events and alarms. SNMP traps are received from specific devices that support the Simple Network Management Protocol. Therefore all operations associated with editing SNMP trap information begin by selecting a specific kind of device (model type). Traps can be mapped to events. An event is an indication that something has occurred on your network and SPECTRUM has taken notice of it. Events can be configured to generate alarms and can be mapped to Alarm Probable Cause messages, which provide recommended actions regarding the alarm. Events are generated when a trap is received if that trap is mapped to an event. Note: Note: You can disable the generation of events and alarms based on traps received from a specific device by adjusting a setting in the Fault Management view for that device model (see Figure 23 on Page 70). Creating New Traps If a trap for a particular model type does not exist, you can create and map a new one. See the Event Configuration Files User Guide (5070) for more information. How To Manage Your Network with SPECTRUM Page 58 Configuring Alarms Alarms are generated by SPECTRUM to let you know when defined thresholds have been exceeded. Each device reports to SPECTRUM when an event occurs. Some events generate alarms automatically, and with some devices you can configure alarms by setting threshold values for traffic, collisions, broadcasts, and errors. Alarms that are generated automatically by SPECTRUM are listed in SPECTRUM’s SS/CsVendor directory for each device management module. Probable cause files for these alarms are listed in the SG-Support/CsPCause directory. The following sections describe how alarms can be managed to meet your specific needs. Clearing Old Alarms Automatically You can configure SPECTRUM to automatically clear any alarm which has existed for longer than a specified amount of time. It is also possible to configure SPECTRUM to clear all old alarms, or just old stale alarms. By automatically clearing the unwanted alarms, you will improve the performance of the SpectroSERVER. The VNM’s Alarm Management Information view contains two attributes that allow you to control the automatic clearing of old alarms. The AlarmAgeOutTime attribute contains the amount of time (in hours) that an alarm in the landscape can exist. Once an alarm has existed for longer than this amount of time, it will be removed. If the attribute contains a value of 0 (zero), this functionality will be disabled. The AgeOutStaleAlarmsOnly attribute contains a Boolean value that indicates whether only stale old alarms should be cleared. By default, it is set to TRUE. Every hour, SPECTRUM checks the status of all alarms in the landscape and clears alarms based on the settings of the above criteria. Therefore an alarm will not be removed at exactly the moment when its existence time has exceeded the time-out. It can be, at most, an hour “overdue.” How To Manage Your Network with SPECTRUM Page 59 Refer to the Alarm Management section of the Distributed SpectroSERVER (2770) guide for information on the VNM’s Alarm Management Information view and how to change the value of attributes in this view. Alarm Policies for Specific Interfaces It is possible to prevent SPECTRUM from generating alarms on specific types of interfaces. In SPECTRUM versions 6.6 SP4 and below, these “specific interfaces” must meet one of the following criteria: • IfType = ds0 (81) • IfType = ppp (23) AND (ifDescr = “Serial*:*” OR ifDescr = “Async*”) In SPECTRUM versions 6.6 SP5 and above, these “specific interfaces” must meet one of the following criteria: • IfType = ds0 (81) • IfType = ppp (23) AND (ifDescr = “Serial*:*” OR ifDescr = “Async*” OR ifDescr = “Virtual-Access*”) The AlarmOnSpecificInterfaces attribute is used to control whether alarms are generated on the specific interfaces defined by the criteria above. By default, it is set to TRUE. It can be set to FALSE using SPECTRUM’s CLI, using the Global Attribute Editor, or by adding the attribute to a GIB and changing the value. When a device’s AlarmOnSpecificInterfaces attribute is set to FALSE, SPECTRUM will find all of the interfaces that meet the above criteria on the device and set the AlarmOnLinkDownTrap (0x11fc2) attribute on each interface to Never. This prevents SPECTRUM from generating alarms based on Link Down Traps on these models. Each of these interfaces will also have their PollPortStatus (0x1280a) attribute set to FALSE, preventing port status polling and alarming. In addition, each of the interfaces will have their DisableTrapEvents (0x11cd0) attribute set to TRUE, which prevents the generation of events (and alarms) for traps that are received for that interface. How To Manage Your Network with SPECTRUM Page 60 When AlarmOnSpecificInterfaces is set to TRUE, SPECTRUM will use the AlarmOnLinkDownTrapIfTypes (0x1290f) attribute on each interface to determine how to set AlarmOnLinkDownTrap. Note: Note: If you change the value of AlarmOnSpecificInterfaces from FALSE to TRUE, the value of the AlarmOnLinkDownTrap, PollPortStatus, and DisableTrapEvents attributes will not be changed. These values must be changed manually. Alarm Management on DLCI Ports The AlarmOnInvalidDLCIs attribute exists for all device models and is used to control whether or not alarms are generated on invalid DLCI ports. The default value for this attribute is FALSE. It can be set to TRUE using SPECTRUM’s CLI, or by adding the attribute to a GIB and changing the value. When AlarmOnInvalidDLCIs is set to FALSE, all DLCI_Port models on the device whose circuit state is currently “invalid” will have their PollPortSatus attribute to FALSE. When AlarmOnInvalidDLCIs is set to TRUE, the DLCI_Ports will be left alone. The following bullets outline how SPECTRUM handles alarming on invalid DLCIs: • If the physical interface (i.e. lower layer interface) is down, an invalid DLCI will have a gray condition. • If the physical interface is up, and AlarmOnInvalidDLCIs is set to FALSE, a brown alarm will be generated on the invalid DLCI. • If AlarmOnInvalidDLCIs is set to TRUE, a red alarm will be generated on the invalid DLCI. The following caveats apply to this alarm management functionality: • The definition of a “specific interface” is not extendable or modifiable. Only interfaces which exactly match the above criteria will be considered “specific interfaces”. How To Manage Your Network with SPECTRUM Page 61 • If a pipe associated with a “specific interface” is made live, alarms will be generated. • If the PollPortStatus of a “specific interface” or DLCI had been set to TRUE, and was subsequently set to FALSE because alarm suppression was enabled for the device, it will not be reset back to TRUE when alarm suppression is disabled. It will remain FALSE. This is because it is not possible to determine the original value after a VNM restart. Manual intervention is required. • If a DLCI port receives a circuitStateChange trap and the new state is Invalid, an alarm will always be generated. There is no way to control this. • If SPECTRUM had previously set the above attributes correctly for either a “specific interface” or a DLCI, and an attribute’s value is subsequently modified, SPECTRUM will not be responsible for any associated behavior. False Management Lost or Contact Lost Alarms There is a SUN recommended security procedure which can increase the chance of a false Management Lost or Contact Lost alarm being generated. This security procedure modifies the arp timeout on a Solaris host. This change in the arp configuration will cause an increased delay in arp messages being sent by the system, which will increase the overall time it takes for a SNMP packet to be sent and replied to. Since the SpectroSERVER starts the timeout timer as soon as it hands the SNMP request off to the Operating System, the overall delay easily exceeds the settings in the SpectroSERVER. One way to identify this behavior is to invoke the following command from a terminal window on the Solaris host: arp -a | wc -l This command will count the number of entries in the ARP table. The count will increase and then it will suddenly decrease back to the initial value. The time it takes for the decrease to occur will depend on the current arp timeout setting. How To Manage Your Network with SPECTRUM Page 62 To work around this problem, you must return the arp timeout value to its default. To do this, follow the instructions below: 1 Open the following file: /etc/init.d/nddconfig 2 Look for an entry similar to the following: ndd -set /dev/ip ip_ire_arp_interval 600000 3 This entry modifies the default arp timeout value. Remove this entry. 4 Type the following command in order to update your system (you must run as root): Solaris 2.8 ndd -set /dev/ip ip_ire_arp_interval 1200000 Solaris version below 2.8 ndd -set /dev/ip ip_ire_flush_interval 1200000 5 Reboot your system to allow these changes to take effect. The arp timeout will return to the default value, which works correctly with SPECTRUM. Using SpectroWATCH You can use SpectroWATCH to monitor any attributes against thresholds you set to generate events and alarms. SpectroWATCH allows you to do the following: • Select the attribute(s) that are important to monitor for fault potential • Set the threshold value for that attribute(s) so that when the threshold is exceeded, an alarm is generated • Select the alarm severity for each threshold • Write the alarm message that will show up in the alarm view or alarm script that will be executed when the alarm is generated How To Manage Your Network with SPECTRUM Page 63 • Monitor either a single attribute or a complex formula of attributes, constant values, mathematical functions, and other existing formulas. For example: two threshold watches could be set on a device’s packet rate: one to generate a yellow alarm when the value exceeds 10,000 (the value determined to be close to critical), and another to generate a red alarm when the value exceeds 15,000 (the value determined to be critical.) For more information, refer to the SpectroWATCH Operator’s Reference. How To Manage Your Network with SPECTRUM Page 64 Configuring for Fault Isolation The following sections cover SPECTRUM’s fault isolation configuration options: • • • • • • • • • • • • Configuring General Fault Isolation Parameters Configuring Management Neighbors (Page 69) Configuring Maintenance Mode (Page 73) Configuring Cross-Landscape Fault Correlation (Page 76) Configuring Port Fault Correlation (Page 78) Configuring Port Status Monitoring (Page 90) Wide Area Link Monitoring (Page 98) Port Layer Alarm Suppression (Page 101) Port Criticality (Page 101) Live Pipes (Page 102) Automatic Port Status Alarm Clearing (Page 106) Suggested Port Fault Settings for Optimal Fault Notification (Page 106) Configuring General Fault Isolation Parameters The Fault Isolation model is a model in SPECTRUM’s database which stores information about various aspects of SPECTRUM’s fault isolation functionality. The Fault Isolation Model Information view allows you to configure these settings. To access the Fault Isolation Model Information view: 1 Highlight the VNM icon and select Configuration from the Icon Subviews menu. This brings up the Landscape Configuration view. 2 In the Configure/Information panel, double-click the Fault Isolation entry to open the Fault Isolation Model Information view (Figure 22). How To Manage Your Network with SPECTRUM Page 65 Figure 22: The Fault Isolation Model Information View The following fields can be configured: ICMP Support Enabled This field controls the value of the Fault Isolation model’s ICMP_SUPPORT attribute. This attribute is used during fault isolation to determine if an attempt should be made to contact a device using the ICMP protocol when trying to ascertain the fault status of the device. When the ICMP_SUPPORT attribute is enabled (set to True) on the Fault Isolation model, SPECTRUM will look at the setting of the ICMP_SUPPORT attribute at the device level. If ICMP support is also How To Manage Your Network with SPECTRUM Page 66 enabled on the device, SPECTRUM will attempt to contact the device using the ICMP protocol. However, when ICMP_SUPPORT is disabled (set to False) on the Fault Isolation model, this setting will take precedence over the setting at the device level and will prevent any attempt to contact the device using the ICMP protocol for fault isolation. The default setting for this field is True. ICMP Timeout The amount of time (in milliseconds) that SPECTRUM will wait for a response to an ICMP “ping.” If a response is not received within this period of time, SPECTRUM assumes the device has “timed out.” The default setting for this item is 3000ms (3 seconds). ICMP Trycount This field (available in SPECTRUM 6.6 SP5 and later) specifies the total number of attempts to contact a device using the ICMP protocol that will be made before SPECTRUM determines the device is down. Lost Device Trycount The number of retries that SPECTRUM attempts for each SNMP request sent to a device after contact with the device is lost. The default setting for this field is 1. Contact Lost Model Destruction This field controls whether a device is automatically destroyed when its CONTACT_STATUS is set to false. When this field is set to Enabled, models that have had their CONTACT_STATUS set to “lost” for a specified period of time (see the Destruction Time Delay field) will be destroyed automatically. When this item is set to Disabled, models will never be automatically destroyed as a result of the value of the CONTACT_STATUS attribute. The default setting for this field is Disabled. Destruction Delay Time This field specifies the length of time (in seconds) that a model must continuously have its CONTACT_STATUS attribute set to “lost” before it is automatically destroyed. The default value is 604800 seconds (7 days). How To Manage Your Network with SPECTRUM Page 67 Destruction Event Generation This field controls whether an event message is created when a model is automatically destroyed. When this field is set to Enabled, an event will be generated each time a model is destroyed as a result of its CONTACT_STATUS being continuously set to “lost” for the length of time specified in the Destruction Delay Time field. When this field is set to Disabled, no event message will be generated. The default setting for this field is Enabled. Port Fault Correlation This field allows you to configure the port fault correlation feature. See Port Fault Correlation Options on Page 79 for an explanation of each of the configuration options. Unresolved Fault Alarm Disposition If SPECTRUM’s information about the connectivity of your network model is incomplete, SPECTRUM may be unable to find the root cause of a network outage. In this case, the status of all devices affected by the outage is set to gray, and a red unresolved fault alarm is generated. This alarm indicates that SPECTRUM has lost contact with a group of devices, but was unable to pinpoint the cause. All of the devices to which SPECTRUM has lost contact are listed in the impact scope of the alarm. The model name and other details of the lost devices also appear in the event that generated the alarm. The Unresolved Fault Alarm Disposition field allows you to control how the unresolved fault alarm is generated. When set to Fault Isolation Model, the alarm will be generated on the Fault Isolation model. When set to Device In Fault Domain, the alarm will be generated on one of the devices with which SPECTRUM has lost contact. When determining which device to generate the alarm on, SPECTRUM looks for the device with the highest criticality. If the highest criticality is shared by two or more devices, SPECTRUM generates the alarm on the first of these devices that it finds. If all devices have the same criticality, then SPECTRUM chooses the device with the lowest model handle. How To Manage Your Network with SPECTRUM Page 68 Configuring Management Neighbors SPECTRUM lets you customize its fault isolation algorithm at the interface level to suppress a red alarm at the device level. This is desirable in cases where a given device still supports data traffic, but management traffic has been interrupted due to a fault with the interface between the device and a neighboring device that lies between it and SPECTRUM. In this case, the default algorithm would generate two red alarms: one for the first device and one for the interface on the neighboring device. In other words, there would be multiple alarms for what was essentially a single fault. However, you can avoid this situation by designating one or more of a device’s connected interfaces as “management neighbors” (as opposed to “data-only neighbors”). Then, when the interface with the management neighbor is down, SPECTRUM will still generate a red alarm for the management neighbor’s interface, but will only assert a condition of “Suppressed” (gray) for the first device. To designate any of a device’s connected interfaces as a management neighbor, do the following: 1 Highlight the device icon in a SPECTRUM Topology view. 2 From the device icon’s Icon Subviews menu, select the Fault Management option. This will display the Fault Management View shown in Figure 23. How To Manage Your Network with SPECTRUM Page 69 Figure 23: 3 Device Fault Management View In the Fault Management View, click the Fault Isolation button. (The other buttons/fields are explained under Fault Management View Settings on Page 107.) This will display the Device Fault Isolation Configuration Options view shown in Figure 24. Note that the sample scenario depicted in this view includes two management neighbors and thus two red alarms are asserted: one for B’s interface to A and one for C’s interface to A. Device A itself is in a “Suppressed” (gray) condition. How To Manage Your Network with SPECTRUM Page 70 Figure 24: 4 Device Fault Isolation Configuration Options View At the bottom of the Device Fault Isolation Configuration Options view, click the Specify Management Neighbors button to display the Select Management Neighbor Interfaces dialog box shown in Figure 25. How To Manage Your Network with SPECTRUM Page 71 Figure 25: The Select Management Neighbor Interfaces Dialog Box 182.12.13.2_2 2 Good 0x42a003d4 5 In the Select Management Neighbor Interfaces dialog box, designate one or more management neighbors by moving the entry (which comprises the interface’s model name, OID, status, and model handle) from the Connected Interfaces list to the Management Neighbor Interfaces list. You can move an entry from one list to the other either by double-clicking it or by selecting it and then clicking the single arrow button that points to the desired list. Clicking a double arrow moves all entries from one list to the other. How To Manage Your Network with SPECTRUM Page 72 Tip: 6 If the selected device has a large number of connected interfaces, you can view only a specific subset of them or find a particular one by using the Filter/Search button at the bottom of either list. When Filter is selected, the list will show only those entries that contain the string you enter in the adjacent text entry box. When Search is selected, the first entry containing the entered string will be highlighted. Clicking the Next button will highlight the next entry that contains the string. Click OK to put your management neighbor designations into effect and close the dialog box, or click Cancel to revert to the previous designations and close the dialog box. Configuring Maintenance Mode Maintenance Mode for Devices In a device’s Fault Management View, setting the Enable SPECTRUM Management button to FALSE places the device model into maintenance mode, suspending management traffic to the device and its components and preventing the generation of any events or alarms on their behalf. When a device has been placed into maintenance mode, its icon displays a device condition color of brown. You can use the Modeling Gateway Toolkit and the Schedule element to create a maintenance mode schedule for a particular device model. See the Modeling Gateway Toolkit Guide (5069) for more information. When a model is in maintenance mode, no events will be processed for that model. That includes events that would normally clear an alarm on the model, as well as events that would create an alarm. For example, a link_down event generated an alarm on the model prior to the model being placed in maintenance mode. If a link_up trap event is generated while the model is in maintenance mode, the alarm is not cleared as the link_up event is not processed. You can configure SPECTRUM to either hide or show secondary alarms when a device is in maintenance mode. This is done using the parameter How To Manage Your Network with SPECTRUM Page 73 “Show Secondary Alarms When Device is in Maintenance” available in the Alarm Update Control tab of the Alarm Manager Preferences. If this parameter is enabled, secondary alarms will be shown when a device is in maintenance mode. If disabled (the default), secondary alarms will be hidden when a device is in maintenance mode and will be shown again when the device is taken out of maintenance mode. This setting will only apply if primary and secondary alarms are enabled in the Alarm Manager filter settings. See the Enterprise Alarm Manager User Guide (2065) for more information on the Alarm Manager. Putting a device model into maintenance mode will, by default, also put its interface models and application models into maintenance mode. Brown alarms will be generated on device models in maintenance mode but will not be generated on application and interface models that have inherited maintenance mode from devices. Optionally, you can use the SPECTRUM Command Line Interface (CLI) to change this behavior: 1 To generate brown alarms on interface models that have inherited maintenance mode, set the device model rollDownAlarmToIF attribute (0x00012a7a) to TRUE. 2 To generate brown alarms on application models that have inherited maintenance mode, set the device model rollDownAlarmToApp attribute (0x00012a7b) to TRUE. See the Command Line Interface (0664) guide for more information about CLI. Interface models can be put into maintenance mode independently of device models, as described in Maintenance Mode for Ports (Page 74). Maintenance Mode for Ports In the Port Fault Management View, setting the Enable SPECTRUM Management button to FALSE places the port model into maintenance mode, suspending management of the port, while still performing regular management on the rest of the device. When in maintenance mode the following items will be true for the port model: How To Manage Your Network with SPECTRUM Page 74 • The condition of the port model will be brown. • Alarms will not be created for the port. • Events will not be logged for the port. • No polling, logging, or other device communication will be done on behalf of the port model. Other device communications will not be affected. • Link down traps sent are ignored. An event is created on the device model indicating that the trap was received, but no alarm is generated. • If Live Pipes is enabled for a connection and one of the end points is a port in maintenance mode, the pipe on the Topology view will appear brown. Status polling for the port will stop. • If a connection is modeled with a WA_Link model connection to two ports, and one of those ports is in Maintenance Mode, a maintenance (brown) alarm will be created on the WA_Link and WA_Segment models. The WA_Link icon will be brown. If Live Pipes is enabled on this link, the pipe will remain green as long as the other port is up. If the other port goes down or is unreachable, the pipe will turn gray. If SPECTRUM loses contact with a device model connected to the port in maintenance mode, the Device Has Stopped Responding to Polls alarm will be suppressed for that device as well as all adjacent devices. If device contact lost alarms are suppressed because of their position relative to a port in maintenance mode, the maintenance mode alarm will reflect these lost devices in its Impact Severity and Impact Scope attributes. These lost devices will be viewable in the Impact Scope View for that maintenance alarm. For more information on Impact Severity and the Impact Scope view, refer to the Enterprise Alarm Manager User Guide (2065). How To Manage Your Network with SPECTRUM Page 75 Configuring Cross-Landscape Fault Correlation In a Distributed SpectroSERVER (DSS) environment, a network administrator may need to model a router from a local landscape in a remote landscape in order for its connections to participate in fault isolation for the remote landscape. This proxy model doesn't need to participate in alarm generation in the remote landscape because it already does so in the local landscape, where it is modeled “normally”. It is in the local landscape that alarms for the router are tracked and trouble tickets are created. (See the Distributed SpectroSERVER Guide (2770) for more information on distributed network management.) In such a scenario, Cross-Landscape Fault Correlation can prevent multiple red alarms for the same outage. With the Enable Event Creation attribute set to FALSE in the proxy model’s Fault Management View, SPECTRUM suspends the creation of events for the model (and any component models such as boards or ports). This effectively disables alarms for the model, but unlike maintenance mode (see Configuring Maintenance Mode (Page 73)), SNMP communication with the proxy model continues, and so too does its participation in fault isolation. Cross-Landscape Fault Correlation Example Figure 26: Network with Multiple Landscapes How To Manage Your Network with SPECTRUM Page 76 Landscape A (the local landscape) contains core routers, including Router R. Landscape B (the remote landscape) contains customer routers with the core Router R modeled a second time, this time as a proxy (Figure 27). This proxy model has its IsEventCreationEnabled attribute set to FALSE (the default setting is TRUE). The device is being polled, but will not generate events or alarms. Figure 27: Router R in Landscapes A and B If Router R goes down (Figure 28), Landscape B loses contact with the proxy and customer routers. However, only one red alarm is generated, in Landscape A. The proxy router stays green in Landscape B, where the alarms are suppressed because the proxy’s IsEventCreationEnabled attribute is set to FALSE. Figure 28: Alarm with Cross-Landscape Fault Correlation How To Manage Your Network with SPECTRUM Page 77 Configuring Port Fault Correlation Note: Note: The Port Fault Correlation feature is available in SPECTRUM 6.6 SP3 and later. SPECTRUM lets you customize its fault isolation algorithm to resolve the root cause of a network outage to the port level. This is most desirable in cases where a single physical port, such as a Frame Relay interface, supports multiple logical connections to remote devices. If the physical port goes down, SPECTRUM can suppress the alarms on all downstream devices in favor of a single red alarm on the physical interface, thus significantly reducing the number of alarms which need attention. The impact severity and scope of the red alarm on the physical interface will contain all downstream devices, as well as the physical interface. How To Manage Your Network with SPECTRUM Page 78 Port Fault Correlation Options Port Fault Correlation is configurable via the Port Fault Correlation setting in the Fault Isolation Configuration View. To access this view, right click on the VNM model icon and select the Configuration option from the VNM’s Icon Subviews menu. In the Landscape Configuration view’s Configure/Information Panel, double-click the Fault Isolation entry. The Port Fault Correlation setting has the following possible values: Disabled, All Connected Ports, Management Neighbors Only, All Connect Ports - Multiple Devices Only: • If set to Disabled, port fault correlation will not run. The root cause of a network outage will remain a red alarm on a device model. Fault Isolation will, however, still examine all of the device’s connected ports to see if they are all in Maintenance Mode. If so, the alarm on the device model will be suppressed. • If set to All Connected Ports, port fault correlation will run, examining all ports that exist on “up” neighbors which are connected to the down device as possible root causes of the outage. No additional manual configuration is required. • If set to Management Neighbors Only, the port fault correlation algorithm will run, but will only examine ports which were previously configured manually as “management neighbors” (see Configuring Management Neighbors on Page 69) as possible root causes of the outage. • If set to All Connected Ports - Multiple Devices Only, port fault correlation will run, examining all ports that exist on “up” neighbors which are connected to the down device as possible root causes of the outage. However, SPECTRUM will only resolve the outage to the port level if there is more than one device model that would have a red alarm which can be correlated to the port alarm. If only one connected device alarm can be correlated to the port alarm, SPECTRUM will not suppress the device alarm. Instead, both the port and device alarm will be generated. How To Manage Your Network with SPECTRUM Page 79 Port Fault Correlation Criteria The following criteria must be met in order for the root cause of an outage to be resolved to the port level: • The down device must have one, and only one, “up” neighbor. If the down device has more than one “up” neighbor, port fault correlation will not be performed. This is done to reduce the number of alarms for a single problem. If multiple up neighbors were a valid criteria, and all connected ports were down, multiple red alarms would exist, all with the same impact severity and scope. If a device has more than one up neighbor, SPECTRUM assumes the problem lies with the device, not the upstream ports and creates a single red alarm on the device. • The down device must have at least one connected port (or management neighbor port) on an “up” neighbor that is down. • If multiple ports on the “up” neighbor connect to the down device (e.g. link aggregation), all of the ports must be down. • A port is considered “down” if it is either operationally down, or the port model has been put into Maintenance Mode. • There must be an alarm on at least one of the down ports. (Otherwise, there would be no alarm to which SPECTRUM could resolve the outage.) • If Port Fault Correlation is set to Management Neighbors Only, management neighbors for the down device must have been configured before the outage occurred (see Port Fault Correlation Options on Page 79). Port Fault Correlation Caveats The Port Fault Correlation feature overrides the Suppress Linked Port Alarms setting in the Live Pipes Configuration View (see Live Pipes on Page 102). When set to TRUE, this setting suppresses the alarm on an upstream port if it's connected to an unreachable device. If Port Fault Correlation is enabled, and the upstream port is the root cause of an outage, SPECTRUM will force the upstream port to be alarmed. How To Manage Your Network with SPECTRUM Page 80 Only the Criticality of the alarmed port will be used in the impact severity/scope calculation of the root cause alarm. The Criticality of any sub-interfaces (such as DLCI ports) will not be included. Port Fault Correlation is supported by Device models only. Models such as Fanouts and Unplaced do not support this feature. WA_Link models have their own mechanism for supporting port fault correlation, Link Fault Disposition, which is explained in Wide Area Link Monitoring on Page 98 in this document. If multiple ports on the “up” neighbor connect to the down device (e.g. link aggregation), and all of the ports are down, multiple red alarms will exist as the root causes of the outage. Each red alarm will contain the same impact severity and scope. The root cause of the outage in this case is, in fact, all of the ports, not just one of them. Port Fault Correlation Examples The following fault scenarios describe the benefits of Port Fault Correlation. Fault Scenario 1 Figure 29: Fault Scenario 1 How To Manage Your Network with SPECTRUM Page 81 In the diagram in Figure 29, assume SPECTRUM must communicate through Router A in order to reach Routers 1 through N, and that this is the only means by which SPECTRUM can reach them. Each remote router is connected to Router 1 using a frame relay link. In SPECTRUM, this is modeled by connecting each DLCI port model to the other device. Assume the physical frame relay interface (FR A in Figure 29) goes down. This means that all virtual circuits provisioned on the interface will go down as well. With Port Fault Correlation disabled, the alarms shown in Figure 30 will occur: Figure 30: Fault Scenario 1: Alarms without Port Fault Correlation If a trap is received for FR A going down (or a live pipe is configured to be on), the physical frame relay interface will have a red alarm on it. In addition, all routers connected to the frame relay interface will have a red alarm on them. This could mean multiple red alarms could be generated by SPECTRUM for a single problem. Port Fault Correlation reduces the number of alarms generated for this problem down to a single alarm without requiring any manual configuration beforehand. Figure 31 shows the results with Port Fault Correlation enabled. How To Manage Your Network with SPECTRUM Page 82 Figure 31: Fault Scenario 1: Alarms with Port Fault Correlation A single red “Bad Link” alarm will be seen in the Alarm Manager. That alarm will have an Impact Scope and Severity which contains the following models: FR A, Routers 1 through N, and all unreachable devices that are downstream from Routers 1 through N. How To Manage Your Network with SPECTRUM Page 83 Fault Scenario 2 This fault scenario illustrates the benefits of setting the Suppress Linked Port Alarms and Port Fault Correlation attributes as recommended in Suggested Port Fault Settings for Optimal Fault Notification (Page 106). In the diagram in Figure 32, assume the VNM must communicate through Routers A and B in order to reach Routers C, D, and E, and that is the only means by which the VNM can reach them. In SPECTRUM, port-level connectivity is modeled as shown. Figure 32: Fault Scenario 2: Multiple “Up” Neighbors Assume Router C goes down. This will cause SPECTRUM to lose contact with Routers C, D, and E, and will make Ports A and B go down as well. If Suppress Linked Port Alarms is set to TRUE, and Port Fault Correlation is set to All Connected Ports, only a single red alarm on Router C will result (Figure 33). How To Manage Your Network with SPECTRUM Page 84 Figure 33: Scenario 2: Multiple “Up” Neighbors The upstream ports (Ports A and B) will have their alarms suppressed because Suppress Linked Port Alarms is TRUE. Even though Port Fault Correlation is enabled, Router C has multiple “up” neighbors, so the fault won't be resolved to the port level. When this occurs, SPECTRUM assumes the fault is with the device itself, not the connected ports. If you set Suppress Linked Port Alarms to FALSE, and Port Fault Correlation is still set to All Connected Ports, Router C and the upstream ports will be alarmed (if the status of the ports is being polled, or SPECTRUM receives a LinkDown trap), as shown in Figure 34. How To Manage Your Network with SPECTRUM Page 85 Figure 34: Fault Scenario 2: Multiple “Up” Neighbors Once again, the fault wasn't resolved to the port level because Router C has multiple “up” neighbors. Since Suppress Linked Port Alarms is disabled, SPECTRUM will alarm the upstream ports. If Router C had only one “up” neighbor, as in the diagram in Figure 35, even if Suppress Linked Port Alarms were set to TRUE (assuming Port Fault Correlation is still set to All Connected Ports), SPECTRUM will resolve the fault down to the port level. Port Fault Correlation (see Port Fault Correlation Criteria on Page 80) forces the upstream port to be alarmed, and the alarm on Router C is suppressed. How To Manage Your Network with SPECTRUM Page 86 Figure 35: Fault Scenario 2: Single “Up” Neighbor Fault Scenario 3 This fault scenario demonstrates what happens when Port Fault Correlation is set to All Connected Ports - Multiple Devices Only. In the diagram in Figure 36, assume SPECTRUM must communicate through Router A in order to reach Routers 1 through N, and that this is the only means by which SPECTRUM can reach them. Each remote router is connected to Router 1 using a frame relay link. In SPECTRUM, this is modeled by connecting each DLCI port model to the other device. How To Manage Your Network with SPECTRUM Page 87 Figure 36: Fault Scenario 3 Assume the physical frame relay interface (FR A in Figure 36) goes down. This means that all virtual circuits provisioned on the interface will go down as well. With Port Fault Correlation disabled, the alarms shown in Figure 37 will occur. Figure 37: Fault Scenario 3: Alarms without Port Fault Correlation How To Manage Your Network with SPECTRUM Page 88 With Port Fault Correlation set to All Connected Ports – Multiple Devices Only, the following alarms will occur because multiple devices can be correlated to the frame relay interface Figure 38: Fault Scenario 3: Port Fault Correlation Set To “All Connected Ports - Multiple Devices Only” Known Port Fault Correlation Anomalies There is one known anomaly associated with Port Fault Correlation. If a red alarm is generated on a port model as the root cause of an outage, you may then choose to put that port model into Maintenance Mode. If so, the red alarm would be replaced with a brown alarm. The brown alarm will still contain the same impact severity and scope (except the maintenance port will no longer contribute to the impact). If you then decide to take the port out of Maintenance Mode, the red alarm will reappear. It is possible, in this scenario, for the impact scope and severity of the red alarm to be lost. How To Manage Your Network with SPECTRUM Page 89 Configuring Port Status Monitoring Note: Note: The Port Status Monitoring features described in this section are available in SPECTRUM 6.6 SP3 and later. SPECTRUM provides the following methods of monitoring the status of ports: • Link Traps (see Link Trap Handling on Page 93) Link traps allow the user to monitor the status of their ports without the cost of polling. However, traps are not always the most reliable notification mechanism of port status. • PollPortStatus (see PollPortStatus Feature on Page 96) The PollPortStatus feature allows users to poll the status of a port even if its connectivity is not modeled in SPECTRUM. • Live Pipes (see Live Pipes on Page 102) Live Pipes let you turn on port status monitoring for individual links. This is a more reliable monitoring method than traps because SPECTRUM will periodically poll the status of the link (with an increased cost in performance). In addition, Live Pipes let you graphically verify which links are being monitored. • WA_Link port monitoring (see Wide Area Link Monitoring on Page 98) WA_Link models automatically enable a live pipe for any connected ports. Port Status Polling Criteria In general, in order for the status of a port to be polled, the following criteria must be met: • The Polling_Status (0x1154f) of the port model must be TRUE. • The Polling_Interval (0x10071) of the port model must be non-zero. • The Polling_Status of the port's device model must be TRUE. How To Manage Your Network with SPECTRUM Page 90 • Neither the port model nor the port's device model can be in Maintenance Mode (the isManaged (0x1295d) attribute for both must be set to TRUE). Note: Note: If a port has always been down since it was modeled in SPECTRUM and the Port Always Down Alarm Suppression attribute is set to Enabled (see Table 6, Port Status Monitoring Settings (Page 105), this port will only be polled once per hour (every 3600 seconds). In addition, all red alarms on these ports are suppressed and a gray condition is asserted. If the Port Always Down Alarm Suppression attribute is set to Disabled, the port will be polled according to the criteria listed above. These settings are available in SPECTRUM 6.6 SP5 and later. How To Manage Your Network with SPECTRUM Page 91 Port Status Events and Alarms SPECTRUM's port status monitoring engine uses the events and alarms listed in Table 1 to notify the user of a change in status. Table 1: Port Status Events and Alarms Event Description Event ID Alarm Description Alarm ID Port Condition Color Port status good 0x10d10 N/A N/A Green Port status bad 0x10d11 BAD LINK 0x1040a Red Port status disabled 0x10d12 LINK DISABLED 0x1040b Brown Port status unknown 0x10d13 LINK STATUS UNKNOWN 0x1040e Gray Port status unreachable 0x10d14 UNREACHABLE 0x1040c LINK Gray Port status initial 0x10d15 N/A N/A Blue Port lower layer down 0x10d16 BAD LINK, BUT ALARM WAS SUPPRESSED 0x1040f Gray Port up, but linked with down port 0x10d17 LINK MAY NOT BE UP 0x10410 Gray Port connected to down port or device 0x10d18 PORT ALARM SUPPRESSED 0x10411 Gray Port status bad, but connected to WA_Link whose LinkFaultDisposition is LinkOnly (see Link Fault Disposition on Page 99) 0x10d2d PORT ALARM SUPPRESSED 0x10d2d Gray How To Manage Your Network with SPECTRUM Page 92 Link Trap Handling Traps provide a means for network devices to let a management system know that a significant event has occurred on the network. Link Down and Link Up traps are perhaps the most important traps when it comes to port status monitoring. These traps tell the management system that a port has either become inoperable, or has come back up. When SPECTRUM receives a Link Down trap, it polls the status of the corresponding port once to verify its status and generates one of the Port Status Events and Alarms listed in Table 1 on the affected port. Note: Note: SPECTRUM generates a yellow alarm on the device model to allow easy access to vendor-specific trap data; however, we no longer generate the trap-specific events and alarms on the affected port model. In SPECTRUM versions 6.6 SP4 and below, SPECTRUM polls the status of the corresponding port when it receives a Link Up trap, usually resulting in the alarm being cleared. In SPECTRUM versions 6.6 SP5 and above, once a Link Down trap is received, SPECTRUM sets the OutstandingLinkDownTrap attribute on the port to TRUE. This will cause SPECTRUM to poll the status of the port regardless of the port status polling criteria. When SPECTRUM receives a Link Up trap for the port, or when the port’s status is determined to be up based on a poll, the value of the OutstandingLinkDownTrap attribute is set to FALSE and polling will take place based on the value of the port status polling criteria. (See Port Status Polling Criteria on Page 90 for more information on when a port is polled.) When all of the ports for which SPECTRUM has received a Link Down trap are back up, the yellow alarm on the device will be cleared. Table 2 provides the attributes that can be used to control how SPECTRUM handles link traps. How To Manage Your Network with SPECTRUM Page 93 Table 2: Link Trap Attributes Attribute ID Description AlarmOnLinkDownIfTypes 0x1290f This attribute contains a mapping of ifType values to a value which determines how to handle the trap for that particular ifType and model type (0 for never, 1 for always, and 2 for check admin). This can be customized in the MTE on a per-model type basis. When a port model is created, its AlarmOnLinkDownTrap (0x11fc2) attribute is automatically populated with the value which corresponds to its particular ifType. AlarmOnLinkDownTrap 0x11fc2 This attribute contains a value which tells SPECTRUM how to handle a Link Down trap on this particular port model (0 for never, 1 for always, and 2 for check admin). This attribute is available via the port model's Interface Trap Configuration View (Page 95). AssertLinkDownAlarm 0x12957 This attribute is used to determine if the yellow alarm should be generated on the device model. It is read from the port model for which SPECTRUM received the trap. This attribute is available via the port model's Interface Trap Configuration View (Page 95). How To Manage Your Network with SPECTRUM Page 94 Interface Trap Configuration View For many device models, you can configure the processing of link down traps received for individual port models either through an Interface Trap Configuration View (accessed from the port model’s Icon Subviews menu) or a Generic Interface Trap Configuration View (accessed from the Trap Configuration button in the port model’s Model Information View). Both varieties of this view allow you to suppress link down alarms for the selected port model and/or its parent device model. For the port model, you can also select the Check Admin option, in which case SPECTRUM will check the interface’s Administrative Status whenever a link down trap is received, generating a red alarm only if the status is “Up” and a brown alarm for any other status. Consult the SPECTRUM management module guide for the type of device you are interested in to see if that module supports a trap configuration view. Figure 39 shows an example of the Interface Trap Configuration View. Figure 39: Interface Trap Configuration View How To Manage Your Network with SPECTRUM Page 95 PollPortStatus Feature The PollPortStatus feature lets you monitor the status of a port even if the port’s connectivity is not modeled. The PollPortStatus attribute exists for both Device and Port models with a different attribute ID for each of the two model types. This lets you enable and disable port status polling at the device and/or port level. By default, PollPortStatus is set to TRUE at the device level and FALSE at the port level. Note: Note: To reduce network traffic, SNMP reads for polled ports on the same device are grouped together into larger SNMP requests. This provides performance benefits that are most noticeable when many ports are polled by this method in a single SpectroSERVER. Utilizing PollPortStatus to Watch a Connected Port’s Status The PollPortStatus attribute at the device level (0x12809) controls port status polling on a per-device basis. If TRUE, polling is enabled for that device. If FALSE, no ports will be polled, even if a port model’s PollPortStatus attribute is TRUE. When changed to FALSE, alarms will be cleared for any port which is not involved in a live pipe. The PollPortStatus attribute at the port level (0x1280a) controls polling for each port model. If this attribute is set to TRUE (and PollPortStatus for the device is also set to TRUE), the status of the port will be polled, and alarms will be generated if needed. When changed to FALSE, any alarm on the port will be cleared. Table 3 shows that a port’s status will be polled only if PollPortStatus is TRUE for both a given port model and its device model. How To Manage Your Network with SPECTRUM Page 96 Table 3: Value of PollPortStatus for Device Model Port Status Polling Conditions Value of PollPortStatus for Port Model Result FALSE FALSE Port status is not polled for any port on device TRUE FALSE Port status is not polled for this port on device FALSE TRUE Port status is not polled for any port on device TRUE TRUE Port status is polled for this port on device SPECTRUM watches the polling interval to determine when to poll port status. When a port is polled, the port’s status is determined and an appropriate alarm is generated (RED, BROWN, or GRAY) if necessary. If a BAD LINK alarm is generated on a port (alarm code 0x1040a), and then polling on that port is disabled by changing the value of PollPortStatus to FALSE and disabling Live Pipes, an event will be generated to automatically clear the BAD LINK alarm. Note: Note: Now that port status polling has been integrated in SPECTRUM, the PollPortStatus attribute can be set to TRUE while the Live Pipes functionality is enabled. This will not cause redundant network traffic. Enabling Port Status Polling You can enable port status polling for all future models of a given type using the Model Type Editor (MTE), or on a current, per-model basis using the SPECTRUM Command Line Interface (CLI). For example, use the Model Type Editor (MTE) to set PollPortStatus to TRUE for both How To Manage Your Network with SPECTRUM Page 97 device and port model types. When polled, interface models will now generate appropriate alarms when needed. For more information see the Model Type Editor User’s Guide (0659). PollPortStatus can also be set at both the device and port level using the Global Attribute Editor. For more information, see the Search Manager User’s Guide (2383). Note: Note: You can use the SPECTRUM Command Line Interface (CLI) to enable or disable PollPortStatus for a single model. See the Command Line Interface (0664) documentation for more information. Wide Area Link Monitoring SPECTRUM polls any ports connected to a WA_Link model for status automatically. This polling is controlled by the Polling_Status and Polling_Interval of the WA_Link model. If the WA_Link model's Polling_Status is TRUE, and its Polling_Interval is non-zero, SPECTRUM will automatically make the pipes connected to a WA_Link “live” and set the PollPortStatus for port models to TRUE. The live pipe lets you visually verify that SPECTRUM is monitoring the status of the connected ports. If you choose to turn off the live pipe, but leave the WA_Link’s Polling_Status set to TRUE, Polling_Interval set to a non-zero number, and the ports’ PollPortStatus set to TRUE, SPECTRUM will continue to monitor the status of the ports. Note: Note: In SPECTRUM versions prior to 6.6 SP5, the PollPortStatus is not automatically set to TRUE. You must make this change manually. If you do not make this change and you turn off the live pipe, the port status will not be monitored. How To Manage Your Network with SPECTRUM Page 98 Link Fault Disposition The Link Fault Disposition setting gives you more control and flexibility over fault alarming. When a wide area connection goes down, alarms could be generated on ports and/or the link model. Using the WA_Link Fault Management View (Page 99) for a WA_Link model, you can set the Link Fault Disposition (0x129e2) attribute. To access this view, rightclick on a WA_Link model and choose Fault Management from the icon subviews menu. The default setting for Link Fault Disposition is BothPortsAndLink. It can be set to one of three different modes: BothPortsAndLink, PortsOnly, or LinkOnly. If this button is set to BothPortsAndLink, alarms will be generated on both the connected ports and the link model. If set to PortsOnly, only the connected ports will be alarmed, and the WA_Link will be suppressed. If set to LinkOnly, only the WA_Link model will be alarmed, and the ports will be suppressed. Figure 40: WA_Link Fault Management View How To Manage Your Network with SPECTRUM Page 99 Wide Area Link Monitoring Scenarios Consider the network topology shown in Figure 41. Table 4 and Table 5 illustrate two WA_Link monitoring scenarios for this topology. Figure 41: Example WA_Link Network Topology Device A Device B Port 1 Port 2 WA_link Table 4: Link Goes Down, Device B Loses Contact Link Fault Disposition Port 1 Condition WA_Link Condition Port 2 Condition BothPortsAndLink Red Red Gray PortsOnly Red Gray Gray LinkOnly Gray Red Gray How To Manage Your Network with SPECTRUM Page 100 Table 5: Link Goes Down, Device B Remains Contactable Link Fault Disposition Port 1 Condition WA_Link Condition Port 2 Condition BothPortsAndLink Red Red Red PortsOnly Red Gray Red LinkOnly Gray Red Gray Note: Note: In Scenario 2, if the Suppress Linked Port Alarms (Page 106) setting is set to TRUE, then only one of the ports will be alarmed (red). The other port will be suppressed (gray). Port Layer Alarm Suppression Devices that support advanced network technologies, such as Frame Relay and Link Aggregation, have logical entries in the ifTable representing higher layer interfaces. SPECTRUM models these logical layers according to the ifStackTable. When a monitored higher layer port goes down (such as a Frame Relay DLCI, or logical trunk interface), SPECTRUM will query the statuses of all lower layer interfaces before alarming the port which went down. If all of the lower layer interfaces are down as well, SPECTRUM will suppress the higher layer interface alarm. A key example is a physical Frame Relay interface going down which has multiple circuits provisioned on it. All of the higher layer DLCI port models will be suppressed, and the single red alarm will exist on the physical Frame Relay interface. Port Criticality You can assign a relative importance value to port models using the port criticality (0x1290c) attribute. The criticality of a port is used in the Impact Severity calculations of an alarm on any port which may cause a network outage. You can also display the criticality of a port in the Web How To Manage Your Network with SPECTRUM Page 101 Alarm View to allow prioritization of port alarms. The port criticality attribute can be set for an individual port using the port model's Port Fault Management View, or the Global Attribute Editor. For more information, see the Search Manager User’s Guide (2383). Figure 42: Port Fault Management View Live Pipes Live pipes lets you turn on port status monitoring for individual links and display the status of a link through the use of status colors. A link is a connection between two devices that has been resolved to the port level. Live pipes display a combined condition color from the two resolved ports representing each side of the link. When a pipe is deleted, SPECTRUM removes all of its underlying associations (such as links_with and connects_to). If the pipe being How To Manage Your Network with SPECTRUM Page 102 deleted represents more than one link, SPECTRUM asks the user to confirm the deletion. For more information on pipes, refer to the SPECTRUM Icons Reference Guide (2518) and Getting Started with SPECTRUM for Administrators (0985). Enabling or Disabling Live Pipes System-Wide Live pipes are enabled system-wide by default. Without live pipes enabled on a system-wide basis, enabling live pipes for individual links is not available. To enable or disable live pipes system-wide: 1 In the Landscape Configuration view’s Configure/Information Panel, either click the LivePipes entry and click OK or double-click the LivePipes entry to access the Live Pipe Model Information view. 2 Toggle the Live Pipes button to Enable or Disable. 3 Select Save All Changes from the File menu. Note: Note: You must exit and restart SpectroGRAPH for enabling or disabling live pipes system-wide to take effect. Enabling or Disabling Live Pipes on Individual Links Live pipes for individual links are disabled by default. All individual pipes are gold or silver until live pipes are enabled. When an individual live pipe is enabled, the ok_to_poll (0x11dd8) attribute is set to TRUE for both ports involved in the link. The status of the linked ports will be monitored if ok_to_poll is TRUE, and the port status polling conditions in Table 3 (Page 97) are met. To enable a live pipe for an individual link: 1 Click on the link to highlight it and select Icon Subviews from the View menu or click on the link with the right mouse button. 2 Select Enable Live Links from the menu. How To Manage Your Network with SPECTRUM Page 103 3 In the Enable Live Links dialog box, set the toggle button and click OK. To disable a live pipe for an individual link: 1 Click on the link to highlight and select Icon Subviews from the View menu or click on the link with the right mouse button. 2 Select Enable Live Links from either menu. 3 In the Enable Live Links dialog box, raise (un-check) the toggle button and click OK. Note: Note: The Enable Live Links option is not available from either menu if live pipes have been disabled system-wide. Receiving Port Alarms In order to receive alarms on a port model, the following conditions must be met: • Live pipes must be enabled system-wide through the Landscape Configuration view. • Live pipes for the link of interest must be enabled separately. • There must be a model on the other side of the link. Note: Note: If there is no model on the other side of the link, an alarm can still be generated when the status of the link changes by setting a SpectroWATCH on the MIB-II ifOperStatus attribute. When the ifOperStatus attribute returns a value other than 1, an alarm is generated. With this method, an alarm can still be generated even if the port is intentionally down (the ifAdminStatus attribute has been set to “OFF.” Additional information about the status of a link is provided through the Link Information view. For information about the Link Information view, see SPECTRUM Icons. How To Manage Your Network with SPECTRUM Page 104 You can set the default value of ok_to_poll in the MTE for any port model type. When a port connection is made, SPECTRUM will set both ports' ok_to_poll attributes to their MTE default values so that the pipe will automatically become “live” if the user wishes. When the connection is removed in SPECTRUM versions 6.6 SP4 and below, ok_to_poll is reset to FALSE. When the connection is removed in SPECTRUM versions 6.6 SP5 and above, ok_to_poll remains at the MTE default value. When SPECTRUM notices a change in a link's status, it generates one of the Port Status events listed in Table 1 (Page 92) on the ports involved in the link, and also changes the color of the pipe to reflect its new status. The settings described in Table 6 appear in the Live Pipes Model Information View, accessible from the VNM model icon’s Landscape Configuration View. They allow you to control the Live Pipes service. Table 6: Port Status Monitoring Settings Setting Attribute ID Description Live Pipes 0x11df9 Setting this (global) option to Disabled turns off all pipes in the SpectroSERVER and no status polling will be performed for any ports associated with a pipe. However, any port which also has the PollPortStatus attribute (see PollPortStatus Feature) set to TRUE is still be polled for status changes. Alarm Linked Ports 0x11fbd Setting this option to TRUE causes ports with a good status to alarm gray when linked with a bad or unreachable port. How To Manage Your Network with SPECTRUM Page 105 Table 6: Port Status Monitoring Settings Setting Attribute ID Description Suppress Linked Port Alarms 0x11fbe Setting this option to TRUE causes ports with a bad status to suppress their red alarms and generate a gray alarm if either the linked port or the connected device is bad or unreachable. Only one port in the link is alarmed red. The other is gray. Port Always Down Alarm Suppression 0x12a03 Enabling this option will cause SPECTRUM to suppress red alarms and assert a gray condition if a port has always been down since first being modeled in SPECTRUM. (available in SPECTRUM version 6.6 SP5 and later) Note: Note: The settings described in Table 6 apply to port status monitoring throughout SPECTRUM, not just ports associated with live pipes. Automatic Port Status Alarm Clearing It is important to note that any port status alarm will be automatically cleared by SPECTRUM if both of the port's ok_to_poll and PollPortStatus attributes are set to FALSE. Suggested Port Fault Settings for Optimal Fault Notification Aprisma recommends that you change the default settings for both Suppress Linked Port Alarms (see Table 6) and Port Fault Correlation (see Configuring Port Fault Correlation on Page 78). These default settings How To Manage Your Network with SPECTRUM Page 106 are Disabled and Management Neighbors Only, respectively. For best results, change these settings to Enabled and All Connected Ports. Note: Note: Setting Suppress Linked Port Alarms to Disabled and Port Fault Correlation to Management Neighbors Only (the default settings) will approximate the fault notification behavior of SPECTRUM 6.6 with Service Pack 2. Additionally, Aprisma recommends changing the default setting for Link Fault Disposition on WA_Link (see Link Fault Disposition on Page 99) models. The default setting of BothPortsAndLink will result in multiple alarms if the link fails. Consider changing the setting to Link Only or Ports Only. Link Only is best in environments where the name of the WA_Link models or notes on these models is meaningful. Changing the setting to Ports Only may be better if fault notification consistency is most important. That is, regardless of the topology, you would prefer an alarm on a port model if there is a link failure. Fault Management View Settings The Fault Management view (accessed from a device or port model’s Icon Subviews menu) provides access to user-configurable fault management settings. Device Model Settings See Figure 23, Device Fault Management View, on Page 70. Disable Trap Events Set this selector button to TRUE to prevent the generation of events (and alarms) for traps that are received from the selected device. Enable SPECTRUM Management Set the Enable SPECTRUM Management button to FALSE to place the device model into maintenance mode and suspend management traffic to the device and its components, thereby preventing the generation of any How To Manage Your Network with SPECTRUM Page 107 events or alarms on their behalf. For more information, see Maintenance Mode for Devices (Page 73). Enable Event Creation When this value is set to FALSE, SPECTRUM suspends the creation of events for the model. This has the effect of disabling alarms for the model and its component models, but unlike maintenance mode, SNMP communication with the model continues. The default value for this setting, available in SPECTRUM versions 6.6 SP5 and above, is TRUE. For more information, see Configuring Cross-Landscape Fault Correlation (Page 76). Device Criticality This text entry box takes a value that indicates the relative importance of the device within the network being modeled. When SPECTRUM loses contact with the device, this value is summed for the device itself and all of its downstream neighbors for which a gray condition is now being asserted (because their actual status cannot be determined). The aggregate device criticality value is displayed as the Impact Severity value of the associated alarm in the Alarm Manager’s Impact Scope view. The default value for all devices is “1”; you can increase this value as desired depending on how important you consider the device to be (the higher the number, the more critical the device is to the network). For more information on the Impact Scope view, refer to the Enterprise Alarm Manager User Guide (2065). Redundancy Options This button allows you to control (enable/disable) whether SPECTRUM attempts to use redundant addresses in the event the device becomes unreachable via its primary address. See Redundancy (Page 180) for more information. Fault Isolation Click this button to display the Device Fault Isolation Configuration Options view. See Configuring Management Neighbors (Page 69) for more detailed information on using this button. How To Manage Your Network with SPECTRUM Page 108 Port Model Settings See Figure 42, Port Fault Management View, on Page 102. Enable SPECTRUM Management Set the Enable SPECTRUM Management button to FALSE to place the port model into maintenance mode and suspend management traffic to it, thereby preventing the generation of any events or alarms. For more information, see Maintenance Mode for Ports (Page 74). Enable Event Creation When this value is set to FALSE, SPECTRUM suspends the creation of events for the model. This has the effect of disabling alarms for the model, but unlike maintenance mode, SNMP communication with the model continues. The default value for this setting, available in SPECTRUM versions 6.6 SP5 and above, is TRUE. For more information, see Configuring Cross-Landscape Fault Correlation (Page 76). Port Criticality You can assign a relative importance value to port models using the port criticality (0x1290c) attribute. See Port Criticality (Page 101). Configuring Fault Management for Pingables Device fault isolation relies upon device models' knowledge of each other as neighbors. When a fault occurs, each device model that is lost sends an ARE_YOU_DOWN action to all of its neighbors. Depending on the answers received from neighboring device models, the lost model will either go gray or red. Neighbors are established by the creation of a Connects_to association between a device model and the port model of another device. The act of pasting a device model on the interface of another device model adds each device model to the other's neighbor list. Prior to SPECTRUM 6.0.3, this presented a problem for Pingable models. They don't have ports, and so they could not be made neighbors without the use of an inferred connector, like a Fanout. However, it is preferable for the relationship between models to be resolved directly, rather than by placing them both in a Fanout. This is because Fanouts and other How To Manage Your Network with SPECTRUM Page 109 inferred connector model types resolve faults differently and less directly during fault isolation. Connecting Two Pingable Models It is possible to connect pingables using one of the following methods: • In SPECTRUM versions 6.6 SP4 and later, you can establish neighbors simply by drawing pipes between device models. Drawing a pipe between two pingables will establish a Connects_to association between the two models and thus establish them as neighbors. • In SPECTRUM versions 6.0.3 and later, you can create a Connects_to association between two Pingable models via CLI. The syntax is: ./create association rel=Connects_to lmh=<model handle of Pingable A> rmh=<model handle of Pingable B> Once the Connects_to association has been established, you will see a gold pipe between the two models. When a fault occurs, each Pingable will send the other an ARE_YOU_DOWN action. Connecting a Pingable Model to a Device’s Port Model When creating a neighbor relationship between a pingable and a device model that has ports, you must paste the pingable onto the device’s port model. See Resolving Port Connections on Page 27 for further instructions. How To Manage Your Network with SPECTRUM Page 110 Fault Management The following SPECTRUM features provide rapid fault detection and are described in this chapter: • Rollup Condition Colors • Device Condition Colors • Audible Alarms This chapter also describes how to access SPECTRUM views to pinpoint a fault’s location using the following methods: • Point and Click on an icon’s double-click zones • Navigate In feature • Icon Subviews menu selections A Fault Isolation Flow Chart (Figure 43) is included as an example for using the detection and pinpointing features as described in this chapter to isolate a fault. This flow chart is not intended to cover all circumstances and fault conditions that may be experienced within your network but is for example only. The following applications, views, and navigational features can be useful in diagnosing network faults. • • • • • • Alarm Views Navigate In Chassis Device View Device Topology View MAC Address Locator Tool Performance Views. How To Manage Your Network with SPECTRUM Page 111 Figure 43: Fault Isolation Flow Chart Rollup Condition Alarm Generated LAN/Landscape Icon indicates a rollup condition. Open the LAN/Landscape Alarm Manager to identify the device, read the probable cause and recommended Use Navigate In to access the LAN Topology containing the faulty Yes Is the alarm significant enough to Is the fault generating a condition color on the No Yes Acknowledge the alarm to Open the Alarm Manager for the device and read the probable cause and recommended action for the spe- No Continued How To Manage Your Network with SPECTRUM Page 112 Continued Open the Device View to identify the faulty port or module managed by the device. Refer to No Is there a rollup condition color on any of the LANs connected to the device? Yes Access the Device Topology View or open the LAN Topology View to identify the faulty device. Access the Alarm Manager for the device generating the alarm and read the prob- Open the port or module performance view. Refer to the Per- Is the fault caused by a device connected to a port (e.g., excessive No Access the Alarm Manager for the device generating the alarm and read the probable cause and recommended action. Refer to the Alarm Manager section. Yes Use the source address table, MALT, RMON, or a LAN ana- How To Manage Your Network with SPECTRUM Page 113 Fault Detection Faults are detected through traps received from the device or configured through SPECTRUM. These traps are used to generate rollup conditions, device conditions, alarms, and event messages for any active port and any device. Condition Colors Condition colors are displayed on an icon to reflect the status of the model represented by the icon. The colors appear in the background of the label that indicates the kind of device or other network entity being modeled, as shown in Figure 44. If an event causes the condition color to change, the event will be registered in the Event Log. The Alarm Manager will indicate the Symptom/Probable Cause and recommended actions for that event. Figure 44: Condition Colors Device Condition Colors Blue - No contact with device has yet been established. Green - Contact established and condition is normal. Yellow - First level of marginal operation or duplicate IP address. HubCSIEMME Orange - Device is functioning but is unresponsive to management requests. Red - Contact with this device has been lost. Gray - Condition suppressed; device cannot be reached due to a known error condition that exists on another device. How To Manage Your Network with SPECTRUM Page 114 Blue (Initial) If SPECTRUM has not made contact with the device since the model was created, the condition color will be blue. If this condition continues, you can “ping” the device (i.e., send a Packet Internet Groper request) to see whether it is alive as follows: • Highlight the icon and select Ping from the Icon Subviews > Utilities menu or click the Ping toolbar icon. The following message will appear on your SpectroGRAPH console indicating that pinging is in progress. Pinging 132.177.118.24 PING 132.177.118.24 64 data bytes Green (Normal) If SPECTRUM has made contact with the device and operation is normal, the condition color will be green. Note: Note: Location fault isolation does not work exactly the same as container fault isolation. If a site (Location) contains devices with gray conditions, the site is still shown with a green condition. However, if a container model (LAN for example) contained these same devices, the container would show a gray condition. Site models generally represent a building or location which give an idea as to what models (devices) are inside that location, but the location itself (its condition) is not reflected in the same way that LAN models are reflected. For Site models, the Rollup condition is displayed. LAN models have both Rollup condition and model condition displayed. Yellow (Minor Alarm) If a situation has occurred that does not impact the overall network operation, but SPECTRUM has received information, the condition color will be yellow. How To Manage Your Network with SPECTRUM Page 115 Example: A module within this hub has been removed. Example: An IP (Internet Protocol) address or physical (Ethernet) address assigned to this model has already been assigned to another model. Orange (Major Alarm) If SPECTRUM cannot retrieve management information from a device, but the devices connected to this device have a normal condition (green), then the condition color will be orange. Example: A firmware problem exists in the device. Red (Lost Contact) If SPECTRUM detects a complete failure of the device or has lost contact with the device, the condition color will be red. Gray (Unknown) If the device status is unknown, alarms for that device will be suppressed, and the condition color will be gray. Example: If contact with a particular device is lost due to network problems on an upstream device (i.e., between the device and SpectroSERVER), the actual condition of the device cannot be determined and its condition color will be gray. Condition color for any devices connected downstream from this device will also turn gray. Rollup Condition Colors When a fault occurs with a device, a trap is generated by the device or by SPECTRUM for network management purposes. SPECTRUM receives the trap and generates a device condition color which indicates the status of the device. The importance of the device to your network determines the significance value you have placed on the condition. How To Manage Your Network with SPECTRUM Page 116 Figure 45 shows the location of the rollup condition color and defines the relationship between the color and the severity. Figure 45: Rollup Condition Colors Yellow - Information Alarm Orange - Minor System or Subsystem Failure Red - Major System or Subsystem Failure For example, you would want to associate a high significance number with a red condition color for a device that is critical to network performance; then, if the device failed, the red condition color would roll up to the model that contains the device, thus indicating that a critical device or devices have failed within that model. Conditions roll up to the next level in the hierarchy as determined by the significance level of the alarm and by the threshold level set for the next model in the hierarchy. For details on configuring rollup condition colors, thresholds, and significance levels, see Configuring for Fault Management on Page 55. How Is a Rollup Condition Detected? Rollup condition colors (Figure 46 and Figure 47) are used to indicate that a fault has been detected as described in the following example: How To Manage Your Network with SPECTRUM Page 117 Figure 46: Example Rollup Condition File View Tools Bookmarks Help The Red rollup color indicates that a critical system or subsystem failure has been detected within this landscape net- Model Name 4 1 7 Landscape Major alarm - rollup condition color is red. File View Tools Bookmarks Help IPClassB LAN_802_3 Major alarm - rollup condition color is red. File View Tools Bookmarks Help Automapped_LAN Bridge LAN_802_3 Failed device generates a red condition color. How To Manage Your Network with SPECTRUM The rollup threshold has been set for this critical LAN to rollup a major system or subsystem failure condition from the models it This critical device has failed and generated an alarm. The significance level assigned to this device will cause a rollup condition based on the threshold Page 118 Figure 47: Example Rollup Condition Colors Information Alarm Rollup Condition Color is Yellow Model Name 4 1 7 Landscape With a yellow rollup threshold set to 3 and the sum of the significance levels for the “Contained” models equal to 3, the rollup condition color will be yellow. Minor Alarm Rollup Condition Color is Orange LAN 802.3 Device Failure Device Condition Color is Red With an orange rollup threshold set to 3 and the sum of the significance levels for the “Contained” devices equal to 4, the rollup condition color will be orange. A noncritical device with a significance level of 4. HubCSIEMME Audible Alarms When new alarms occur, an audible sound indicates which severity of alarms have entered the system. The default sound is a voice stating the severity, e.g., “Critical Alarm,” or “Minor Alarm.” You can customize this audible sound in the Enterprise Alarm Manager (see Enterprise Alarm Manager (Page 120) Preferences dialog box using the Notification tabbed page. To learn how to customize audible alarms, refer to the Enterprise Alarm Manager User’s Guide. If you have the EAM view for your network open and minimized (Figure 48), an audible alarm will sound to indicate an alarm has been How To Manage Your Network with SPECTRUM Page 119 generated. If Auto Raise is selected from the Options menu, the EAM will open automatically when an alarm is generated. Figure 48: Minimized Alarm View Fault Isolation Fault detection is the first step in fault isolation. Utilizing the proactive fault management features described in the previous section, SPECTRUM detects that a fault has occurred and generates an alarm. Once the fault has been detected, you can use the pinpointing features of SPECTRUM to navigate to the view that identifies the specific network, device, channel, or port that generated the alarm. Enterprise Alarm Manager Enterprise Alarm Manager (EAM) views exist for the whole network, LAN icons, and devices. It is recommended that you keep the views for models essential to your network open and minimized at all times. The Enterprise Alarm Manager provides the Symptom/Probable Cause of the alarm, suggests actions to correct the error, and allows you to assign a repair person to the task. Accessing the Enterprise Alarm Manager To access the EAM, choose Tools > Alarm Manager, or, to view alarms on a specific model, highlight the applicable model icon and select Model Alarms from the Icon Subviews menu. How To Manage Your Network with SPECTRUM Page 120 SPECTRUM provides a feature called Auto Raise which automatically opens the EAM if it has been minimized on your work station. This feature is enabled from the EAM Options menu (Figure 49). Figure 49: Enterprise Alarm Manager Tool Bar Buttons (provide the same functions as the menus) Menu selections Alarm Information Panel Enterprise Alarm Manager: Main File View Model Alarms Troubleshooter Options Help Filter Sort Order Select All Ack Unack Clear Assign Unassign Ping Telnet MIB Tools What’sThis? Hints System Probable Cause Events* History* Alarm Status Trouble TicketID Location Device Notes 111.222.3.4. DIFFERENT TYPE MODEL GnSNMPDev SYMPTOMS: Device may be missing some device-specific intelligence. PROBABLE CAUSES: Severity Date/Time Critical Critical Critical Major Major Major Minor Minor Minor Model Name Fri 12 May 2000 15:11:08 Wed 10 May 2000 15:11:08 Wed 3 May 200012:59:33 Sat 29 Apr 2000 16:28:34 Wed Apr 12 2000 17:00:00 Mon 15 May 200008:48:11 Thu 20 Apr 2000 11:34:23 Wed 19 Apr 2000 12:34:21 Wed 12 Apr 2000 17:00:00 Search golem golem praetorius praetorius Administrator AlarmMgmt 111.222.3.4 golem praetorius Network Address Vendor Name 111.222.3.4 Cabletron Systems Unknown Model Type GnSNMPDev Prev Shown Next Displayed 9 of 9 Filtered by: Severity Critical: 3 Model Class Major: 3 Minor: 3 Total: 6 Servers Alarm List How To Manage Your Network with SPECTRUM Alarm Condition Panel (totals of each severity) Page 121 Using the Enterprise Alarm Manager Once an alarm has been generated, use the Enterprise Alarm Manager to isolate the fault as follows: 1 If multiple alarms exist, click on an alarm from the list within the view (Figure 50) to display that model’s alarm information and its icon in the Alarm Information panel. The icon provides the double-click zones and Icon Subviews menu selections for that model. Figure 50: Severity Critical Critical Critical Major Major Major Minor Minor Minor 2 Date/Time Fri 12 May 2000 15:11:08 Wed 10 May 2000 15:11:08 Wed 3 May 200012:59:33 Sat 29 Apr 2000 16:28:34 Wed Apr 12 2000 17:00:00 Mon 15 May 200008:48:11 Thu 20 Apr 2000 11:34:23 Wed 19 Apr 2000 12:34:21 Wed 12 Apr 2000 17:00:00 Selecting the Alarm to be Isolated Model Name golem golem praetorius praetorius Administrator AlarmMgmt 111.222.3.4 golem praetorius Network Address Vendor Name 111.222.3.4 Model Class Cabletron Systems Unknown Model Type GnSNMPDev In the Alarm Information panel, click on the Probable Cause tab (Figure 51) to read a complete description for that alarm. This view identifies the symptom, suggests a probable cause, recommends an action, and provides the name of an assigned troubleshooter. How To Manage Your Network with SPECTRUM Page 122 Figure 51: Reading the Probable Cause for the Alarm System Probable Cause Events* History* Alarm Status Trouble TicketID Location Device Notes 111.222.3.4. DIFFERENT TYPE MODEL GnSNMPDev SYMPTOMS: Device may be missing some device-specific intelligence. PROBABLE CAUSES: everity Critical Critical Critical Major Major Major Minor Minor Minor 3 Date/Time Fri 12 May 2000 15:11:08 Wed 10 May 2000 15:11:08 Wed 3 May 200012:59:33 Sat 29 Apr 2000 16:28:34 Wed Apr 12 2000 17:00:00 Mon 15 May 200008:48:11 Thu 20 Apr 2000 11:34:23 Wed 19 Apr 2000 12:34:21 Wed 12 Apr 2000 17:00:00 Model Name golem golem praetorius praetorius Administrator AlarmMgmt 111.222.3.4 golem praetorius Network Address Vendor Name 111.222.3.4 Model Class Cabletron Systems Unknown Model Type GnSNMPDev Depending on the type of fault and size of your network, you may want to use one of the following pinpointing options to isolate the fault. • To see the impact the fault has on the network and other devices in the network before proceeding directly to the fault, refer to the Navigate In section for details. • If the fault is related to an individual port, you can go directly to the Chassis Device View to identify the port and view port-specific performance information. Refer to Chassis Device View on Page 125 for more details. Navigate In and Point and Click There are two ways to navigate through the topology hierarchy to locate the device that is generating the alarm. One way is to navigate through the applicable LANs using the down arrows (point and click), the other is to use the Navigate In feature. Use the Point and click feature as shown in Figure 52: How To Manage Your Network with SPECTRUM Page 123 Figure 52: Point and Click SpectroGRAPH File View Tools Bookmarks Help Model Name 4 1 7 Landscape Down Arrow Double-click The Down Arrow Double-click the down arrow on any model that contains a faulty network or device. Use the Navigate In feature as follows: 1 Highlight the Landscape or LAN icon in a Topology view (Figure 53) then select Navigate > Navigate In from the Icon Subviews menu. 2 Double-click the IP Address of type LAN_802_3 showing the rollup condition to access its Navigate In list. Repeat this process until you have opened the list containing the faulty device indicated by the rollup condition colors. The LAN containing the device is listed at the top. 3 From this Navigate In list you may do one of the following: • Double-click the LAN containing the faulty device to access the LAN topology view. This view will show the network affected by the faulty device. The faulty device and any affected devices will be indicated by rollup and device condition colors. Refer to Fault Detection on Page 114 for details on condition colors. • Double-click the faulty device listed to open the Device Topology view for that device (see Device Topology View on Page 127). How To Manage Your Network with SPECTRUM Page 124 Figure 53: Navigate In View Step 1 Highlight the Landscape Icon with the middle mouse button and select Navigate and Navigate In. Step 2 Double-click opens the Navigate In view for that LAN indicated by the arrow. SpectroGRAPH: Topology: Universe File View Tools Help Bookmarks Model Name 4 1 7 Landscape IP Address of IP Address of IP Address of IP Address of IP Address of IP Address of IP Address of type IP Address of IP Address of IP Address of type IP Address of type LAN Containing the devices and LANs listed below Step 3 Double-click opens the Device Topology view for that device. Chassis Device View Use the Chassis Device view (Figure 54) to identify the faulty module or port. The Chassis Device view will provide you with port and module information at a glance. Refer to the management module guide for the particular device for Device view information. How To Manage Your Network with SPECTRUM Page 125 Accessing the Chassis Device View Access the Chassis Device view using one of the following methods: • Click on the Device View button on the device icon. • Highlight the device icon and select Device View/Chassis from the Icon Subviews menu. Figure 54: Example Chassis Device View Logical Modules Module Channel or Isolation Station Module Number Module Type Port Number Port Status Frame Rate 2 Network A 1 FOM-22 1 SEG Pkts 0 2 LNK Pkts 0 3 LNK Pkts 0 4 LNK Pkts 0 5 SEG Pkts 0 6 ON Pkts 487 7 LNK Pkts 0 8 LNK Pkts 0 9 SEG Pkts 0 EMM-E6 10 SEG Pkts 0 11 ON Pkts 147 Bridge Port Bridge Status Bridging A FWD B FWD C FWD D1 FWD D2 OFF Repeaters A UNLOCKED B UNLOCKED C UNLOCKED Redundant Port Repeater Ports Network Port Security FDDI 12 SEG Pkts 0 How To Manage Your Network with SPECTRUM Page 126 Device Topology View The Device Topology view for a device provides you with port status and access to specific port performance and error views. Accessing the Device Topology View There are several ways to access the Device Topology view: • Double-click on the device in the Navigate In list. • Double-click on the down arrow on any device icon. • Highlight the icon and select DevTop view from the Icon Subviews menu. Using the Device Topology View The Device Topology view can be used to isolate the fault to the port level. For example, if a port on a device is bad the Device Topology view (Figure 55) for the device will show the devices connected to a particular port or interface. If the device connected to the port is gray (suppressed), the port is bad and contact to the device connected to the port is lost. How To Manage Your Network with SPECTRUM Page 127 Figure 55: Using the Device Topology View Click the module to display its port SpectroGRAPH : Device Topology : 111.222.333.4 The condition color status label is gray indicating a suppressed network down stream of the device this Device Topology view is for. 1 0 I R The condition color status label for the icon that A Port Connections ON B ON C ON ETHERNET ETHERNET ETHERNET 0:0:1D:19:AC:52 0:0:1D:19:AC:53 0:0:1D:19:AC:54 0:0:0:0 0:0:0:0 111.222.333.4 Interface Icons Performance View The Performance view for a device model provides you with the following general performance information for isolating faults: • • • • Multi-Attribute Line Graph Error Breakdown Pie Charts (some management modules) Frame Breakdown Pie Charts (some management modules) Port Frame Size Breakdown Pie Charts (some management modules) How To Manage Your Network with SPECTRUM Page 128 • Protocol Breakdown Pie Charts (some management modules) Accessing the Performance View Access the Performance view using one of the following methods: • Highlight the device icon and select Performance view from the Icon Subviews menu. • Double-click on the Performance View label on the device icon. • Highlight the port icons and select Performance view from the Icon Subviews menu. Using the Performance View The device, modules, and ports have individual Performance and Detail views. A Performance view summarizes statistics corresponding to the chosen primary application as color-coded entities, and gives a statistical and graphical breakdown for current, average, and peak values. What is in the Performance View? The Performance view provides statistical and graphical information on network, device, application, and port performance as follows: Multi-Attribute Line Graph The Multi-Attribute Line Graph will give you a quick indication of problems (spikes) with load, frame rate, collision rate, error rate, and channel packet errors, but for more detailed information, you must access the Detail Views with error and frame breakdown pie charts. The Multi-Attribute Line Graph (Figure 56) provides a general indication of device or application activity. The attributes displayed are pre-selected and use colors to represent different statistics. Buttons allow you to modify the statistical presentation of the Multi-Attribute Line Graph. For more information on the Multi-Attribute Line Graph, refer to SPECTRUM Views. Multi-Attribute Line Graphs also include three buttons described below.: How To Manage Your Network with SPECTRUM Page 129 Lin/Log This button toggles between a linear and logarithmic scale presentation of the graph. Scroll to Date-Time This button allows you to set the viewing area of the graph to begin at a specified date and time. Change Time Scale This button allows you to specify the Y axis time scale for the graph. Figure 56: Multi-Attribute Graph Graph Properties Scroll to Date-Time Detail Pie Charts Color-coded pie charts, accessed by clicking the Detail button, present breakdowns of application statistics (Figure 57). Each statistic is presented as a total amount since the device was initialized, and as a percentage of overall traffic. Pie charts show the current (total) attribute values, the accumulated attribute values (accumulated from the time the Accum button is clicked), or the difference (delta) between the current and previous attribute values for each related model, depending on which How To Manage Your Network with SPECTRUM Page 130 mode is selected. The models presented are determined by the relation selected when the field is created or modified. When Delta is selected, this pie chart can be used to compare the difference between the previously polled value and the current value of an attribute from several models, and shows each model’s contribution (percentage) to the total of the delta values. When Total is selected, this pie chart provides the current attribute values for each of the related models and each model’s contribution (percentage) to the total of values. When Accum is selected, this pie chart provides the accumulated attribute values for all related models since the Accum button was clicked and each model’s contribution (percentage) to the total accumulation of values. To restart the counter, select the Clear button. Values continue to accumulate until you clear them or another presentation mode is chosen. Figure 57: Pie Chart Packet Breakdown Delivered 1515660 73.85% Forwarded 67818 3.30% Transmitted 22398 1.09% Errors 9996 0.49% Discards 436528 21.27% Total 2052400 Total Delta How To Manage Your Network with SPECTRUM Accum Clear Page 131 Error and Frame Breakdown Pie Charts Error Breakdown pie charts can be useful for isolating problems. The total errors will give you a general indication of performance, but the accumulated errors can show you what is happening at the moment. To see the accumulated errors follow these steps: 1 Access the Error or Frame Breakdown pie charts from the performance view. 2 Click on the Accum button to start the accumulated count of errors or frames. 3 Click on the Clear button to clear the errors and start at zero. 4 Wait for about 60 seconds and then look at the accumulated errors (which will be the current errors from the time of zeroing). Troubleshooting with the Performance View When you begin to look at error rates and other traffic information, keep in mind that most of the statistics provided have to do with usage levels, packet formation problems, access problems, or network design problems. Knowing which statistics indicate which sorts of problems can help to narrow the search for your network troubles to one general area. For example, high numbers of active users and high utilization rates may explain why your net is experiencing high levels of collisions and may indicate a need for bridging or routing to control traffic levels. Protocol and frame size statistics can help you identify the portions of your network that might best lend themselves to isolation. You may want to group users by protocol, or provide a group of users with an appropriate server if they are consistently using a server outside their network segment. An evaluation of frame size statistics may tell you where and/or when big file transfers are clogging up the cable, or where management packets make up the bulk of traffic (indicating a need for a DLM or RMON device). Error Rates are a function of frame rates. A high number of errors may actually be a very small percentage of overall traffic, resulting in no significant network disturbance. Looking at error rates as a function of How To Manage Your Network with SPECTRUM Page 132 traffic levels can be a good way to detect errors that are occurring simply because your network is getting busier. The following descriptions will help to isolate the problem: Collisions (and sometimes runts) are a natural by-product of a busy network; if you notice the levels of these two errors gradually increasing over time, or suddenly increasing at certain times of the day (accompanied by an increase in traffic levels), you may want to consider using bridges or routers for traffic management. High collision counts can also indicate other problems, such as a data loop (redundant active network connections) or a bad NIC (someone transmitting without listening first). CRC and Alignment errors can indicate some MAC layer/packet formation problem — either packets are not being properly formed in units of 8 bits, or the element responsible for either the transmit or receive CRC calculation is malfunctioning, causing perfectly good packets to be discarded as errors. These error types can also indicate some transmission medium problem — noisy or otherwise unreliable cable that is corrupting the packet as it is transmitted. Remember, according to the error priority schemes employed by Cabletron devices, packets counted as alignment errors may also have CRC errors; packets counted as CRC errors, however, do not contain truncated bytes. Out-of-Window (OOW) Collisions can either indicate that some portion of your network is out of spec — that is, the furthest distance between two nodes exceeds the 51.2 µs collision window— or that you have a bad transceiver which is transmitting without first listening for carrier sense (either intermittently or consistently). If your network begins to experience OOW collisions, use the port-level statistics screens to determine which nodes are recording these errors. Note that these nodes are not necessarily the ones which are the source of the problem: the only node that will record an OOW collision is the one that was transmitting for more than 51.2 µs before the collision occurred. If you have an established network which suddenly begins to experience OOW collisions, make sure the maximum propagation delay has not been violated and that your network is still in spec; be especially sure to check the distance between the node or nodes that are recording the errors and the nodes How To Manage Your Network with SPECTRUM Page 133 furthest away from those, and between the nodes recording the errors and the newest nodes. If you find that your network does meet the propagation delay spec, chances are you have a node which is transmitting without first listening for carrier sense. This can be a more difficult problem to track, since the malfunctioning node may only fail to listen occasionally, and since not all of its failures to listen will result in OOW collisions — some may simply result in collisions (if the 51.2 µs window has not closed), and some will get through fine (if no one else happens to be transmitting). The best way to track this kind of problem is to gradually bridge your network until you have isolated the OOW collisions on one segment, then check each node on that segment until you find the culprit. You might also look for a segment which is experiencing both OOW collisions and a sudden increase in regular collisions. Runts and Giants can both be caused by packet formation problems or by corrupted packets (packets being truncated or lengthened, or frame size bits being corrupted); as stated above, runts can also result from collisions. Most frequently, giant packets are the result of a jabbering node — one that just transmits continuously, either for long periods of time or only occasionally — and indicate a bad transmitter on a NIC. Framesize Statistics Aside from runt and giant packets, which are caused by errors, the sizes of the packets being transmitted on your network can give you an indication of what kind of traffic your network is seeing — many large packets can mean a lot of file transfers; small packets can mean a lot of management traffic — and can also give you an indication of the efficiency of your network — too many small packets indicates an inefficient use of bandwidth (too much overhead for too little data); too many large packets can slow your network down. You can use the frame size data to effectively bridge and/or route your network for a more evenly balanced flow of traffic. % Load, Bytes, and Packets provide a good overall sense of network usage. You can use these statistics to pinpoint peak usage times or segments of your network where usage is particularly heavy, which in turn can alert you to upcoming needs to expand, bridge, or route your network. One thing to note about packet and byte counts: on the newer How To Manage Your Network with SPECTRUM Page 134 Cabletron management devices (such as the EMME and the MRXI-24), packet and byte counts include errors; on older devices (such as the IRM2 or IRM3), they do not. High Broadcast packet counts can indicate the presence of a broadcast storm — an unending flood of ARP and RARP packets transmitted by a defective or improperly configured bridge or router. These storms are a serious matter and can completely shut down your network by monopolizing the bandwidth. Packet Size data can provide valuable information about how efficiently your network bandwidth is being used — too many small packets indicates a high overhead cost for the data being transmitted; too many large packets can clog your network and hurt performance. Packet size statistics can also give you a general sense of the types of data being transmitted: many large packets may indicate large file transfers; many small packets may indicate high levels of management traffic. Many packet size-related problems can be solved by bridging or routing to isolate network segments with special needs. Protocol Statistics breakdowns can be helpful if you are planning to add bridges, routers, or gateways to your network; you may want to group users by protocol so traffic needn’t pass through multiple segments to reach a designated protocol server or to avoid any problems of incompatibility. Also, different protocols have different overhead costs and may tend to use either very large or very small packets, so protocol statistics may give you some insight into other network performance issues: You may discover, for example, that a protocol which tends to have large packets also receives heavy use at a particular time of day or on a particular network segment, causing a noticeable slowdown in network traffic. Such problems can be solved by isolating those users on their own bridge- or router-protected network segment. Protocol Type data can be very useful in planning for bridges, routers, gateways, and even additional servers; you may want to group and/or isolate users based on protocol, especially if users are sending packets beyond their own network segment to reach a specific protocol server on another segment or if there are any issues of incompatibility. In addition, different protocols have different overhead requirements and How To Manage Your Network with SPECTRUM Page 135 transmission characteristics, which may have different effects on overall network performance. Beyond the Performance View These views provide you with information on network, device, port activity and application performance to be used to isolate a fault to a particular network segment, module, port, and/or node. You may need to use packet analyzers or other hardware-related diagnostic tools to isolate to the packet level. Depending on the significance of the fault, you may decide to disconnect that port, module, or segment to keep the rest of the network operating or choose to acknowledge the alarm and continue operation. How To Manage Your Network with SPECTRUM Page 136 SPECTRUM Intelligence SPECTRUM comes with the patented Inductive Modeling Technology™ (IMT), which consists of a suite of intelligence circuits that work with the Virtual Network Machine to help configure, manage, and monitor your network. Each intelligence circuit is a group of functions that together perform a particular task. These circuits are referred to as intelligence or intelligence functions. Static Configuration of Device Models Many network devices are configured during their manufacturing process and are difficult to modify later. For SPECTRUM modeling purposes, these devices are considered to have a static configuration—i.e., once SPECTRUM intelligence models these devices, they are not reconfigured. SPECTRUM creates models for the ports, matching the types of port models to the types of ports on the device (such as T1 or Ethernet). For each port model that is created, an association is established between the port and the device via the “HASPART” relation and may be typically viewed with the device model’s Application view. When the model is destroyed, all of the port models associated with the device are also destroyed. Dynamic Configuration of Device Models Some network devices can be configured dynamically by removing boards and installing new boards without removing the device from the network. SPECTRUM intelligence provides for automatic modeling of these devices and their connections upon their creation and then performs verification and remodeling if necessary after each VNM polling cycle. Thus SPECTRUM continuously monitors and changes these models to match How To Manage Your Network with SPECTRUM Page 137 the actual device on the network. To view the configuration of the model, use the Device, Device Topology, or Application View for the model. Whenever a model is created, SpectroSERVER polls the device and creates an appropriate configuration, including the number, type, and order of modules and ports. For each device model created, a relation exists between that model and the parent device via the “HASPART” model type relation rule. This relation also exists between the boards and any ports on the boards. After each polling cycle, SpectroSERVER reexamines and, if necessary, changes the parent model and its related models to match changes to the device configuration. Whenever you add a new board to a dynamically configured device, SPECTRUM creates a model to represent that board and each of the ports on the board. If a board model is destroyed, all of the port models that form part of the board are also destroyed. The Pulled Board List Creating and destroying models is time consuming. When you remove a board from the device, SPECTRUM does not destroy the board model, but rather keeps a copy of the board model in a “pulled board list.” The board model’s “HASPART” relation to the hub model is removed. SPECTRUM reassociates this model to the device if you re-install the old board. If you add a new board to replace the old board, SPECTRUM associates the new board model to the device and the old board model is placed in the Lost and Found view. If you remove a board model from the pulled board list, the board model is destroyed from the Lost and Found view and no longer exists. Here is a general list of pulled board list attributes: Max_Pulled_Bd_Cnt The maximum number of models allowed to exist in the pulled board list. When the value is exceeded, the oldest model in the list is removed from the list. Pulled_Bd_Cnt The current number of models in the pulled board list. How To Manage Your Network with SPECTRUM Page 138 Pulled_Bd_List The Pulled_Bd_List is not readable by the user. It contains a list of board models that have been pulled from a dynamically configured device. When a board is re-installed in the device, SPECTRUM removes the board model from the Pulled_Bd_List. Attributes for the pulled board list are usually displayed in the Configuration view. If they are not present there, they can be added using the Generic Information Block (GIB) Editor. To change the maximum number of models allowed in the pulled board list, use the Update feature. GIB Editor features are described in the GIB Editor User’s Guide. Router Reconfiguration Events Router reconfiguration actually involves two separate processes: interface reconfiguration (which ensures the device’s interfaces are properly modeled), and device discovery (which ensures proper modeling of other devices, LANs, etc. that are connected to those interfaces). Depending on whether both or either one of these processes occurs, SPECTRUM generates one of the following events to help you keep track of the configuration changes: ROUTER_RECONFIG_EVENT (0x1001c) - This event is generated when a device is reconfigured and both interface reconfiguration and device discovery occur. INTERFACE_RECONFIG_EVENT (0x1001d) - This event is generated for a device whenever interface reconfiguration occurs. DEVICE_DISCOVERY_EVENT (0x1001e) - This event is generated for a device whenever device discovery occurs (i.e., connections off the device’s interfaces are being rediscovered). How To Manage Your Network with SPECTRUM Page 139 Condition vs. Rollup Condition SPECTRUM provides intelligence circuits that allow you to see changes in your network devices and their performance by simply glancing at the icons that represent them. The icons use color to indicate two different types of status: Condition and Rollup Condition. Condition reflects the contact and alarm status of the device model represented by the icon (see Table 7 on Page 143). Rollup Condition (Table 8 on Page 146) is the composite status of models that are “children” of the model represented by the icon. (Children models are related to parent models through the “collects” relation in the Topology hierarchy and through the “contains” relation in the Location hierarchy.) The Rollup Condition generally changes as you move up in the hierarchy, because each level reflects the blending of conditions from a greater number of individual models. The location of the Condition and Rollup Condition colors varies according to the type of icon, as shown in Figure 58 (Page 140). For device and topology (LAN) icons, Condition is displayed in the diagnostic doubleclick zone and the Rollup Condition is displayed in the down-arrow double-click zone for the icon. The circle at the base of location model icons displays either the Condition or the Rollup Condition, whichever is more critical. Figure 58: Condition and Rollup Condition Colors on Icons Condition Color Location Flashing Condition colors indicate a change in condition. Model Name Model Name Model Name Model Name Model Type Model Type Model Type Rollup Condition Color How To Manage Your Network with SPECTRUM Page 140 When a model goes into an alarm state (yellow, orange, or red) its condition indicator flashes (unless you have changed the default setting of SpectroGRAPH’s iconBlink resource, which is “True”—see Defining SPECTRUM Resources for more information on the iconBlink resource.). You can acknowledge the alarm, and stop it from flashing, by selecting Acknowledge from the model’s Icon Subviews menu. When an alarm automatically clears itself, it is logged in the Event view and the condition indicator does not flash. If you need it to flash to indicate that it has cleared an alarm, select Flash Green Enabled from the model’s Icon Subviews menu. The model will then flash green when the alarm automatically clears, making it obvious that the model transitioned from an alarm state to a normal state without any user interaction. Alarm acknowledgements roll up to container models just as other status changes do, but only if there is at least one other model with no alarms in the same container. Note that the rolled-up alarm on the container will only clear once you have acknowledged a sufficient number of models within the container to add up to the threshold for the indicated color. In other words, if your Red_Threshold is 10, and you only acknowledge one red model whose Value_When_Red is 7, the container will continue to flash. If you acknowledge a second red alarm, the flashing would stop. All external models have at least one polled attribute, which is read by SpectroSERVER, which then reads all polled attributes at the frequency specified for the polling interval. For information on adjusting or disabling polling, see Adjusting Management Capacity on Page 35. How To Manage Your Network with SPECTRUM Page 141 Attributes Determining Condition and Rollup Condition There is a unique set of attributes that are related to Condition and Rollup Condition. Their values are used in determining Condition and Rollup Condition for models: Condition The Condition attribute value reflects the contact status as well as any more specific alarm in effect for a device model. This value determines the Condition color on topology and location icons as explained in Table 7. How To Manage Your Network with SPECTRUM Page 142 Table 7: Contact Status vs. Condition Color Contact Status Condition Color Description Initial Initial BLUE Either the model has not yet established contact with the device it represents, or it represents an insignificant device with which contact has been lost . Established Normal GREEN The model has successfully established contact with the device it represents, and the device is functioning normally. Established Minor YELLOW This is the first level of marginal operation. Either the model has successfully established contact with the device it represents, but there is an abnormal condition that does not affect overall network operation (e.g., a module has been removed from the device), or the IP address assigned to this model was already assigned to another model Lost Major ORANGE This is the second level of marginal operation. The management agent on the device has failed and is not responding to any communication from SPECTRUM, but the device is still relaying data to its downstream neighbors. This condition occurs only on data-relay type devices such as hubs and is typical of a firmware failure. How To Manage Your Network with SPECTRUM Page 143 Contact Status Condition Color Description Lost Critical RED This condition indicates a total failure of the device and requires management’s attention to repair or replace it. Lost Suppressed (Unknown) GRAY Contact has been lost with this device AND with a device that is upstream from this device (i.e., between this device and SPECTRUM), thus the actual condition of this device is unknown and alarms for the model representing it are suppressed. The gray condition color will also be displayed for all models that are downstream from this device. All adjacent (directly connected) models, whether upstream or downstream, will have a contact status of “Lost.” Lost Maintenance BROWN SPECTRUM cannot contact the device because the model has been placed into maintenance mode (see Device Criticality on Page 108). Condition_Value The Condition Value is the numeric value, representing a model’s overall condition. This value is passed up to a parent model and included in the Composite Condition. The model’s overall condition is either the Condition or the Rollup Condition, whichever is more severe. (The Condition Value indirectly receives the value of the (administrator defined) How To Manage Your Network with SPECTRUM Page 144 Significance Level corresponding to a model’s overall condition.) Composite_Condition The sum of all Condition Values for models that are contained by a location model or collected by a topology model. Rollup Condition SPECTRUM computes the Rollup Condition attribute value using the (administrator defined) Rollup Threshold and Composite Condition. The resulting attribute value determines the color that will be displayed to indicate the overall condition of models that are contained by a location model or collected by a topology model. The Rollup Condition for a model is displayed as a color on an icon, or the dot in a Location View icon. If the iconBlink resource is set to “True,” the condition color flashes. Refer to Defining SPECTRUM Resources for more information on the iconBlink resource. The possible colors are described in Table 2. How To Manage Your Network with SPECTRUM Page 145 Table 8: Color Rollup Condition Colors Meaning Green The value of the Composite Condition attribute for this model’s children is less than the Yellow (Rollup Condition) Threshold. Yellow The Composite Condition Value for this model’s children equals or exceeds the Yellow Threshold but is less than the Orange Threshold. Orange The Composite Condition Value for this model’s children equals or exceeds the Orange Threshold but is less than the Red Threshold. Red The Composite Condition Value for this model’s children equals or exceeds the Red Threshold. Condition and Rollup Condition Sensitivity The following two attributes serve as parameters that can be used to emphasize or diminish the impact on Rollup Condition from the Condition Values of particular models. By adjusting these attribute values you can control when the Rollup Condition color changes. These attributes are typically found in the Information (GIB) View for a model. The values for these attributes can be changed by the network administrator by updating the value shown in each field. Rollup Thresholds The Rollup Thresholds are the three attributes that control the Rollup Condition color (Yellow, Orange, and Red) for a model. Rollup Thresholds are administrator-defined values that are entered on a model-by-model basis. The Composite Condition value received from the model’s children is compared with these attributes to determine a Rollup Condition Color. For example, if a model’s Composite Condition value is equal to or greater How To Manage Your Network with SPECTRUM Page 146 than its Orange Threshold (but less than its Red Threshold) the model’s Rollup Condition Color is Orange. The default values for Rollup Thresholds are: Yellow Threshold=3 Orange Threshold=6 Red Threshold=10 Significance Level The Significance Level attributes define the numeric value for Yellow, Orange, and Red Conditions and Rollup Conditions. Like Rollup Thresholds, Significance Level values are administrator-defined values, entered on a model-by-model basis. Significance Level field labels begin with the words “value when.” The default Significance Level values are: Value_When_Yellow=1 Value_When_Orange=3 Value_When_Red=7 Typically, models (devices) are divided into two classes, “significant” or “insignificant.” A significant device is any device that requires an administrator’s attention for proper network operation. Insignificant devices are typically endpoint devices, such as a PC or workstation. Insignificant devices usually toggle between Green and Blue (Condition Value = 0). Significant models can be made insignificant by changing their Value_When_Red attribute value to 0 (zero). How To Manage Your Network with SPECTRUM Page 147 Rollup Condition Flow Figure 59 illustrates the Rollup Condition process. Read the flow from bottom to top, but keep in mind that it shows a single path in the propagation of Rollup Condition and that there may be many children passing Condition Values to a parent model. Figure 59: Rollup Condition Process To Parent model Determine the Rollup Condition. Compare the Composite Condition to Rollup Thresholds. Set the Rollup Condition Color on the icon. Obtain the Significance Level for the current Condition of the device. Set the Condition Value to the Significance Level. Yes Use the Condition Values from children to calculate the Composite Condition Determine the Condition for the device. Set the Condition Color on the icon. Condition more severe than No Obtain the Significance level for the current Rollup Condition. Condition Values from other children Determine the Rollup Condition. Compare the Composite Condition to Rollup Thresholds. Set the Rollup Condition Color on the icon. Obtain the Significance Level for the current Condition of the device. Set the Condition Value to the Significance Level. Yes Use the Condition Values from children to calculate the Composite Condition Determine the Condition for the device. Set the Condition Color on the icon. Condition more severe than No Obtain the Significance level for the current Rollup Condition. Condition Values from children of this model How To Manage Your Network with SPECTRUM Page 148 Example of Rollup Condition Propagation Figure 60 depicts the process of rollup condition in a Topology hierarchy. Figure 60: Rollup Condition Process in a Topology Hierarchy 6. Determine the Rollup Condition. Compare the Composite Condition to Rollup Thresholds. Yellow Threshold= 3 Orange Threshold=6 Red Threshold=10 7. Set the Rollup Condition color to Orange on the icon (Composite Condition > Orange Threshold but < Red Threshold). Rollup Condition: OVERALL_ NET Condition: Overall Net Orange Network 4. Apply the Significance level for the current Rollup Condition to set the Condition Value. (7) No Condition more severe than Green 1 5. Calculate the Composite Condition (sum of the Condition Values from children = 8). Condition Values: 8= 7+1+ Rollup Condition (Red) is Yes Rollup Conditions: Yellow Red FINANCE ENGRG TWLAN TW LAN Engrg LAN Finance LAN LAN_802_3 LAN_802_3 LAN_802_3 Conditions: Green 3. Set the Rollup Condition color to Red on the icon (Composite Condition > Red Threshold) Green (No Color) Green Green Condition Values: 2. Determine the Rollup Condition Compare the Composite Condition to Rollup Thresholds. Yellow Threshold= 3 Orange Threshold=6 Red Threshold=10 1. Calculate the Composite Condition (sum of the Condition Values from children = 11). 11 = 7 + 0 + 1 + Rollup Conditions: Red Green (No Color) Conditions: Green Green How To Manage Your Network with SPECTRUM Yellow Orange Green Green Green (No Color) Green Page 149 This example depicts two layers of a Topology hierarchy. The example assumes the use of default Rollup Thresholds and Significance Levels. The lowest view shown in the figure contains five devices: two hubs, a router, and two end-point devices. These are collected by TWLAN of type 802_3_LAN. On the same level as TWLAN, are two other LANs: FINANCE of type 802_3_LAN and ENGRG of type 802_3_LAN. The network group model named OVERALL_NET collects three LAN_802_3 network group models. In the example the following Rollup Conditions and Conditions, at lower levels, determine the top-level Rollup Condition for the network group model named OVERALL_NET: Devices collected by the network group model named TWLAN. Hub#1 Condition = Green Rollup Condition = Red Condition Value = 7 Hub#2 Condition = Green Rollup Condition = Orange Condition Value = 3 Router#1 Condition = Green Rollup Condition = Yellow Condition Value = 1 End-Point Devices PC#1 & PC#2 Condition = Green Rollup Condition = Green Condition Value = 0 Network group model named FINANCE. Condition = Green Rollup Condition = Green Condition Value = 0 How To Manage Your Network with SPECTRUM Page 150 Network group model named ENGRG. Condition = Green Rollup Condition =Yellow Condition Value = 1 Every model within a Topology View receives its Collects relation from the model that it is collected by. Therefore, all models contribute to the Rollup Condition of the network group model. The following steps provide a detailed flow of condition values that contribute to the Rollup Condition for the model named OVERALL _NET shown in the example of Figure 60. 1 Determine the Composite Condition for the TWLAN network group model. Composite Condition is the sum of the collected models’ Condition Values. In this case, the device models have Condition Values of: “Orange” Condition hub model has a Condition Value of 3. “Red” Condition hub model has a Condition Value of 7. “Green” Condition PC#1 model has a Condition Value of 0. “Yellow” Condition router model has a Condition Value of 1. “Green” Condition PC#2 model has a Condition Value of 0. Therefore, for the TWLAN model: Composite Condition = (3 + 7 + 0 + 1 + 0) = 11 2 Determine the Rollup Condition for TWLAN. In this case: Composite Value = 11 TWLAN Yellow Threshold = 3 TWLAN Orange Threshold = 6 TWLAN Red Threshold = 10 Composite Value Š Red Threshold Therefore: Rollup Condition for TWLAN = Red 3 Assign Significance Levels to TWLAN Condition and Rollup Condition. In this case, Significance Levels are: Value When Yellow = 1 How To Manage Your Network with SPECTRUM Page 151 Value When Orange = 3 Value When Red = 7 Therefore: Rollup Condition = Red Condition = 7 4 Set Condition Value for TWLAN model. In this case: Rollup Condition more severe than Condition Therefore: Condition Value = Rollup Condition Significance Level = 7 The three network models, TWLAN, ENGRG, and FINANCE pass their Condition Values up to the network group model named OVERALL_NET. Changes in the Condition or Rollup Condition for the device models at lower levels can produce changes in the topology models further up in the Topology hierarchy. The Rollup Conditions for these three networks produce the following Rollup Condition for OVERALL_NET. 5 Determine the Composite Condition for the OVERALL_NET network group model. Composite Condition is the sum of the collected models’ Condition Values. In this case, the network models have Condition Values of: Red Condition TWLAN model has a Condition Value of 7. Green Condition FINANCE model has a Condition Value of 0. Yellow Condition ENGRG model has a Condition Value of 1. Therefore, for the TWLAN model: Composite Condition = (7 + 0 + 1) = 8 6 Determine the Rollup Condition for OVERALL_NET. In this case: Composite Value = 8 Yellow Threshold = 3 Orange Threshold = 6 Red Threshold = 10 Composite Value Š Orange Threshold, but < Red Threshold How To Manage Your Network with SPECTRUM Page 152 Therefore: Rollup Condition for OVERALL_NET = Orange Organization Models Unlike device, location, and network group icons that indicate status using colors, icons that are unique to the Organization hierarchy do not provide any visual status indication. However, device, location, and network group icons can appear in the Organization hierarchy, and when they do, they reflect the same Condition and Rollup Condition as that displayed when they appear in the location and topology hierarchies. Fault Isolation Fault Management is one of the key requirements of network management. A fault is different from an error in that it is an abnormal condition whose repair requires management attention. The problem conditions could be due to bad firmware, bad network, or bad hardware. Each of these problems requires a different response from the network manager. Thus the goal is to determine the exact location of the faults and to get the attention of the network administrators as quickly as possible. SPECTRUM intelligence has the capability of isolating a network problem to the most probable faulty component. To speed up the fault isolation and to reduce unnecessary traffic, two actions occur: Are-You-Down Action Upon losing contact with the device it represents, a model sends the Are-You-Down action to all of its neighbors to determine its own condition. If all the neighbors return a response of TRUE, the model’s condition color will turn gray (meaning “my device might be down, but it is impossible to tell because all the neighbors are down.”). However, if any of the neighbors return a response of FALSE, the model’s condition color will turn red (meaning “my device must be down, because one of the neighbors is up.”). Are-You-Up Action How To Manage Your Network with SPECTRUM Page 153 Upon re-establishing contact with the device it represents, a model sends this action to its neighbors to speed up the fault isolation. Upon receiving this action, each neighbor will return TRUE if it has an established contact status. If the model’s contact status is lost, and the next-time-to-poll is more than 60 seconds, then the model pings the device for quicker fault isolation. Every time a model’s status changes, or the information available to SPECTRUM changes, a new assessment occurs. SPECTRUM intelligence keeps the SpectroGRAPH presentation as up to date and accurate as possible, but it depends on correct modeling to accurately assess the contact status and determine device failures on the network. Correct modeling includes placing your VNM model in proper relation to the other models that represent your network (i.e., it must have a resolved connection in the Device Topology view of a model that represents a device to which the VNM host is actually connected). When the VNM model is properly connected and SPECTRUM loses contact with a model, the icon representing that model will display a condition color of Gray, Orange, or Red, which helps the network administrators to locate the faults immediately. How Model Category Affects Contact Status Each fault is associated with a particular condition, which is represented by a particular color that displays on the icon representing the model where the fault occurs. The condition color reflects both the contact status and the alarm status of the model, as was shown in Table 7. However, the contact status and condition color asserted for a model also depend upon which of the following categories a model belongs to. Table 9 summarizes how the categories to which a model and its neighbors belong will influence its contact status and condition color. Significant Device Models Any device that requires an administrator’s attention for the smooth operation of the network is called a significant device. To change an insignificant model into a significant model change the value of the attribute Value_When_Red (0x01000e) to Seven. How To Manage Your Network with SPECTRUM Page 154 Insignificant Device Models An insignificant device (e.g., end user PC) toggles between Blue and Green contact states and does not generate alarms or event messages to get the attention of the administrator. To change a significant model into an insignificant model change the value of the attribute Value_When_Red (0x01000e) to Zero. Inferred Connectors These are dumb models that do not poll, but keep track of a list of their Data Relay neighbors. Possible inferred connectors are: WA_Segment, Fanout, etc. SPECTRUM will automatically enable live pipes for all ports connected to a WA_Segment. Note: Note: SPECTRUM intelligence does not expect Fanout models to be connected to each other, thus this configuration will result in inaccurate contact status displays. If two Fanouts are connected to each other and each of them is in turn connected to a device with a contact status of green, the fanouts will nonetheless turn red. If two fanouts are connected to each other with no other devices connected to either, both fanouts will turn gray. Composite and Discrete Topology Models The contact status of LAN, LAN 802.3, LAN 802.5 etc. models is determined by the contact status of its collected children. A LAN model with lost contact status will turn either Red or Grey, depending on the condition of its collected models. Wide Area Links In SPECTRUM, Wide Area Links (WA_Links) are modeled in conjunction with fanout or wide area segment (WA_Segment) models (see Note below). This allows for proper roll-up of the Wide Area Link condition. For instance, in using a fanout, devices at either end of the WA_Link are connected to the fanout model. You can view this by navigating into the fanout’s cablewalk view. In turn, the fanout model must be connected to the appropriate port of the device model. This cross-con- How To Manage Your Network with SPECTRUM Page 155 nection is very important for fault isolation to work. WA_Link models work best when modeling point-to-point types of Wide Area connections (such as T1 and T3 lines). Note: Note: WA_Link models can accommodate only one WA_Segment model. If you attempt to paste more than one WA_Segment model into a WA_Link model’s Topology view, the second one will be destroyed immediately and an alarm will be generated. The following is a typical Wide Area network modeled in SPECTRUM: VNM Rtr1 Rtr1 WA_Link WA_Segment Rtr2 Rtr2 Wide Area Segments WA_Segments poll the InternalPortLinkStatus (IPLS) attribute of each interface model which Connects_To the WA_Segment. This is an active poll, meaning that the IPLS of each connected interface is read at every polling interval rather than simply watched for a change in the attribute. Therefore, SPECTRUM does not have to lose contact with one of the connected routers in order for a Fault Isolation alarm to be generated on a WA_Link. The polling of the connected ports’ IPLS will be regulated by the WAL_Link model’s Polling_Interval and Polling_Status attributes. When the Polling_Interval changes to zero (0) or Polling_Status goes to FALSE, polling of the connected port’s IPLS is stopped. • If one of the connected interfaces has an IPLS of BAD (for example, Admin Status is ON, but Oper Status is OFF), then the WA_Segment’s Contact_Status is set to LOST and the WA_Segment will go GRAY. The WA_Link will go RED. How To Manage Your Network with SPECTRUM Page 156 • If one of the connected interfaces has an IPLS of DISABLED (for example, Admin Status is OFF), then the WA_Segment’s Contact_Status is set to LOST and the WA_Segment will go GRAY. The WA_Link will go ORANGE. This is because the alarm must be severe enough to be viewed in the Alarm Manager, but it is not a “Contact Lost” alarm. If the DISABLED interface causes SPECTRUM to lose contact with the remote router, then the WA_Link will go RED. This is the regular InferConnector-type Fault Isolation working. How To Manage Your Network with SPECTRUM Page 157 Table 9: Model Category vs. Condition Color Model Category Model’s Neighbors (connected models) Significant Devices (Modeling Hub-types only.) connected to a VNM... Significant Devices with no connections to other models (a zero connector count)... Significant Devices connected to an established Data Relay neighbor... Composite and Discrete Topologies in which all of the collected children have a lost contact status and at least one of those collected children is Red... Inferred Connectors where the fanout model has lost contact but one of its neighbors is good and the associated port has bad port link status, then it... Significant Devices, Inferred Connectors, and WA_Links where all neighbors have also lost contact status... Composite and Discrete Topologies in which all of the collected children have a lost contact status and none of those collected children are Red... How To Manage Your Network with SPECTRUM Condition Color turn Red after losing contact. turn Gray after losing contact. Page 158 Model Category Model’s Neighbors (connected models) Condition Color Significant Devices (Modeling Hub-types only.) connected to an end-point turn Orange after neighbor (e.g., PC) that has losing contact. established contact status... WA_Links WA_Segment (or fanout) is good and one of the routers is lost then... Significant Devices connected to a model with turn Green. an Established contact status... Composite/Discrete Topologies and WA_Links in which any of the collected children has established contact status, then the LAN will also... Inferred Connectors connected to a model with an established contact status where at least one of its neighbors is Good and its associated port (port connected to the Fanout) status is Good... Significant and Insignificant Devices not yet connected to other turn Blue. devices... Composite/Discrete Topologies and WA_Links when all collected children of a LAN have initial contact status, then the LAN will also have the initial contact status... How To Manage Your Network with SPECTRUM Page 159 Examples The following examples illustrate how SPECTRUM fault isolation operates with various network configurations and problem scenarios. Example 1: This example demonstrates that fault isolation is a pro-active mechanism, and does not depend upon polling all of the connected models. Consider a simple network topology as shown in Figure 61. The device H1 is connected to the VNM model. Devices H1, H2 and H3 poll every 3 minutes. H4 polls every 5 minutes. The PC polls every 30 minutes. Figure 61: Network Topology Example 3 H1 VNM 3 3 H3 H2 5 30 H4 PC Assume H2 is BAD. As a result H2 would turn red, H4 would turn gray, PC (insignificant model) would turn blue, while H1 and H3 should remain green. Fault isolation is initiated as soon as H2, H4 or PC polls. If H4 is lost, it sends an Are-You-Down action to H2. If H2 is lost by then, it sends TRUE to H4, otherwise it pings itself and then sends the response to H4. This causes H4 to turn gray. Now H2 is lost, and it sends Are-You-Down action to H1. Since H1 is established, H2 has to decide between orange and red conditions. H2 How To Manage Your Network with SPECTRUM Page 160 pings PC. Since PC can not respond H2 will turn red. The ping from H2 puts PC in a lost state. Since PC is an insignificant device it will turn blue. Note: Note: H1 and H2 are connected by dragging H2 onto a port of H1 and H1 onto a port of H2. Example 2: This example demonstrates fault isolation when modeling a fanout. Assume fanout is red, D2 and D3 are gray, and then D3 polls successfully. Figure 62 depicts this scenario. Figure 62: Network Topology with a Fanout 2 VNM D1 P1 Fanout 4 P2 D2 4 P3 D3 30 PC D3 will have an established contact status and turn Green. Then D3 will send an Are You Up action to the fanout. The fanout reads device P3’s (D3’s port connection to the fanout) internal link port status. Assuming the port has good status the fanout will turn green with established contact status. This means that as long as P1 (D1’s port connection to the fanout) has good internal link port status, the contact status of the inferred connector will remain good. How To Manage Your Network with SPECTRUM Page 161 What if: D2 goes bad? D2 will lose it’s contact status and sends an Are-You-Down action to the Fanout. The Fanout will ping D1, and finds D1 to be good. The intelligence then examines the status of P1. Assuming Link-Status of P1 is good, the Fanout will return FALSE to model D2. This causes D2 to turn Red. What if: P1 is bad? This is the same case as disconnecting the network connection to the Fanout. If D3 polls first, it will lose its contact status and send an are-youdown action to the Fanout. The Fanout will ping D1 as finds it as a good neighbor. Fanout then reads the internal-port-link-status of the port P1. Since P1 is bad, the Fanout will lose its contact status and turns Red. The Fanout will return TRUE to the model D3. This causes D3 to turn Gray. D2 will also turn Gray in the same way as D3. PC being the insignificant device will turn Blue immediately after losing its contact status. Example 3: This example shows how SPECTRUM manages devices using other redundant paths if a link is shutdown administratively (i.e., admin-status equals down). Figure 63 depicts a network with redundant WA Links. Here VNM manages Rtr3 through link WL-1 and Rtr2 using link WL-2. Now let us assume that the network administrator shuts down the WL-1 link. This causes WL-1 to turn gray. Rtr3 will turn red because VNM can not talk to it through WL-1. The redundancy intelligence of Rtr3 will modify its agent address, so that VNM can talk to it using links WL-2 and WL-3. This causes Rtr3 to turn green again. The link WL-1 will still have the gray condition. How To Manage Your Network with SPECTRUM Page 162 Figure 63: Redundant Wide Area Links New York VNM Rtr1 WL-1 London Rtr3 WL-2 WL-3 Rtr2 San Francisco Example 4: This example demonstrates that fault isolation for an Inferred Connector requires specific modeling. Assume that two routing devices, Rtr1 and Rtr2, are connected at both ends of the WA_Link and that their ports are P1 and P2 respectively. WA_Link models need to be associated with a WA_Segment (or fanout) model through the Collects relation to enable the proper roll-up of the WA_Link condition. The devices at either end of the WA_Link need to be connected to the WA_Segment collected by the WA_Link model. You do this by navigating into the device’s Device Topology view and resolving the WA_Segment off-page reference icon to the appropriate port. You can view the connections by navigating into the WA_Segment’s Cablewalk view. This cross-connection is very important for fault isolation to work, as shown in Figure 64. How To Manage Your Network with SPECTRUM Page 163 Figure 64: Network Topology Rtr1 VNM P2 P1 Rtr2 WA_Link Rtr1 WA_Segment Rtr2 WA_Link Assume P1 is the port on Rtr1 and P2 is the port on Rtr2. The routers connected to the WA_Segment will cause it to behave as described in the following table. Note that the port link status becomes important in determining the status of the WA_Link only when both Routers are “contact established.” Table 10: WA_Segment Conditions Rtr1 Note: Note: Rtr2 WA_Link initial initial blue established lost red lost lost gray established established Check Port States See Note below. If both Rtr1 and Rtr2 have a contact status of “established” then the port status of P1 and P2 will determine the condition of the WA_Link. If any port is BAD, the WA_Link will be RED. If any port is DISABLED, the WA_Link will be ORANGE. Otherwise, the WA_Link will be GREEN. How To Manage Your Network with SPECTRUM Page 164 Duplicate Addresses SPECTRUM intelligence automatically detects when duplicate Internet Protocol (IP) addresses are entered in the SpectroSERVER database. Although some devices are allowed to have a duplicate IP address, Cabletron hub devices should be configured with only one IP address per device. Alarm condition colors warn you of duplication, as shown in Table 11 (Page 166). Note: Note: Beginning with SPECTRUM version 6.6, the policy for modeling duplicate IPs in SPECTRUM has changed in the following way. SPECTRUM is now capable of modeling different devices that share IPs, provided that each device has at least one IP that is unique to that device. This was done to accommodate certain networking technologies such as load balancing that create identical IP addresses across a range of devices. Devices that share some interface IP addresses can now be modeled manually or by using AutoDiscovery. Devices that share all of their interface IP addresses in common cannot be modeled manually or by using AutoDiscovery in SPECTRUM. How To Manage Your Network with SPECTRUM Page 165 Table 11: Duplicate Address Alarms and Color Status Alarm Same MAC Address & Different IP Address Color Yellow This alarm occurs when there are two or more Models with the same MAC address and at least one model with a different IP address. Same IP Address & Different MAC Address Orange This alarm occurs when there are two or more models with the same IP address and at least one model with a different MAC address. Same IP Address & Same MAC Address Yellow This alarm occurs when there are two or more models with the same IP address and the same MAC address (duplicate addresses). Duplicate MAC Address Yellow The special case alarm for duplicate models where at least one of the models does not have an IP address. There is only one model that has this characteristic: Physical_Address model type. To get these alarms a model type needs both the MAC address and the IP address. For example, the model types Pingable and PhysicalAddress do not have both addresses so you will not see the above alarms. To manually clear a duplicate address: 1 Open the Enterprise Alarm View. 2 Select the model that has the duplicate IP address alarm so that its icon is visible in the Alarm View. Determine if you want to allow two How To Manage Your Network with SPECTRUM Page 166 devices to have the same IP address. If not, you will need to change one of the devices to a unique IP address, and then use the Update feature to change the IP address within SPECTRUM. 3 To clear the duplicate, select the Clear button in the Alarm View. The alarm is removed. Once the duplicate IP address alarm clears, the status color on the model’s icon returns to a normal green condition, unless another alarm is present for the model. For detailed information on the Enterprise Alarm Manager, refer to the Enterprise Alarm Manager User’s Guide. For more information on the Update feature, refer to the GIB Editor User’s Guide. Automatic Naming and Addressing in SPECTRUM SPECTRUM has an automatic model naming and addressing feature, which is implemented through the AUTO_NAME attribute (attribute ID 0x00011979). The value for this attribute is set to “TRUE” by default for each model type in your modeling catalog. You can disable automatic naming and addressing on a model type-by-model type basis by using the Model Type Editor (MTE) to set the value to “FALSE.” Otherwise, the feature functions as described below. If you create a new model using only the IP address, SPECTRUM automatically attempts to supply a name for the model in one of three ways: • using NIS (Network Information System) or DNS (Domain Naming Service) to get the name from the modeled device; • checking the local /etc/hosts file for the name associated with the modeled device’s IP address; • using the IP address as the model name. How To Manage Your Network with SPECTRUM Page 167 Note: Note: The priority order of the source that will be used to supply a name for the model (IP Address, Name Service, or sysName) is dictated by the "Model_Naming_Options" attribute on the VNM model. This attribute can be modified on the VNM model's control view. See the Configuring Landscapes section of the Distributed SpectroSERVER User Guide (2770) for more information. Likewise, if you create a new model and supply only a model name, SPECTRUM will attempt to use NIS, DNS, or the local /etc/hosts file to retrieve the modeled device’s IP address. In either case, as long as the value for AutoName is TRUE for a particular model type, SPECTRUM will automatically maintain the names for models of that model type as follows: in the event the IP address for a model changes and the original model name was supplied by SPECTRUM, a new name will be supplied using one of the three methods listed above. However, if the original name was supplied by the user and differs from the name that would be supplied through automatic naming, then the original name will be preserved. Board and port models are also automatically named by default, each being assigned the name of the parent device model suffixed with the board/port Instance ID. For example, if a model of model type Hub_CSI_IRM2 is named IRM2_UK, and the modeled device has a port with the Instance ID of 2.5, the name of the port will be IRM2_UK.2.5. If the device name is changed to IRM2_US, then the name of the port becomes IRM2_US.2.5. However, if the device name was user-specified and the user then changes the port name from IRM2_US.2.5 to LAB_PORT, then the automatic naming will not be not used for that port in the event of subsequent IP address changes. Some boards (mainly standalone MIMs) contain their own intelligence. In such cases, setting AUTO_NAME to FALSE as previously described will disable the autonaming intelligence and allow the board’s own intelligence to work. How To Manage Your Network with SPECTRUM Page 168 Monitor Points Network group models (FDDI, LAN_802_3, or LAN_802_5) are conceptual models that represent a logical collection of actual devices and/or other network groups. A Monitor Point is a specific device within a network group model that provides the statistics that are used to calculate network activity in the network group model’s Performance View. Statistics, such as Packet Rate and Bits Per Second, are gathered by a Monitor Point and used to calculate total network activity. The specific Monitor Point device is selected by SPECTRUM based on the value of its Monitor Precedence attribute (0x11a44). Within a network, the device model having the highest Monitor Precedence value becomes the monitor point for that network group. If more than one device has the same Monitor Precedence value, the first device seen is chosen. Typically, the highest precedence is for network analyzing devices, followed by successively less intelligent devices. (Default precedence values can be altered by updating them in a Network’s Information View.) How To Manage Your Network with SPECTRUM Page 169 Monitor Point Statistics The collection, rollup and calculation of monitor point statistics is determined by two base model types: Composite (model type handle = 0x1002c) and Discrete (model type handle = 0x102e1). Each of these model types provide the intelligence (inference handlers) that produce a specific set of monitor point statistics. Model types that inherit these model types, inherit the ability to gather and calculate the monitor point statistics. Figure 65 shows the monitor point model type hierarchy and an example of model type inheritance by network group model types. Figure 65: Monitor Point Model Type Hierarchy Entity_Type ComTopol- Net- DisLAN_Typ LAN_802 Collects Moni- HU HU PC LAN_802 Lan- PC HU Potential Monitor In general, monitor point calculations, for model types derived from the Discrete model type, consist of reading attribute values from a Monitor Point device model, then using those values to establish the attribute values of a Discrete network group model (WA_Link, FDDI, LAN_802_3, LAN_802_5). How To Manage Your Network with SPECTRUM Page 170 Discrete network group models (derived from the Discrete model type) are, in turn, collected by a composite network group model (derived from the Composite model type). Composite model types use the statistics gathered in the Discrete model types to calculate their own statistics. Composite Monitor Point Statistics Five model types are derived from the Composite model type: • • • • • BroadBand_Net LAN Network Universe WAN The Composite model type calculates it’s monitor point statistics by comparing or summing attribute values from each of the models that it COLLECTS. Four attributes are calculated by finding the corresponding maximum value in each of the collected models: • • • • Load Loadx100 PacketRate SoftErrorRate One attribute is found by summing the values of the corresponding attribute from each of the collected models: • Hard_Error_Count How To Manage Your Network with SPECTRUM Page 171 Discrete Monitor Point Statistics The model types derived from the Discrete model type use a more sophisticated calculation. These calculations are performed by inference handlers attached at the respective model types derived from the Discrete model type. Four network group model types are derived from the Discrete model type: • WA_Link • LAN_Types - LAN_802_3 - LAN_802_5 - FDDI Each of these models calculates its Monitor Point statistics a little differently. The following sections show the equations used by the inference handlers for each model type to calculate attribute values. In the WA_Link/LAN_Types model types described in the following sections, the calculations performed in the inference handlers are shown as equations. The values on the left are the attributes of the LAN_Types/ WA_Link level. The values on the right are values read from the Monitor Point device model. The calculation of Load is multiplied by a factor of 100 to show Load as a percentage. In certain models, where Load is a relatively small value, the Loadx100 attribute is used to obtain a better graphical presentation of the actual value. Loadx100 multiplies the value of Load by another factor of 100. How To Manage Your Network with SPECTRUM Page 172 WA_Link (model type handle = 0x102e2) The Wide Area Link (WA_Link) model type is used to model a connection between the interfaces of two connected routers. The monitor point calculations are computed using one of the two interfaces that are connected to this model. Figure 66 shows how the interfaces are connected to the WA_Link model and illustrates the MONITORS association. Figure 66: WA_Link Relations WA_link Router 1 Interface Router 2 Interface fanout Monitors Connects Collects HASPART The monitor point calculations that are performed for this model use information calculated from the data_relay_prt model type. The data_relay_prt model type is the base model type for router interfaces. WA_Bandwith is calculated by an inference handler attached to the WA_Link and is set to the if_speed of the data_relay_prt model. If this value is zero, then the default of 1540000 (T1 interface) is used. Table 12 shows monitor point calculations for the WA_Link. How To Manage Your Network with SPECTRUM Page 173 Table 12: WA_Link Monitor Point Calculations Attribute = Formula WAMonMaxBitsPerSecIn × 100 Load = --------------------------------------------------------------------------WABandwidth SoftErrorRateIn = WAMonSoftErrorRateIn Attribute = Formula WAMonMaxBitsUsedPerSecIn × 100 × 100 Loadx100 = -----------------------------------------------------------------------------------------------------WABandwidth PacketRateOut = WAMonPacketRateOut WAMonBitsUsedPerSecOut × 100 LoadOut = --------------------------------------------------------------------------------WABandwidth WAMonBitsUsedPerSecIn × 100 LoadIn = ----------------------------------------------------------------------------WABandwidth Lan_MP_SysUpTime = MonSysUptime SoftErrorRate = WAMon_Max_SoftError_Rate PacketRateIn = WAMonPacketRateIn PacketRate = WAMon_Max_Packet_Rate SoftErrorRateOut = WAMonSoftErrorRateOut How To Manage Your Network with SPECTRUM Page 174 WAMonBitsUsedPerSecOut × 100 LoadOut = --------------------------------------------------------------------------------WABandwidth WAMonBitsUsedPerSecIn × 100 LoadIn = ----------------------------------------------------------------------------WABandwidth Lan_MP_SysUpTime = MonSysUptime SoftErrorRate = WAMon_Max_SoftError_Rate PacketRateIn = WAMonPacketRateIn PacketRate = WAMon_Max_Packet_Rate SoftErrorRateOut = WAMonSoftErrorRateOut How To Manage Your Network with SPECTRUM Page 175 FDDI (model type handle = 0x10032) The FDDI model type is used to model an FDDI network. The monitor point calculations for this model type are computed using the model that is associated to the LAN model with the MONITORS relation. The following table shows the monitor point calculations performed for this model type Table 13: FDDI Monitor Point Calculations Attribute = Formula Attribute = Formula Load = UtilizationRate LAN_MP_SysUpTime = Uptime Packet_Rate = FrameRate RingStatus = RingStatus Soft_Error_Rate = ErrorRate Note: Loadx100 is not calculated for the FDDI model type. How To Manage Your Network with SPECTRUM Page 176 LAN_802_5 (model type handle = 0x1003d) The LAN_802_5 model type is used to model a Token Ring network. The monitor point calculations for this model type are computed using the model with the highest precedence that is associated to the LAN_802_5 model with the MONITORS relation. The following table shows the monitor point calculations performed for this model type. Table 14: LAN_802_5 Monitor Point Calculations Attribute = Formula LAN_Bandwidth = Mon_Bandwidth TR_Band_Util = TR_Mon_Utilization TR_Bytes_Per_Second = TR_Mon_Bytes_Per_Second Attribute = Formula TR_Errs_Per_MFrame = TR_Mon_Errs_Per_MPacket LAN_MP_SysUpTime = Mon_Sys_Uptime TR_Active_Stations = TR_Mon_Active_Stations TR_Frame_Rate = TR_Mon_Frame_Rate Soft_Error_Rate = TR_Mon_Errs_Per_MFrame TR_Ring_Speed = TR_Mon_Ring_Speed HardErrorCou n t = TRMonHardErrorCount Note: Packet_Rate is not calculated for the LAN_802_5 model type. How To Manage Your Network with SPECTRUM Page 177 LAN_802_3 (model type handle = 0x1003c) The LAN_802_3 model type is used to model an Ethernet network. The monitor point calculations for this model type are computed using the model with the highest precedence that is associated to the LAN_802_3 model with the MONITORS relation. LAN_Bandwidth is an attribute in the LAN_802_3 model type. This attribute is set to 10,000,000. The following table shows the monitor point calculations performed for this model type. Table 15: LAN_802_3 Monitor Point Calculations Attribute = Formula Attribute = Formula CollisionRate = CollisionRate HardErrorRate = MonHardErrorRate LAN_MP_SysUpTime = Mon_Sys_Uptime PacketRate = MonPacketRate MonBitsUsedPerSec × 100 Load = ---------------------------------------------------------------MonBandwidth MonBitsUsedPerSec × 100 × 100 Loadx100 = ------------------------------------------------------------------------------MonBandwidth SoftErrorRate = MonErrorRate HardErrorCount = MonHardErrorCount Monitor Point Calculations in SpectroWATCH SpectroWATCH is an intelligent data collection system used to manage networks. Many of the monitor point calculations for SPECTRUM are now in watches in SpectroWATCH. This makes it easier to view the existing formulas and add new ones. The monitor point calculations in the preceding sections require reading attributes from other models within the database and are not yet implemented as watches in SpectroWATCH. However, in a future release, even these will be moved to SpectroWATCH, and all of the monitor point calculations will reside in watches, making How To Manage Your Network with SPECTRUM Page 178 them more readily accessible. Refer to your SpectroWATCH Operator’s Reference for a list of monitor point calculations performed in SpectroWATCH. Detection of Firmware Problems SPECTRUM allows for automatic detection of problems in some device firmware. Using the “connects_to” and “collects” model type relations formed via the Device Topology view, SPECTRUM detects if a network management firmware problem exists in the hub device. The connects_to relation denotes that one model is attached or connected to another (for example, a PC model connects to a hub model via a port on the hub). The collects relation denotes that one model collects information from another model (for example, the Topology view model collects information from the hub devices contained within that view). If SPECTRUM cannot retrieve management information from a hub device, but can still contact devices connected to this hub, then a hub firmware management problem can be deduced and the hub icon’s condition color is set to orange. You should then use the Alarm View along with the Diagnostic View and the Performance View to help isolate and correct this problem. Device Threshold Events When a device’s internal threshold values are exceeded, the device’s firmware generates an unsolicited alert message, or trap. SPECTRUM intelligence evaluates these trap messages and generates an appropriate message in the Event Log View and/or Alarm View. Additionally, this process also sets the device icon’s condition to yellow or orange indicating a minor alarm. For information on alarms and events, refer to the SPECTRUM management module guide for the specific device. How To Manage Your Network with SPECTRUM Page 179 Redundancy SPECTRUM allows you to configure redundancy not only for the network management system itself, but for paths between devices. Redundant SpectroSERVERs (Fault Tolerance) SPECTRUM provides for the loss of a critical server by allowing you to have up to two backup servers. These backup SpectroSERVERs are ready to go within minutes of the loss of the main server. The backup SpectroSERVERs have the same landscape handles, but different precedences, and do not actually poll any devices on the network until they are made the primary server. Frequently scheduled on-line backups ensure the database is as close as possible to the primary server’s database. Any traps should be configured to be sent to all backup SpectroSERVERs so that in the event of the loss of the primary SpectroSERVER, traps will still be sent to the backup server as it is operating. For more information on setting up redundant SpectroSERVERs, refer to the section on fault tolerance in the Distributed SpectroSERVER manual (2770). Redundant Paths Devices that have more than one way of being reached have redundancy. If the redundancy feature is enabled on a device and it cannot be reached directly, SPECTRUM attempts to reestablish contact using the IP addresses of the device’s other interfaces (see Configuring Device Redundancy (Page 181)). This provides an alternate line of communication by forging a connection over a redundant path. If SPECTRUM still cannot reach the device, its icon changes to condition red (contact lost) and the alarm is recorded in the Alarm View. Note: Note: The redundancy feature is not supported by all device models. To determine whether a given device supports the feature, consult its SPECTRUM management module guide. How To Manage Your Network with SPECTRUM Page 180 Configuring Device Redundancy Redundancy is enabled by default in the Redundancy Administrative Status field. To access the field, click the Redundancy Options button in the Device Fault Management View (Page 70) of the modeled device. (The field can also be accessed in the Redundancy and Model Reconfiguration Options view, accessible from the device’s Configuration view.) Disabling the Redundancy feature changes the value of the RedundancyEnabled (0x00011d2c) attribute to FALSE. The default value is TRUE. Editing the Redundancy Preferred Addresses List In addition to the RedundancyEnabled (0x00011d2c) attribute, each router model in SPECTRUM also has the following attributes: • Primary Address (PA): (0x00011d2a) - This is the IP address of the interface SPECTRUM uses to communicate with the device. This address is initially used to model the device. • Agent ID or Network Address: (0x0001027f) - This is the IP address of the interface through which communication with the device is active. Initially, the network address and the PA are the same, but later may be different from the PA at a given time when redundancy determination is in progress. • Redundancy Preferred Addresses (RPA): (0x00011d2b) - This is a list of the device’s interface IP addresses and is used in determining a redundant connection. The RPA list is created when the router is first modeled. Note: Note: The SPECTRUM modeling process includes a loopback functionality. The first valid loopback address detected for a device will be used as the value for the device model’s PrimaryAddress, Network_Address, and Model_Name attributes. To enable this functionality, you must add use_loopback=true to the .vnmrc file. See Defining SPECTRUM Resources (2559) for additional information on the .vnmrc file and its associated resources. How To Manage Your Network with SPECTRUM Page 181 When contact with the device is lost, redundancy is activated in two ways. One way is that the addresses in the RPA list are checked every polling interval to determine their availability. Once an address from the RPA is reached, this stops. The other way is that a timer is started and attempts to reach the PA upon its expiration. When the PA is reached, this second action stops. You can configure the router model’s Primary Address and the Redundancy Preferred Addresses list through the Preferred Addresses button, found in the Redundancy and Model Reconfiguration Options view. Click the button to display the Preferred Addresses dialog box (Figure 67). Figure 67: Preferred Addresses Dialog Box Redundancy Preferred Addresses Each IP address in this list is numbered according to the order in which it will be used by SPECTRUM. You can change the list to suit your network needs. • To add an address to the RPA list, click the Insert... button and enter the IP Address and desired position. How To Manage Your Network with SPECTRUM Page 182 • Highlight an address in the RPA list and click Delete to remove it from the list. • Highlight an address in the RPA list and click Move to change the order in which it is accessed. In the dialog box that appears, enter the source and destination positions for the address that you want moved. • Highlight an address in the RPA list and click Primary to make the address the Primary Address (PA). Click Save to save the modifications you make to the RPA list, or click Cancel to exit the Preferred Addresses dialog box without saving the changes. Primary Address This field displays the value of the current PA (the field is not editable). Redundancy Excluded Addresses The Redundancy Excluded Addresses list displays those IP addresses you want SPECTRUM to exclude from use in a redundant path. To add an address to the Redundancy Excluded Addresses list, highlight the address in the RPA list on the left and click the right arrow button. Or, you can click the Insert... button under the Redundancy Excluded Addresses list to insert an address at a given position in the list. Device Redundancy Events Events are logged in the Event Log when a redundancy related action occurs. They are generated when: • A Primary Address that is in use has been determined to be no longer valid and the Network Address, which is valid, becomes the new Primary Address (0x00010914). • It is determined that a preferred address in the RPA list is no longer valid and thus deleted from the RPA (0x00010914). • A given Primary Address is not reachable and does not belong to the device modeled (0x00010915). How To Manage Your Network with SPECTRUM Page 183 • The Network Address differs from the Primary Address (0x00010916). • A model is incapable of redundancy, which happens when the PA used to model the device can be reached but the agent on the device does not provide the necessary MIB-II information to be able to determine redundancy (0x00010917). Alternate Path Repeater Management Alternate Path Repeater Management (APRM) prevents loss of contact with repeater models caused by the activation of router redundancy. APRM is designed for devices capable of functioning as IP routers and that contain the following repeater models types: CSIRptr, FddiSMT, and MMACPlusRptr. You can enable/disable APRM from the Alternate Path Repeater Mgmt. button in the Model Information View of the repeating application. The default state for APRM is disabled. Note: Note: As the repeater model types of bridges and other non-routing devices do not inherit router redundancy there is no need to enable APRM for them. When contact with a device is lost and router redundancy activates, SPECTRUM attempts to recontact the device through one of the model’s preferred addresses. The address used to successfully recontact the device becomes the model’s network address. When APRM is enabled for the relevant repeater models their network addresses change to match the device model’s. This insures contact with the repeaters as long as there is contact with the device. When APRM is disabled the new network address is not rolled down to the device’s repeater models, which causes SPECTRUM to lose contact with the relevant repeater models. APRM has three possible states: enabled, disabled or activated. Should the APRM state change for a repeater model an event is generated and logged to that repeater’s Event Log. How To Manage Your Network with SPECTRUM Page 184 Device Lost and Found SPECTRUM has a lost/found intelligence circuit which automatically detects a device that does not relate to any Location, Topology, or Organization View. Whenever an existing device becomes unrelated to any view, the lost/found intelligence sends it to Lost and Found. This can happen when a device has been erased from a view but never pasted into another view, or when the exact physical location of the device is unknown. Device Type Verification SPECTRUM intelligence automatically verifies the device type for any SNMP device that you add to your network via the Edit menu’s New Model option, or by AutoDiscovery. The SNMP device verification process follows the following steps: 1 SPECTRUM communicates with the newly modeled device to read the device’s SysObjectID and SysDesc attributes. 2 SPECTRUM intelligence goes through several identification methods to identify the device. • Identify Device by OID Method - If the value of SysObjectID contains the DeviceID, then this method is used. This method relies on the SysObjectID being a unique identifier for a device type. • Identify Device by System Descriptor Method - If the SysObjectID does not contain the DeviceID, then this method is used. This method searches the SPECTRUM database for the model type(s) whose System_Desc_Verify attribute (0x110c6) matches the value of SysDesc that was returned from the device. • Identify Device by Keyword Method - If the Identify Device by System Descriptor and Identify Device by OID methods fail to identify the device, then this method is used. This method uses both the SysObjectID and SysDesc values from the device to identify it. How To Manage Your Network with SPECTRUM Page 185 • If the device has not been identified by any of these methods then it is modeled as a generic snmp device (given that the SNMP Management Module is installed). 3 If multiple model type handles are returned then SPECTRUM intelligence uses the following method to choose the appropriate model type. • The Disposable_Precedence attributes of each model type is compared and the one with the highest Disposable_Precedence is selected as the model type to be used. • If no model type has a greater Disposable_Precedence, an event (event code 0x10644) is generated containing the IP Address and two model type names. Should this occur you should use MTE (Model Type Editor) to modify the Disposable_Precedence attribute of one of the two model types so that one of the values is larger. Then, the next time the model type with the highest Disposable_Precedence will be selected. Device Type Mismatch If the device type can be determined from these attributes, it is compared against the device type selected when the new model (New Icon) was added to the database. If the device is not the same as the new device, the device icon’s condition is set to yellow and an entry is recorded in the Alarm View and Event Log. When the SPECTRUM administrator is issued a device mismatch alarm, there are two things to check. Check the IP address of the device that is being modeled. If the wrong IP address was given, use the Update feature to correct the IP address for your new device model. Secondly, check the actual device type to make sure that the correct model was chosen from the given list of models. If the wrong model type was selected, select the mismatched model and choose the Destroy option from the edit menu and then re-create the correct model. The Model Mismatch intelligence circuit can be enabled or disabled by modifying attribute Verify_Model_Mismatch(0x00011197c). To disable the How To Manage Your Network with SPECTRUM Page 186 feature for a given model type, set Verify_Model_Mismatch to FALSE using MTE. To turn the intelligence circuit on, set it to TRUE. In-Band Configuration of Device Alarms SPECTRUM lets you manually configure alarm thresholds for many device models. Normally this is done via a GIB View. For some devices this is a Config Alarms View, accessed through the model’s Configuration View. Refer to the appropriate SPECTRUM management module guide for your device for more information on the configuration process. For more information on alarm thresholds, refer to your device’s hardware/ software manuals. Interface Intelligence Interfaces are ports that have both a physical address and a network address. These type of ports would be found on routers and bridges where network identity is important, unlike the ports on a repeater that may have a physical address, but do not have network addresses. The following diagram shows the derivation of the interface port: Port SnmpPif Inet_If_Port CSI_Rptr_Port (other data_relay_prt cisco_If_Port (other data_relay_prts) How To Manage Your Network with SPECTRUM Page 187 Port is the base model type for all ports. Derived from Port are two basic types of ports: repeater ports and interface ports. • Inet_If_Port is derived from Port and Enet Monitor model types. Inet_If_Port is the base class for all interface ports that are potential monitor points for network statistics. • Data_relay_prt is derived from Inet_If_Port and SnmpPIf. SnmpPIf is the base class for all model types that communicate with SNMP agents. Data_relay_prt is the base model type for all interface ports that do their own reading and polling. It is an instantiable model type and is used to model a generic interface port. More specific types of interface ports, such as Cisco_If_Port, are derived from data_relay_prt. The inference handler CsIHInterfaceIntLinkStatus, which computes InternalPortLinkStatus, is attached at Inet_If_Port so all interfaces will inherit the desired functionality. The intelligence for interfaces use ifAdminStatus and ifOperStatus defined in the MIB-II definition of the interface group in RFC 1158. These variables can have the state of ON or OFF, and are defined below: • ifAdminStatus: desired interface state - This is the state that the administrator wants the interface to be in. This attribute shows whether or not the interface has been shut off. The values of this attribute are ON and OFF. • ifOperStatus: current interface state - This attribute shows the actual state of the interface. The values of this attribute are UP and DOWN. UP means that the interface is communicating with the network properly. DOWN means the interface has lost connection with the network. These two variables are used to calculate a SPECTRUM internal attribute named INTERNAL_PORT_LINK_STATUS (IPLS - 0x101fb) This attribute’s possible values are LINK_STATUS_GOOD (LSG)), LINK_STATUS_BAD (LSB), and LINK_STATUS_UNKNOWN (LSU). This attribute is used to create and clear alarms, both on the interface and the device it is part of. It is also used to generate events concerning the attempt to reach of the interface. Table 16 shows how these variables are calculated. How To Manage Your Network with SPECTRUM Page 188 Table 16: IfAdminStatus Internal Port Link Status Conditions IfOperStatus INTERNAL_PORT_LINK_STATUS ON ON LINK_STATUS_GOOD ON OFF LINK_STATUS_BAD OFF ON LINK_STATUS_UNKNOWN OFF OFF LINK_STATUS_UNKNOWN The interfaces IPLS is set to LSU when SPECTRUM has lost contact with the device. Interface Alarms IPLS (Internal Port Link Status) is used to generate alarms for both the device model and the interface model. The only interface alarm is GRAY. These alarms can only be seen by going into the Detailed Alarm View of an interface model. It is interesting to note that the alarm for an interface with an IPLS of LSB will have a gray alarm. All models with alarms are displayed in the General Alarm View. This is undesirable for interfaces. Interfaces are considered to be a part of a larger device such as a router. When a router goes down, all of its interfaces goes down as well. A red alarm is generated for the router. It would be confusing to have all of the interfaces producing red alarms as well, cluttering the Alarm View and making it difficult to locate the router. A device will watch the IPLS of each of its interfaces. If any of the interfaces has an IPLS of LSB, then the device will generate a YELLOW alarm with a probable cause of CS_ALARM_CAUSE_PORT_LINK_STATUS_BAD. This alarm, once set, will not be reasserted until it is cleared. Only one alarm will be asserted for all ports. The first interface with an IPLS of LSB creates an alarm. The second and successive interfaces with an IPLS of LSB are put into a bad port list. Once the list is clear the YELLOW alarm is dissasserted. How To Manage Your Network with SPECTRUM Page 189 Each of the interfaces watches its own IPLS. Whenever the interface has an IPLS of LSB or LSU it will generate a GRAY alarm. This alarm will only show up in the interfaces Alarm View. When the device becomes unreachable the interface’s IPLS is set to LSU. A GRAY alarm with a probable cause of CS_ALARM_CAUSE_DEV_CONTACT_STATUS_LOST is generated. When the interface has been administratively shut off the interface’s IPLS is set to LSU. A GRAY alarm with a probable cause of CS_ALARM_CAUSE_ADMIN_SHUT_OFF is generated. When the interface becomes unreachable, its IPLS is set to LSB. A GRAY alarm with a probable cause of CS_ALARM_CAUSE_PORT_LINK_STATUS_BAD is generated. Interface Events The interface generates two events that deal with its status. The events contain information about the attempts to reach the interface. These events will be generated when a device has a YELLOW alarm due to a bad interface. Each event message contains the interface number and IP address. For example: Tue 20 Jul, 1994 - 13:31:50 Interface 2 (IP address = 129.128.127.2, type = Gen_IF_Port) on device cisco1 of type Rtr_CiscoMIM is unreachable. - (event [00010623]) As stated before, IPLS has three possible values. This makes it important to know the last two states of an interface’s IPLS to make a proper judgement about its current state. If the interface IPLS is LSB, and it was previously LSU, it is important to know if it was previously LSB or LSG. Due to this, each interface keeps the values of the last two states of IPLS. Events are generated based on these two saved values, and the current value. Table 17 shows which states generate events based on the IPLS. How To Manage Your Network with SPECTRUM Page 190 S PE C T R U M I n t e l l ig e n c e Interface Intelligence Table 17: Events Generated from IPLS States 2 Previous Previous Current Event GOOD UNKNOWN GOOD none GOOD UNKNOWN BAD UNREACHABLE GOOD BAD GOOD REACHABLE GOOD BAD UNKNOWN none BAD GOOD BAD UNREACHABLE BAD GOOD UNKNOWN none BAD UNKNOWN GOOD REACHABLE BAD UNKNOWN BAD none UNKNOWN GOOD BAD UNREACHABLE UNKNOWN GOOD UNKNOWN none UNKNOWN BAD GOOD REACHABLE UNKNOWN BAD UNOWNN none How To Manage Your Network with SPECTRUM Page 191 Index Symbols % Load, Bytes, and Packets 134 .csi files used in backgrounds 44 .vnmrc 181 A Accum Button 131 acknowledgement rollups 141 Adding or Changing Devices on your Network 51 address tables 13 Addresses duplicate 165 Adjusting Management Capacity 35 AgeOutStaleAlarmsOnly attribute 59 Alarm Symptom/Probable Cause 120 Alarm generation on DLCI Ports 61 Alarm generation on interfaces 60 Alarm Thresholds 56 AlarmAgeOutTime attribute 59 AlarmOnInvalidDLCIs 61 AlarmOnLinkDownIfTypes 94 AlarmOnLinkDownTrap 94 AlarmOnSpecificInterfaces 60 All Connected Ports - Multiple Devices Only 79, 87 AssertLinkDownAlarm 94 Audible Alarms 119 colors 119 Autodiscovery 12, 17 Automatic Addressing 167 Automatic Naming of models 167 B Background Color 43 Raster 42, 43 backup servers 180 Broadcasts 56 C Change Background Dialog Box 42 Time Scale Button 130 Changing Backgrounds 42 channel packet errors 129 Chassis Device View 125 Clear Button 131 Clearing old alarms 59 collision rate 129 Collisions 56, 133 Colors device condition 114 rollup condition 116 Completing Your Network Model 17 Composite_Condition 145 Condition adjusting 146 attributes 142 colors indicating 114 How To Manage Your Network with SPECTRUM Page 192 device model 140 icon color display areas 140 Organization Models 153 Rollup 140 Configuring Alarms 59 Configuring Traps 58 connecting a Pingable to a device model 110 connecting two Pingables 110 Contact Lost alarm 62 contact lost polling 39 container models 55 core 9 CRC and Alignment 133 Criticality 81 Cross-Landscape Fault Correlation 76 D Delta Button 131 Device View 9, 55, 111 Device Condition Colors Blue (Initial) 115 Gray (Unknown) 116 Green (Normal) 115 Orange (Major Alarm) 116 Red (Lost Contact) 116 Yellow (Minor Alarm) 115 Device Models dynamic configuration 137 insignificant 155 significant 154 static configuration 137 Device Topology (DevTop) View 15, 127 Disposable_Precedence 186 down_device_poll_interval_multiplier 40 Duplicate Addresses 165 E Enable Event Creation for a device model 76 for a port model 108, 109 Enable SPECTRUM Management 107, 109 Enterprise Alarm Manager 120 error rate 129 Error Rates 132 Errors 56 Excluding an IP address from the RPA list 183 F fanout 155 Fault Correlation Cross-Landscape using Enable Event Creation attribute 76 Fault Detection 114 Fault Isolation 120, 153 Configuration 65 configuring using port fault correlation 78 Fault Isolation model 65 Fault Isolation Model Information view 65 Firmware detecting problems 179 frame rate 129 Framesize Statistics 134 G Generating an Inventory Report 18 GIB View 42 group subnets 45 How To Manage Your Network with SPECTRUM Page 193 H L Height 43 High Broadcast 135 landscape handles 180 Lin Button 130 Linear Scale 130 link down traps 95 Link Fault Disposition 99 Link traps 90 Live Pipes 90, 102 load 129 Log Button 130 Logarithmic Scale 130 logical relationships 13 loopback 181 Lost & Found 51 I ICMP 66 ICMP pings 13 Identify Device Methods 185 IEEE 802.3 12 IMT 137 In-Band Configuration of Device Alarms 187 Inductive Modeling Technology 137 Inferred Connectors 155 insignificant models 57 Intelligence alarm thresholds 187 device type OID verification 185 duplicate address detection 165 firmware problem detection 179 lost and found 185 Interface connected 72 events 190 Intelligence 187 management neighbor 72 Interface Model Name Suffix 25 Interface Model Names Customizing 25 Interface Trap Configuration View 95 inventory list 12 Inventory Report 17 IP Address 124 IP addresses Redundancy Preferred Addresses (RPA) 181 M Maintaining Your Network Model 51 Maintenance Mode Applications 74 Device Models 108 Devices 73 Interfaces 74 maintenance mode 107, 109 Management Lost alarm 62 Management Neighbors Configuration 69 Max_Pulled_Bd_Cnt 138 Modeling Services 46 Monitoring Points 169 Multi-Attribute Graph Example 130 Multi-Attribute Line Graph 129 How To Manage Your Network with SPECTRUM Page 194 N Navigate In 123 neighbors 69 Network Address 181 network diagram 12 network model 13 O Off-Page Reference panel 16 ok_to_poll 105 Online Database Backup Configuration view 53 Optimizing Your Network Model 13 Org_Owns model 48 Out-of-Window (OOW) Collisions 133 OutstandingLinkDownTrap 93 P Packet Size 135 Performance View 128 Pie Chart Relation with Instance ID Example 131 Pie Charts 130 Pingables 109 Point and Click 123 polling devices that are down 39 polling intervals 35 PollPortStatus 90 port alarms 104 Port Always Down Alarm Supression 91, 106 Port Connection Representation 15 Port Criticality Setting 101 Port Fault Correlation 78 Caveats 80 Criteria 80 Examples 81 setting configuration options 79 port level connections 15 Port Status Automatic Alarm Clearing 106 Configuring monitoring of, 90 Events and Alarms 92 Polling Criteria 90 Preferred Addresses Dialog Box 182 Primary Address 181 changing in the Preferred Addresses dialog box 183 Protocol Statistics 135 Protocol Type 135 Pulled Board List 138 Pulled_Bd_Cnt 138 Pulled_Bd_List 139 R Redundancy 180 Redundancy Administrative Status 108 Redundancy Excluded IP Addresses 183 Redundancy Preferred Addresses (RPA) editing the list of 182 Rename Interface Models 26 Report Depth 20 Format 21 Generator 18 Type 19 Rollup Condition 145 How To Manage Your Network with SPECTRUM Page 195 adjusting 146 example 149 process flow 148 See also Condition 142 Rollup Condition Color Orange 117 Red 117 Yellow 117 Rollup Condition Colors 116 Rollup Condition Thresholds 56 Rollup Thresholds 146 RPA list 181 Runts and Giants 134 T Threshold Events device internal threshold 179 TIFF files used in backgrounds 44 Topology Models composite 155 discrete 155 Total Button 131 Traffic 56 Troubleshooting 132 U S Scheduling Database Backups 53 Maintenance Mode 73 Scroll to Date-Time Button 130 Select Color Index Dialog Box 43 Select File Dialog Box 44 Service container model 48 Show Secondary Alarms When Device is in Maintenance 74 Significance Level 147 Significance Levels 56 significant models 57 Simple Network Management Protocol 58 SpectroWATCH 63, 104 SPECTRUM intelligence 137 subnet address range 13 Suppress Linked Port Alarms 80 SysDesc 185 SysObjectID 185 unresolved fault alarm 68 Unresolved Fault Alarm Disposition 68 Using Organizational Views 45 Using SpectroWATCH 63 W WA_Links 155 WA_Segment 155, 163 Wide Area Links 155 Width 43 How To Manage Your Network with SPECTRUM Page 196
© Copyright 2025