IBM E027SLL-H Troubleshooting Guide
IBM E027SLL-H - Tivoli Monitoring - PC Manual
 |
View all IBM E027SLL-H manuals
Add to My Manuals
Save this manual to your list of manuals |
IBM E027SLL-H manual content summary:
- IBM E027SLL-H | Troubleshooting Guide - Page 1
IBM Tivoli Monitoring Version 6.2.3 FP1 Troubleshooting Guide GC32-9458-05 - IBM E027SLL-H | Troubleshooting Guide - Page 2
- IBM E027SLL-H | Troubleshooting Guide - Page 3
IBM Tivoli Monitoring Version 6.2.3 FP1 Troubleshooting Guide GC32-9458-05 - IBM E027SLL-H | Troubleshooting Guide - Page 4
Note Before using this information and the product it supports, read the information in "Notices" on page 267. This edition applies to version 6, release 2, modification 3, fix pack 1 of IBM Tivoli Monitoring (product number 5724-C04 ) and to all subsequent releases and modifications until otherwise - IBM E027SLL-H | Troubleshooting Guide - Page 5
Contents Tables xiii About this information xv Chapter 1. Introduction to troubleshooting 1 Sources of troubleshooting information . . . . . 1 Problem classification 1 Viewing the IBM Support Portal 2 Subscribing to IBM support notifications . . . . . 2 Chapter 2. Logs and data collection for - IBM E027SLL-H | Troubleshooting Guide - Page 6
'stop pending' state and cannot be manually started 86 When running the setup.exe, an unknown publisher error message displays 86 The error "Could not open DNS registry key" occurs 86 Agent not connecting to Tivoli Enterprise Monitoring Server 86 iv IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 7
network-mounted drive 87 Installation Application Support on Windows systems 88 Problems installing directly leak 92 Manage Tivoli Enterprise Monitoring Services does not start on AIX V6.1 blocked by another process that is using the IBM Chapter 6. Connectivity troubleshooting 115 Cannot log on - IBM E027SLL-H | Troubleshooting Guide - Page 8
properly after adding agents support for Tivoli Enterprise Portal Server receive a message 133 Password problem using the LDAP Security option usage 138 vi IBM Tivoli Monitoring: Troubleshooting Guide Data is not 148 Running the Tivoli Management Services Discovery Library Adapter, results in - IBM E027SLL-H | Troubleshooting Guide - Page 9
command 153 Troubleshooting monitoring server problems on distributed inconsistency between the History and Object windows 157 Monitoring server application support completes all seeding functions in the Manage Tivoli Enterprise Monitoring Services panel shows the Tivoli Enterprise Monitoring - IBM E027SLL-H | Troubleshooting Guide - Page 10
agent starts up as root 176 Large historical collections slow monitoring agents 176 Unable to access History Collection Configuration for any agent 176 viii IBM Tivoli Monitoring: Troubleshooting Guide Agent names and icons are displayed incorrectly 177 64 bit monitoring agents are not started - IBM E027SLL-H | Troubleshooting Guide - Page 11
ras1 logs . . . . 193 When upgrading a System Service Monitors agent from 3.2.1 to 4.0, receive KDY3008E message . . a baseline command 200 Chapter 11. Command troubleshooting 201 On Solaris 8 operating systems, checkprereq or acceptbaseline commands fail 206 Problems with Take Action commands and - IBM E027SLL-H | Troubleshooting Guide - Page 12
Support . . . 218 . 219 . 219 . 219 . 219 . 220 . 221 . 221 Chapter 13. Database troubleshooting connect to the database 225 Oracle problem with JDBC drivers prior to 11 troubleshooting . . 232 Log files for Netcool/OMNIbus Event Synchronization 232 x IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 13
by the IBM Tivoli Monitoring troubleshooting 242 General event synchronization troubleshooting Reporting troubleshooting 245 History report results in error 247 Date and time format in IBM Audit Facility troubleshooting 255 Audit IBM Tivoli Monitoring processes 257 Documentation library 259 IBM - IBM E027SLL-H | Troubleshooting Guide - Page 14
xii IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 15
Tables 1. Location of log files for the IBM Tivoli Monitoring components 36 2. Installation log files 39 3. fails 207 18. Utilities for backing up the database . . . . 223 19. Resolving problems sending events to Netcool/OMNIbus 233 20. Event status updates in Netcool/OMNIbus are not forwarded to - IBM E027SLL-H | Troubleshooting Guide - Page 16
xiv IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 17
this information This guide provides problem determination and resolution information for the issues most commonly encountered with IBM® Tivoli® Monitoring components and related products. You can use this guide in conjunction with the other books for your product. © Copyright IBM Corp. 2005, 2012 - IBM E027SLL-H | Troubleshooting Guide - Page 18
xvi IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 19
determining its cause. For example, a performance problem might be caused by a limitation of your hardware. If you are unable to solve a problem on your own, contact IBM Software Support for a solution. See Chapter 2, "Logs and data collection for troubleshooting," on page 5 for information on the - IBM E027SLL-H | Troubleshooting Guide - Page 20
services. It brings all the support resources available for IBM hardware and software offerings together in one place. About this task Perform the following actions to access technotes for this product: Procedure 1. Open the http://ibm.com website and select Support & downloads > Technical support - IBM E027SLL-H | Troubleshooting Guide - Page 21
Results You begin receiving "IBM My notifications" emails about the products you have selected and at the interval you specified. Chapter 1. Introduction to troubleshooting 3 - IBM E027SLL-H | Troubleshooting Guide - Page 22
4 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 23
data collection for troubleshooting If you have a problem that you are unable to solve using the information in this guide or on the IBM Support Portal, gather the information that relates to the problem and contact IBM Software Support for further assistance. Appropriate IBM Tivoli Monitoring RAS1 - IBM E027SLL-H | Troubleshooting Guide - Page 24
Tivoli Enterprise Portal Server server process, KfwServices, the core is created in the /opt/IBM/ITM/archtype/cq/bin/ directory. Procedure To retrieve information on where the core file is USER GENERATED SIGNAL Recommended Actions CORRECT THEN RETRY 6 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 25
PROBLEM PERSISTS THEN DO THE FOLLOWING CONTACT APPROPRIATE SERVICE REPRESENTATIVE Detail Data SIGNAL NUMBER 11 USER'S PROCESS ID: 32248 FILE SYSTEM SERIAL NUMBER 10 INODE NUMBER 655367 PROCESSOR ID 0 CORE FILE NAME /opt/IBM IBM Support to diagnose problems KpcCMA.RAS files IBM Tivoli Monitoring on - IBM E027SLL-H | Troubleshooting Guide - Page 26
serviceability file. These files contain system dump information similar to the drWatson.log, but are generated by the IBM v Description of the operation scenario that led to the problem. v Incorrect output, such as Tivoli Enterprise Portal screen 8 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 27
Services can encounter problems such as missing workspaces or historical data, or a reflex automation script that does not run when it should. In many cases you can recover from these problems by following a few steps. Note: Use the trace settings indicated in these troubleshooting instructions - IBM E027SLL-H | Troubleshooting Guide - Page 28
workspace, see "Resolving Oracle DB Agent problems - diagnostic actions" on page 32. Similar problems might exist for other monitoring agents. 5. Check that application support has been added. See "Resolving application support problems" on page 11. 10 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 29
. What to do next For more information and instructions on installing application support see "Configuring application support for nonbase monitoring agents" in the IBM Tivoli Monitoring Installation and Setup Guide. For instructions on installing application Chapter 3. Common problem solving 11 - IBM E027SLL-H | Troubleshooting Guide - Page 30
is running and that application support has been installed correctly. For information on monitoring agents on z/OS® see each product's Problem Determination Guide. Procedure Diagnostic and corrective example: KMQMI171I Node JSG1:SYS:MQESA is online. 12 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 31
Verify the state of the monitoring agent in Manage Tivoli Monitoring Services. 2. Compare the status of the node in the physical Navigator then the problem is located in the portal server. Run the portal server trace, collect logs, and call IBMSoftware Support. Chapter 3. Common problem solving 13 - IBM E027SLL-H | Troubleshooting Guide - Page 32
remote monitoring server is connected to the hub monitoring server. Some causes of monitoring server problems: v A remote monitoring server has shut down v Loss of connectivity between the the Managed System Status workspacein the portal client. 14 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 33
/ v15r1/topic/com.ibm.itm.doc_6.2.3fp1/itm623_messages.htm) reference guide. Operator responses and general information are provided for each message. 2. Allow the portal client enough time to establish a connection with the portal server. 3. Is DB2® running? Chapter 3. Common problem solving 15 - IBM E027SLL-H | Troubleshooting Guide - Page 34
On the GUI interface, open Manage Tivoli Monitoring Services right-click the portal server, and select Reconfigure. problems include: v Portal server database user password is out of sync. v User ID does not match the operating system logon user ID. 16 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 35
reconfigure the portal server, run the tacmd configureportalserver command. If the problem persists, take one or more of the following steps: - of the db2admin local account: 1. Click Control Panel > Administrative Tools > Services. 2. Right-click DB2 - DB2 and select Properties. 3. Select the - IBM E027SLL-H | Troubleshooting Guide - Page 36
ibm.com/infocenter/tivihelp/ v15r1/topic/com.ibm.itm.doc_6.2.3fp1/itm623_messages.htm). 2. Look in the portal server or portal client logs for more information concerning the message. The portal client does not respond You can encounter a problem . 18 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 37
to the screen. Procedure Disable DirectDraw by setting the sun.java2d.noddraw client variable to false. See the "Portal client configuration settings" topics in the IBM Tivoli Monitoring Administrator's Guide. Chapter 3. Common problem solving 19 - IBM E027SLL-H | Troubleshooting Guide - Page 38
more information on actions that relate to these diagnostics, see the problem resolution tasks. See also the troubleshooting topics in the IBM Tivoli Warehouse Proxy Agent User's Guide (http://publib.boulder.ibm.com/infocenter/tivihelp/ v15r1/topic/com.ibm.itm.doc_6.2.3fp1/wpa/wpagent_user.htm) and - IBM E027SLL-H | Troubleshooting Guide - Page 39
. See the "Controlling port number assignments" topics in the IBM Tivoli Monitoring Installation and Setup Guide for more information on the COUNT and SKIP options for port number allocation. Resolving warehouse proxy agent problems - configuration If the Warehouse Proxy agent is connected to the - IBM E027SLL-H | Troubleshooting Guide - Page 40
behavior leads to unexpected data. About this task To resolve summarization and pruning agent problems, you perform diagnostic and corrective actions. These actions include comparing real-time data from a workspace view with the unexpected data. 22 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 41
see Tivoli Management Services Warehouse and Reporting (http:// www.redbooks.ibm.com/abstracts/sg247290.html). This IBM Redbooks publication describes aggregation About this task To resolve persistent data store for z/OS problems, you perform diagnostic and corrective actions. These actions include - IBM E027SLL-H | Troubleshooting Guide - Page 42
allow the slot to be read. One possible cause is a problem with DELETE processing. The warehouse code, which is the only code The persistent data store cannot span a data row across multiple blocks. One other possibility is that either the API calls to the IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 43
IBM Tivoli OMEGAMON XE and Tivoli Management Services on z/OS: Common Planning and Configuration Guide. Historical data does not get collected for some monitoring server attribute groups Problem limitation in the current implementation of ITM history collection for these attribute groups that should - IBM E027SLL-H | Troubleshooting Guide - Page 44
check that the situation opened an event. About this task To resolve situation-specific problems, you perform diagnostic and corrective actions. These actions include checking if the agent .0034-1114:kpxreqds.cpp,661,"determineTargetsFromNodelist") 26 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 45
situation authored correctly? At the agent, trace (UNIT:kdsfilt all). a. Yes: The problem might be related to the monitoring agent. See the Troubleshooting appendix of the distributed agent's User's Guide or the Troubleshooting Guide of the z/OS monitoring agent. b. No: See step 5. 5. Look in the - IBM E027SLL-H | Troubleshooting Guide - Page 46
data collection for troubleshooting," on page 5 for information on what types of data to collect before contacting Support. Also consult the IBM Support Portal (http://www.ibm.com/support/entry/portal/software). A reflex automation script does not run when it should You can encounter a problem that - IBM E027SLL-H | Troubleshooting Guide - Page 47
application, or process running on the system might be the cause of the problem. Also, you can use the ITMSUPER tools, such as the Connectivity the following steps: Procedure Preliminary diagnostics 1. Determine whether an IBM Tivoli Monitoring component is the root cause. Another application or - IBM E027SLL-H | Troubleshooting Guide - Page 48
definition and formula. 5. Does the situation contain any wildcard * characters against UTF8 columns? v Yes: See "Resolving situation problems - corrective actions" on page 31. 6. Switch between situations to see which one is causing the high CPU. 30 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 49
log to verify whether the monitoring agent has made a connection to the monitoring server. See "Setting traces" on page 43 for more information. Resolving firewall problems - corrective actions You can resolve firewall problems by contacting IBM Software Support. Chapter 3. Common problem solving 31 - IBM E027SLL-H | Troubleshooting Guide - Page 50
across firewalls, open a problem report with IBM Software Support or refer to the IBM Support Portal (http://www.ibm.com/support/entry/portal/ software). 2. For more information, see the "Firewalls" topics in the IBM Tivoli Monitoring Installation and Setup Guide. Resolving Oracle DB Agent - IBM E027SLL-H | Troubleshooting Guide - Page 51
by setting an environment variable: v COLL_DISABLE_CURSORS a. Launch Manage Tivoli Enterprise Monitoring Services. b. Right-click the row that contains the name of the monitoring agent : Set FREEBYTES to zero Navigation Tree : Databases->Database Summary Chapter 3. Common problem solving 33 - IBM E027SLL-H | Troubleshooting Guide - Page 52
true Oracle_SystemTS_PctFree_Warning = always false 5. For more information on the cursors, see the Oracle Agent 6.2.0-TIV-ITM_ORA-LA0001 README or a higher version of the README. 34 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 53
when component software fails to operate as intended. IBM Software Support uses the information captured by trace logs to trace a problem to its source or to determine why an error occurred. The principal log type is the reliability, availability, and serviceability (RAS) trace log. RAS logs are in - IBM E027SLL-H | Troubleshooting Guide - Page 54
/deployment/log/javawsnnnnn.trace where 'nnnnn' is a unique, randomly generated numeric suffix to support generational logs (for example, the last generated log will not be overlayed by the most fixed name and is overlayed with each execution cycle. 36 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 55
Table 1. Location of log files for the IBM Tivoli Monitoring components. (continued) Component Windows UNIX-based systems Tivoli Enterprise Monitoring Server install_dir\logs\hostnamePC_ HEXtimestamp-nn.log install_dir/logs/hostname_PC_timestamp.log where: where: - IBM E027SLL-H | Troubleshooting Guide - Page 56
.log agent where Not supported. PC Specifies the product code. hd is the product code for the IBM Tivoli Warehouse Proxy agent IBM Tivoli Summarization and Pruning the first is always retained, since it includes configuration parameters. 38 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 57
>.log This log is created if an abort occurs for either a first time installation or a modification of previous installation of IBM Tivoli Monitoring. v ITM_HOME\InstallITM\_.log This log is created during a normal clean installation. v ITM_HOME\InstallITM\MOD_ - IBM E027SLL-H | Troubleshooting Guide - Page 58
. For tracing and logging Java code (that is run on UNIX systems), this mechanism enables problem debugging. Two sets of information are created - logs and traces. Logs (*.log) are globalized v CH/config/ITMInstallRAS.properties (for installation) 40 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 59
install(config)/ plugin_type/pc callpoint.trc (.log) *.stderr *.stdout Tivoli Distributed Monitoring upgrade log file All upgrade actions performed by the IBM Tivoli Monitoring Upgrade Toolkit are recorded in a central log with an associated user ID and a time stamp. Upgrade actions taken outside - IBM E027SLL-H | Troubleshooting Guide - Page 60
2005.07.21 15:30:24.199 CST YFELDMA1.austin.ibm.com AMXAMX ScanTMR 1 - IBM E027SLL-H | Troubleshooting Guide - Page 61
") ODBC source table is now online to the data provider" Setting traces When you encounter an error with IBM Tivoli Monitoring that requires contacting IBM Software Support, you might be asked to submit a copy of the error log. The error log is part of the trace - IBM E027SLL-H | Troubleshooting Guide - Page 62
select File > Trace Options. 3. Select a trace class from the list or as instructed by IBM Software Support (such as UNIT:Workspace ALL): v ALL provides data for all classes. Use the Portal Server through Manage Tivoli Enterprise Monitoring Services. 44 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 63
where the Tivoli Enterprise Monitoring Server is installed, select Start > Programs > IBM Tivoli Monitoring > Manage Tivoli Enterprise Monitoring Services. 3. Right-click the Tivoli Enterprise Monitoring Server service. 4. Select Advanced > Edit Trace Parms to display the Trace Parameters window - IBM E027SLL-H | Troubleshooting Guide - Page 64
of the fields. 6. Click OK to set the new trace options. 7. Click Yes to recycle the service. On Linux systems, set the following variable in $CANDLEHOME/config/lz.ini: KBB_RAS1=ERROR(UNIT:kdy includes tracing of system calls during SNMP processing. 46 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 65
you must obtain the RAS1 log that matches the problem occurrence before contacting IBM Software Support. You can set tracing options for individual universal OK to set the new trace options. 6. Click Yes to recycle the service. Setting the trace option for the Warehouse Proxy agent Procedure 1. On - IBM E027SLL-H | Troubleshooting Guide - Page 66
trace option for the Summarization and Pruning Agent Use the IBM Tivoli Universal Agent trace facility to diagnose problems with the Summarization and Pruning Agent. See "Setting the the Java-based internal tracing (hostname_sy_java_timestamp-n.log). 48 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 67
KBB_RAS1=ERROR (UNIT:ksy4 ALL) ResultSet level trace for summarization. Affects the content of the Java-based internal tracing (hostname_sy_java_timestamp-n.log). KBB_RAS1=ERROR (UNIT:ksy5 ALL) Column value level trace for summarization. Affects the content of the Java-based internal tracing ( - IBM E027SLL-H | Troubleshooting Guide - Page 68
:ksh all) (UNIT:kui all) . On UNIX systems, manually edit the $CANDLEHOME/bin/tacmd shell script to add a line IBM Tivoli Monitoring upgrade toolkit Table 4. Setting the trace option for the Tivoli Monitoring upgrade toolkit Trace option Instructions IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 69
monitoring environment is configured for IBM Tivoli Monitoring event forwarding, you the event forwarding trace facility to diagnose problems with event forwarding. About this task the following instructions to set the trace levels: : 1. In Manage Tivoli Enterprise Monitoring Services, right-click - IBM E027SLL-H | Troubleshooting Guide - Page 70
IBM Tivoli Enterprise Console Situation Update Forwarder trace facility to diagnose problems with the IBM Tivoli Enterprise Console Situation Update Forwarder. The trace for the IBM update fileName=configuration_file_name logLevel=trace_level where: 52 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 71
individual monitoring agent's user's guide. Dynamically modify trace settings for an IBM Tivoli Monitoring component You can access log-on credential for the system. This method uses the IBM Tivoli Monitoring Service Console. The Service Console is accessed using a web browser. Access the utility - IBM E027SLL-H | Troubleshooting Guide - Page 72
the ras1 list command. OUTPUT - Shown as OUT when using the ras1 list command. State - Shown as ST when using the ras1 list command. 54 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 73
more specific setting, use it. ALL can sometimes be necessary when isolating a problem. It is the equivalent of setting "Error Detail Flow State Input Output Metrics (UNIT:kbbcrcd ANY) Using the IBM Tivoli Monitoring Service Console The IBM Tivoli Monitoring Service Console enables you to read logs - IBM E027SLL-H | Troubleshooting Guide - Page 74
Tivoli Enterprise Portal Server and enables users to access the IBM Tivoli Monitoring Service Console. You can prevent users from accessing the service console that is available through the integral web server (http:// portal_server_host_name:1920). 56 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 75
To block access to the service console, disable the integral web server. However, if you disable the with IBM Software Support for gathering diagnostic information. Procedure 1. From your Windows desktop, select Start > Programs > IBM Tivoli Monitoring > Manage Tivoli Enterprise Monitoring Services. - IBM E027SLL-H | Troubleshooting Guide - Page 76
name from kfwquery where id like 'zkum.%%';" Clearing the JAR cache If you encounter problems, IBM Software Support might instruct you to uninstall and to clear the Java archive (JAR) cache. Procedure 1. If opens and prompt you to enter a user ID. 58 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 77
pdcollect tool Use the pdcollect tool to collect the most commonly used information from a system. Technicians in IBM Software Support use this information to investigate a problem. The pdcollect tool is used to gather log files, configuration information, version information, and other information - IBM E027SLL-H | Troubleshooting Guide - Page 78
upgrade the Tivoli Enterprise Portal Server, the Tivoli Enterprise Services User Interface Extensions software is automatically installed in the same your configuration with the IBM Tivoli Monitoring distributed utility, Install: BackspaceCheckUtility. 60 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 79
replaces MSG2 logging. With MSG2 logging, physical space problems can occur due to MSG2 logs that grow without bound until a process is stopped. IBM Tivoli Monitoring Operations Logging enables you to configure log file management to avoid these problems. Note: This functionality is not supported - IBM E027SLL-H | Troubleshooting Guide - Page 80
depends on the amount of space allocated at the time each dataset is created. To use IBM Tivoli Monitoring Operations Logging on z/OS systems, complete the following procedure: 1. Create the log skipped, subsequent OpsLog DD statements are ignored. 62 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 81
installed. The ITMSUPER Tools are included in the IBM Support Assistant (ISA), a free local software serviceability workbench that helps you resolve questions and problems with IBM software products. See IBM Support Assistant (http://www-01.ibm.com/ software/support/isa). Chapter 4. Tools 63 - IBM E027SLL-H | Troubleshooting Guide - Page 82
64 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 83
problems and resolutions" on page 84 v "UNIX-based system installation problems and resolutions" on page 88 v "Troubleshooting z/OS-based installations" on page 96 v "Uninstallation problems each monitoring agent is migrated to IBM Tivoli Monitoring. Note: The IBM Tivoli Monitoring download image or - IBM E027SLL-H | Troubleshooting Guide - Page 84
file to IBM Software Support for analysis. UNIX-based systems installation frequently asked questions Table 7. Frequently asked questions for UNIX-based systems installation Problem Solution The product center or HP Open-view, SNMP is required. 66 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 85
for UNIX-based systems installation (continued) Problem Solution Pressing the backspace key, characters Consider using "^?" as the key code. Verify your configuration with the IBM Tivoli Monitoring distributed utility, Install: BackspaceCheckUtility. When running the install.sh troubleshooting 67 - IBM E027SLL-H | Troubleshooting Guide - Page 86
support files are loaded without causing the TEMS to restart. After that, if you run the tacmd listapplinstallrecs command to verify the application support installation, the support is not listed in the command output. As a result, a lower version 68 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 87
created by the Startup Center, create the DSN manually by using the Warehouse Proxy Agent configuration instructions. See "Configuring a Warehouse Proxy Agent on Windows (ODBC connection)" in the IBM Tivoli Monitoring Installation and Setup Guide. For more information, check the WAREHOUSE_ODBC.log - IBM E027SLL-H | Troubleshooting Guide - Page 88
be discovered, you are given the opportunity to categorize the system manually in a later step. When you assign systems to the components, installation and configuration troubleshooting topics for help with monitoring agent problems that occur during IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 89
endpoint. 2. Uninstall the Upgrade Toolkit. 3. Clean the endpoint manually. 4. Reinstall the Upgrade Toolkit. 5. Attempt to upgrade the causing this problem: 1. Open the status.properties, 2. Look for an entry like the following example: #Copyright IBM Corporation and configuration troubleshooting 71 - IBM E027SLL-H | Troubleshooting Guide - Page 90
an agent: The return code SQL1_OpenRequest status = 79 occurs in the agent log when the application support is added during an upgrade. This return code results from an attempt to delete a table entry to a new version using remote deployment. 72 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 91
the DB2 agent connection is to RTEMS 2 rather than RTEMS 1. The same problem occurs when agents at V6.2.3 or earlier are updated using group deploy or Seeding support for Monitoring Agent for Microsoft SharePoint Server [8 of 10] KCIIN1602E ERROR - file not found: /boadata/IBM/ troubleshooting 73 - IBM E027SLL-H | Troubleshooting Guide - Page 92
a version not supported by IBM Tivoli Monitoring V6 IBM Tivoli Monitoring V6.2.1 (or later) monitoring agents on the managed system by running kinconfig.exe -rKproductcode. v OR v Install another IBM Tivoli Monitoring V6.2.1 monitoring agent (or later). 74 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 93
to resolve the problems. Where Remote Deployment of agents is not supported Remote Deployment is not supported for OMEGAMON agents. It is also not supported in environments agent does not support. This issue was noticed for the DB2 agent. Chapter 5. Installation and configuration troubleshooting 75 - IBM E027SLL-H | Troubleshooting Guide - Page 94
support files contained in the support package are not installed. Using the Application Support Installer with the Silent Installation Response file option is not supported. The recommended mechanism for the installation is using the GUI interface. 76 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 95
On Windows systems: 1. Run cmd. 2. Get the IBM Java location by running the following script: CANDLEHOME\InstallITM\GetJavaHome. bundle on a 2.6 kernel-level system). Look at the logs (Software Package Block (SPB) logs are located in the temporary directory of the system, /tmp for troubleshooting 77 - IBM E027SLL-H | Troubleshooting Guide - Page 96
. Problems with supported. When attempting to install IBM Java 1.5.0 on Windows 64 bit system nothing happens Only 32-bit browsers are supported on the AMD 64 Windows environment due to the lack of a native 64-bit Web Start or Java Plug-in supports. 78 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 97
the currently installed GSKit product (for example, C:\ibm\gsk7). v When the error is reported, manually configure the Monitoring Agent for DB2 Service Startup Parameters to use the correct user name and and you will receive this message: Chapter 5. Installation and configuration troubleshooting 79 - IBM E027SLL-H | Troubleshooting Guide - Page 98
of the problem. Table 8. lcfd log file Windows install_dir/Install/Abort IBM Tivoli Monitoring timeStamp.log UNIX-based systems install_dir/logs/candle_installation.log Contact IBM Software Support if you cannot install the agent. See Chapter 2, "Logs and data collection for troubleshooting," on - IBM E027SLL-H | Troubleshooting Guide - Page 99
-standby feature, see the IBM Tivoli Monitoring High-Availability Guide for Distributed Systems). You should refrain from deploying or updating agents when IBM Tivoli Monitoring is converting to reached, migration of the table is aborted. Chapter 5. Installation and configuration troubleshooting 81 - IBM E027SLL-H | Troubleshooting Guide - Page 100
the replacement of support packages required for self-describing mode. Following error(s) detected: list of error(s) Self-describing mode for agent name is not enabled. When the problem is fixed, reinstall agent name to enable self-describing mode. 82 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 101
with installation self-describing mode support. If the reinstallation fails, Serverrequire you to manually use commands messages to identify the specific problems in the monitoring server installation about these commands, see the IBM Tivoli Monitoring Command Reference. To troubleshooting 83 - IBM E027SLL-H | Troubleshooting Guide - Page 102
mechanism to install the new monitoring server support across your environment. See "Adding application support to a monitoring server on z/OS" in the Configuring IBM Tivoli Enterprise Monitoring Server on z/OS guide. Windows installation problems and resolutions On Windows systems, the installation - IBM E027SLL-H | Troubleshooting Guide - Page 103
hard drive. Problems that are cleared by rebooting the Windows system There are a set of problems continue' message is found in the Abort IBM Tivoli Monitoring for Databases.log file. Windows Server 2008, there are a number of manual steps and workarounds that must be performed: 1. troubleshooting 85 - IBM E027SLL-H | Troubleshooting Guide - Page 104
from the Windows task manager and then attempt the manual start. Your Tivoli Enterprise Portal Server can then be installation cannot complete. To install the software without this problem, map the network drive and run the setup.exe file from a DOS IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 105
a local drive if you continue to receive the error. Installation on a Windows 2003 server fails with Error Number: 0x80040707 The following error can occur while installing IBM Tivoli Monitoring V6.1 on a Windows 2003 Server: This error can occur for the following reasons: 1. Windows Service Pack - IBM E027SLL-H | Troubleshooting Guide - Page 106
further assistance resolving the error, contact IBM Software Support.The agent received incorrect SQL.The problems and resolutions Review the symptoms and possible resolutions to solve IBM Tivoli Monitoringinstallation problems on Linux or UNIX system. 88 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 107
support v The product application support installation (seeding) step at a lower version of IBM Tivoli Monitoring than the Tivoli of application support to the support for all base monitoring agents and other supported support product NT was manually installed at the hub (STATE column is - IBM E027SLL-H | Troubleshooting Guide - Page 108
higher level of application support already on a server Resolving the problem v Run the manual product seeding IBM Tivoli Monitoring Installation and Setup Guide and "Self-describing monitoring agents" in the IBM Tivoli Monitoring Administrator's Guide. 90 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 109
IBM/ITM/li6243/gs/bin/private_verifyinstall ----/opt/IBM Details of gskit in /opt/IBM/ITM/li6243/gs Got runGSkit: script: /opt/IBM/ITM/li6243/gs/ into /opt/IBM/ITM/tmp/badGSKit_keep /tables/TEMS/depot/PACKAGES/unix/kci//install.sh Replace the following part: if [ -n "$fieldSep" ] - IBM E027SLL-H | Troubleshooting Guide - Page 110
://www-01.ibm.com/support/docview.wss?uid Services does not start on AIX V6.1 If you get a Java exception after invoking itmcmd manage to start Manage Tivoli Enterprise Monitoring Services on a 64-bit AIX V6.1 system, install operating system fix packs. 92 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 111
Services can be started: v 6100-00 - AIX APAR IZ16878 v 6100-01 - AIX APAR IZ16847 The updates are available from IBM Support: AIX® V6.1 Fix Pack information for 64-bit systems (http://www-304.ibm.com/support package is installed, the problem no longer occurs. Receive KUIC02101W troubleshooting 93 - IBM E027SLL-H | Troubleshooting Guide - Page 112
the nodename and the host name are not in sync then the you must shutdown all IBM Tivoli Monitoring processes before starting an addition product installation or upgrade. EIF Slot Customization does as the instance user: $ su - db2inst1 $ db2start 94 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 113
. The problem can be manually recreated by problem with starting Java Virtual Machine" If you try to install IBM Tivoli Monitoring on Red Hat 5 with SELinux set to "permissive" or "disabled" mode ("enforced" mode is not supported by IBM Tivoli Chapter 5. Installation and configuration troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 114
checklist Use the IBM Tivoli Monitoring z/OS initialization checklist to troubleshoot problems with your Tivoli Monitoring installation on z/OS. The z/OS initialization checklist includes the following sections: v "Tivoli Monitoring Services Engine initialization" v "RAS1 service initialization" on - IBM E027SLL-H | Troubleshooting Guide - Page 115
service initialization The Reliability, Availability and Servicability (RAS1) service refers to the RAS1 building block (Basic Services component) used for diagnostic tracing. Nearly all diagnostic information for IBM TCP/IP service failed. Chapter 5. Installation and configuration troubleshooting 97 - IBM E027SLL-H | Troubleshooting Guide - Page 116
ACF2) permission for the OMVS segment to use TCP/IP services. Failure to grant this permission (which is granted to problems, you can use console operator command SETOMVS MAXPROCUSER=xxx to increase the current MAXPROCUSER value (as seen by D OMVS,O). 98 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 117
task JCL of the IBM Tivoli Monitoring task (pointing to a usable file in USER.PARMLIB(TCPDATA)), resolver support can be enabled. This can also result in TCP/IP service initialization failure for the IBM Tivoli Monitoring address space. The absence of Installation and configuration troubleshooting 99 - IBM E027SLL-H | Troubleshooting Guide - Page 118
similar problem results service initialization The IBM Tivoli Protocol" manual. v IBM Tivoli Monitoring processes build and query a list of possible Tivoli Enterprise Monitoring Server hub addresses, called the Server list. This server list contains 100 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 119
is not the case, then there might be a mismatch between the number of transports services configured to this Tivoli Enterprise Monitoring Server (the KDC_FAMILIES environment variable) versus the number of of location brokers: the local Chapter 5. Installation and configuration troubleshooting 101 - IBM E027SLL-H | Troubleshooting Guide - Page 120
).135. KDSNC007 Local Location Broker is active Repairing errors in Local Location Broker service initialization: Bind failures due to insufficient authorization are reported with Errno. 2 ( segment to the Omegamon Platform started task name. 102 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 121
).135. Global Location Broker is active Repairing errors in Global Location Broker service initialization: GLB service failures occur because there are errors in member KDCSSITE of RKANPAR. Each Server to ensure it is connected. Chapter 5. Installation and configuration troubleshooting 103 - IBM E027SLL-H | Troubleshooting Guide - Page 122
describes problems that might occur with a z/OS-based installation. How do you change monitoring server storage parameters using the Configuration Tool? About this task You can increase the following storage-related parameters if IBM Software Support personnel instructs you to do so: v Web Services - IBM E027SLL-H | Troubleshooting Guide - Page 123
the B option next to the RTE on the KCIPRTE RTE main menu to instruct Configuration tool generates the pp#1xxxx mRTE Build job. 2. Ensure that the on z/OS (http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/ com.ibm.itm.doc_6.2.3fp1/itm623fp1_zconfigtems.htm). troubleshooting 105 - IBM E027SLL-H | Troubleshooting Guide - Page 124
error that the transfer failed, take steps to resolve the error. To confirm the problem, enter the following command in the logs directory on the portal server: grep "PORT > Integrated Cryptographic Service Facility (ICSF) installed?and specify N(No). 106 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 125
the portal server if it is running and restart it. Backing up the ICAT and CSI environment Manually merge existing datasets with IBM Tivoli Monitoring datasets if IBM Software Support instructs you to do so. Procedure 1. Back up all of the following datasets: v Runtime v Target v INSTDATA v INSTDATW - IBM E027SLL-H | Troubleshooting Guide - Page 126
support files, the IBM Tivoli Monitoring Online (F1) Help produces an error when selected. The browser displays "The page cannot be displayed" with a URL of http://host:9999/help/index.jsp. To solve this problem stopped and started for this procedure. 108 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 127
is no entry, proceed to the next step. 2. Open the Windows Explorer and navigate to the IBM Tivoli Monitoring installation directory (C:\IBM\ITM by default). 3. Launch the Manage Tivoli Enterprise Monitoring Services utility by double-clicking the KinConfg.exe file located in either the Install or - IBM E027SLL-H | Troubleshooting Guide - Page 128
the instructions in IBM Tivoli Monitoring without attempting to upgrade. If the entry exists in the Add and Remove Programs tool, you can still use your existing Candle OMEGAMON installation. Launch Manage Candle Services to start all components. 110 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 129
Services. 6. Verify that all related Candle OMEGAMON and IBM Tivoli Monitoring services have been removed. These services match those listed in the Manage Tivoli Enterprise Monitoring Services window Delete. 13. Close the Registry Editor. Chapter 5. Installation and configuration troubleshooting 111 - IBM E027SLL-H | Troubleshooting Guide - Page 130
to uninstall the release again. If it fails again, contact IBM Software Support for assistance. See Chapter 2, "Logs and data collection for troubleshooting," on page 5 for information on what types of data to collect before contacting IBM Support. Both products are available in the Add and Remove - IBM E027SLL-H | Troubleshooting Guide - Page 131
install_Dir/Install/Abort IBM Tivoli Monitoring timeStamp.log install_Dir/logs/candle_installation.log You can manually uninstall the operating system agent by running the command for your platform as listed in Table 11 on page 114: Chapter 5. Installation and configuration troubleshooting 113 - IBM E027SLL-H | Troubleshooting Guide - Page 132
and data collection for troubleshooting," on page 5 for information on what types of data to collect before contacting Support. See the IBM Support Portal (http://www.ibm.com/support/entry/portal/software). will be an attempt to remove the database. 114 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 133
Chapter 6. Connectivity troubleshooting Review the connectivity troubleshooting topics for problems you might experience with logging in, passwords, and communication among IBM Tivoli Monitoring components. When the Tivoli Enterprise Portal detects a connection error, it can repair the error and - IBM E027SLL-H | Troubleshooting Guide - Page 134
Guide. Start DB2 or wait for DB2 to finish initializing. If you receive message KFWITM009I you can look at the most recent trace log to verify that the portal server is initialized by searching for the text string Waiting for requests. Startup completed. 116 IBM Tivoli Monitoring: Troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 135
the reason for this procedure being applied is LDAP being out of service, ensure you also change the monitoring server configuration to not authenticate problems and resolutions when you have trouble with the connection to the Tivoli Enterprise Portal Server. Chapter 6. Connectivity troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 136
Cannot connect to Tivoli Enterprise Portal Server Problem Resolution KFWITM001W Unable to connect to Tivoli Enterprise the failure: v SQL1224N A database agent could not be started to service a request, or was terminated as a result of a database system IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 137
Tivoli Enterprise Portal Server (continued) Problem Resolution A remote Tivoli Enterprise Portal client IBM Tivoli Monitoring > Manage Tivoli Enterprise cannot find the Tivoli Enterprise Monitoring Services. Portal Server 2. Check that the portal server service Connectivity troubleshooting 119 - IBM E027SLL-H | Troubleshooting Guide - Page 138
(continued) Problem Resolution Cannot System Account and place a check in the Allow Service to Interact with Desktop box. 4. Click OK to to connect, contact IBM Software Support. See Chapter 2, "Logs and data collection for troubleshooting," on page 5 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 139
the programs section of your firewall software and include the IP addresses of other IBM Tivoli Monitoring components in the access control for the firewall. Portal server is initializing server stopped or is recycling. See also KFW_CMW_RECYCLE_DELAY. Chapter 6. Connectivity troubleshooting 121 - IBM E027SLL-H | Troubleshooting Guide - Page 140
cannot open a workspace in the Tivoli Enterprise Portal, verify that the database agent and manager are working correctly. The configuration settings might need updating. 122 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 141
of a key database manager process. If the problem is not due to any of the above, services. 6. Start and stop DB2: % db2stop % db2start What to do next Restart the portal client. A monitoring process fails to start on Linux or UNIX after changing a .profile for root IBM troubleshooting 123 - IBM E027SLL-H | Troubleshooting Guide - Page 142
that address this problem. Search on "linux guest clock" in the VMware Knowledge Base (http://kb.vmware.com/selfservice/microsites/microsite.do). See also IBM Service Management Connect (http this is less than the real world time, this means that 124 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 143
the Linux guest clock is running slow. This causes the IBM Tivoli Monitoring product to behave erratically if the clock is not fixed. Chapter 6. Connectivity troubleshooting 125 - IBM E027SLL-H | Troubleshooting Guide - Page 144
126 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 145
are supported. JavaWebStart Tivoli Enterprise Portal fails to display help screens The kjr.browser.default value must be set on non-Windows environments. For more information, see the Web Start: Specifying the browser location subsection in the IBM Tivoli Monitoring Installation and Setup Guide - IBM E027SLL-H | Troubleshooting Guide - Page 146
steps: 1. Click Start > Programs > IBM Tivoli Monitoring > Manage Tivoli Monitoring Services. 2. In the Manage Tivoli Monitoring Services window, right-click Tivoli Enterprise Portal > the non-ASCII characters from distinguished name or the user ID. 128 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 147
Locate the install_dir/architecture/cj/bin/cnp.sh file on the Linux system. 2. Export TEP_JAVA_HOME=/opt/ibm/java-i386-60/. 3. Export KCJ_HOME=... . Some attribute groups showing a different name in the Tivoli on the host name instead of the IP address: Chapter 7. Portal client troubleshooting 129 - IBM E027SLL-H | Troubleshooting Guide - Page 148
to the TCP/IP host name, but it is replaced if CTIRA_HOSTNAME is set. This gives you control over deciding where an agent is positioned. Configuring an agent to have a specific CTIRA_HOSTNAME value manages cases where the IP address changes, and then 130 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 149
adding agents support for Tivoli Enterprise Portal Server Launch the Manage Tivoli Enterprise Monitoring Services utility on CandleGetJavaHome script was not found. Call IBM Software Support. To resolve this problem launch the CandleGetJavaHome.bat script located Portal client troubleshooting 131 - IBM E027SLL-H | Troubleshooting Guide - Page 150
settings failed. (Access is denied. ) Follow the instructions on theMicrosoft Support (http://support.microsoft.com/) website to resolve this error in your environment before proceeding. on the left, select Local Policies -> User Rights Assignments. 132 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 151
, the Tivoli Enterprise Portal client exits. If you encounter this problem, use any recent Sun JRE (1.5.0_14 or above). This problem is limited to IBM's 1.5 java. Note: A Sun bug report bug_id=6578895 which describes the issue was fixed in Sun's 1.5.0_14. Chapter 7. Portal client troubleshooting 133 - IBM E027SLL-H | Troubleshooting Guide - Page 152
"On Linux, IBM Tivoli Enterprise Monitoring Agent topics do not display in the Help Contents or Index tabs" on page 135. Java exception logging onto the Tivoli Enterprise Portal from a browser If you encounter an exception that has the following text: 134 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 153
back in and change it manually. HEAPDUMPs and JAVACore files are placed on IBM Java Plug-in, which is automatically installed on your computer with the portal client. Adjust the IBM Java Plug-in properties if performance is slow or your workstation receives Chapter 7. Portal client troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 154
size to 128 MB. Set the maximum heap size to 256 MB. If you continue to experience problems, increase the maximum setting in increments of 64 MB until the symptoms disappear. v When memory c. Click OK. 6. Click OK in the Java(TM) Control Panel. 136 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 155
the 32bit_jre_install_dir/bin/javacpl.exe file. Java errors occur with the IBM v1.4.2 JRE If you experience any problems using the IBM 1.4.2 JRE use the version shipped with the Tivoli Enterprise Portal v6.1. IBM Tivoli Monitoring v6.2 ships with IBM 1.5.0 JRE. Use this version if you have installed - IBM E027SLL-H | Troubleshooting Guide - Page 156
among these are the minimum free percentage (-Xminf) and maximum free percentage (-Xmaxf) parameters. IBM Java documentation provides the following descriptions: -Xminf A floating point number, 0 through 1, that specifies the minimum free heap 138 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 157
IBM Developer Kit and Runtime Environment, Java 2 Technology Edition, Version 1.4.2 Diagnostics Guide (http://publib.boulder.ibm.com/infocenter/javasdk/ v1r4m2/topic/com.ibm VMWare does not support DirectDraw. Perform Manage Tivoli Enterprise Monitoring Services to set Java Runtime troubleshooting 139 - IBM E027SLL-H | Troubleshooting Guide - Page 158
About this task If you did not enable Manage Tivoli Enterprise Monitoring Services during installation, the Workflow Editor is disabled and the following tools these history situations were automatically defined by an IBM Tivoli Monitoring component. 140 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 159
application support, product-provided situations do not appear in the Tivoli Enterprise Portal Situation editor or do not auto start. This problem occurs (ITM Tivoli Monitoring Services or an OMEGAMON XE monitoring agent) on the PRODUCT SELECTION MENU. Chapter 7. Portal client troubleshooting 141 - IBM E027SLL-H | Troubleshooting Guide - Page 160
on z/OS, refer to the configuration guide for your monitoring agent. For instructions on installing application support for monitoring agents installed on a distributed system (Windows, UNIX, Linux) see the IBM Tivoli Monitoring: Installation and Setup Guide. kshsoap client fails because of missing - IBM E027SLL-H | Troubleshooting Guide - Page 161
76 characters] is: Error line(391): Unable to delete file: C:\IBM\ITM\tmaitm6\KUDCMA_DB2.exe (4300DFCE.0003-828:kdytasks.cpp,3291,"setConfig") that occur at the same time are loaded too slowly Manually set the variable KFW_CMW_EVENT_SLEEP in cq.ini on Linux or Portal client troubleshooting 143 - IBM E027SLL-H | Troubleshooting Guide - Page 162
9. Recycle Tivoli Enterprise Portal Server service. Existing OMEGAMON product imagery displays after upgrading to IBM Tivoli Monitoring V6.1 About this task This feature is not supported in environments with a z/OS Tivoli Enterprise Monitoring Server. 144 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 163
the application text. If no localized version of a style sheet, such as ws_press.css, is available, the English version will be used. Chapter 7. Portal client troubleshooting 145 - IBM E027SLL-H | Troubleshooting Guide - Page 164
146 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 165
do not find the resolution to your problem, review the topics in Chapter 6, "Connectivity troubleshooting," on page 115. Performance impacts of CPU consumption for the HTTP and HTTPS protocols. Users who run the IBM HTTP Server do not have permission to the content directory During configuration and - IBM E027SLL-H | Troubleshooting Guide - Page 166
SysWOW64\odbccp32.cpl and manually create the "ITM Services Discovery Library Adapter, results in a book that does not contain the fully qualified host name Edit the resulting xml file and change the shortname to the fully qualified host name. 148 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 167
you are asked if you want to recycle the service. Cannot create a Tivoli Enterprise Portal Server database ibm.com/infocenter/db2luw/v9/ topic/com.ibm.db2.udb.uprun.doc/doc/t0008238.htm) in the IBM DB2 Database for Linux, UNIX, and Windows Information Center. Chapter 8. Portal server troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 168
are assigned to a query for a workspace view. The best practice for resolving this problem is as follows: 1. Create a managed system list that specifies the explicitly defined managed of managed systems that you explicitly define in the query. 150 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 169
Enterprise Portal Server - as a server component (as opposed to IBM Tivoli Monitoring v6.1, where the Eclipse Help Server was treated as Eclipse Help Server. Non-root stopping or starting agents causes problems You might experience issues while starting or stopping agents on troubleshooting 151 - IBM E027SLL-H | Troubleshooting Guide - Page 170
internal error, indicated by KFWITM392E Internal error occurred during login. If you have already used the eWAS commands, see "Starting and stopping eWAS" in the IBM Tivoli Monitoring Administrator's Guide for recovery instructions. 152 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 171
. Troubleshooting monitoring server problems on distributed systems The problems described in this section might occur on distributed systems. For information about configuring theTivoli Enterprise Monitoring Server, refer to the IBM Tivoli Monitoring Installation and Setup Guide . © Copyright IBM - IBM E027SLL-H | Troubleshooting Guide - Page 172
, all versions of IBM Tivoli Monitoring incorporating the IBM Tivoli Directory Server Client at the monitoring server are exposed to an unaudited security risk of exposure of passwords in the clear. To avoid displaying passwords in the clear when troubleshooting LDAP problems, use the following - IBM E027SLL-H | Troubleshooting Guide - Page 173
. 3. Restart the hub monitoring server. 4. Restart the modified Warehouse Proxy Agents. Upgrade inconsistency between the History and Object windows The historical collections created in previous versions of IBM Tivoli Monitoring are not eligible to be members of a group in a later version of the - IBM E027SLL-H | Troubleshooting Guide - Page 174
server parameter for this purpose. For all other purposes, PAM is not supported by adding the parameter KDS_VALIDATE_EXT=Y to a monitoring server configuration. Remote Tivoli cause a subsequent start of Tivoli Enterprise Monitoring Server to fail. 156 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 175
during product testing. The IBM Tivoli Monitoring configuration tool checks the output produced by the seeding functions, and it reports that the monitoring server application support was added successfully. if a core or dump file is not produced. Chapter 9. Monitoring server troubleshooting 157 - IBM E027SLL-H | Troubleshooting Guide - Page 176
.log where : Node The system host name Process_ID The program process ID Windows example: C:\IBM\ITM\CNPS\logs\seedApp.log where: App The 3-character product code, such as knt for exported, then the settings will be active again after restart. 158 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 177
process runs on the first IBM Tivoli Monitoring process started up. If that first process stops, the web server swaps to another IBM Tivoli Monitoring process. The web server supports the service console and port forwarding logic as well as SOAP. Chapter 9. Monitoring server troubleshooting 159 - IBM E027SLL-H | Troubleshooting Guide - Page 178
as the IBM Tivoli Monitoring Universal Agent or the Monitoring Agent for Unix OS. When the problem occurs, both instances of the hub monitoring server are listed on the "IBM Tivoli Monitoring Service Index" ") ip.pipe bound to port 14206: base=1918, 160 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 179
the RAS1 trace log that indicate timestamp problems such as Ignoring invalid lastTimeStamp. This stop or start the Tivoli Enterprise Monitoring Server service, after the set timeout, the Tivoli Enterprise In any other situation, call IBM Software Support. Providing the wrong path troubleshooting 161 - IBM E027SLL-H | Troubleshooting Guide - Page 180
. Consult the operating system manuals for information on how to support to a remote Tivoli Enterprise Monitoring Server, the hub Tivoli Enterprise Monitoring Server must be running and reachable by the remote Tivoli Enterprise Monitoring Server. 162 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 181
Monitoring Server on a Windows system, the agent support is typically applied. This fails if the hub for details and contact the system administrator. A new login solves the problem and enables the tacmd listsystems command. Log indicates hub monitoring servers Monitoring server troubleshooting 163 - IBM E027SLL-H | Troubleshooting Guide - Page 182
") Close failed request The Platform view in the Manage Tivoli Enterprise Monitoring Services panel shows the Tivoli Enterprise Monitoring Server as running as a 32 bit application, but , the query will OR together all of the subnodes. 164 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 183
in a hot standby environment The tacmd listappinstallrecs command returns the application support installation records and displays the current self-describing agent product installation status practice is to run the commands on the current hub. Chapter 9. Monitoring server troubleshooting 165 - IBM E027SLL-H | Troubleshooting Guide - Page 184
not reappear if the mirror had not promoted itself. Troubleshooting monitoring server problems on z/OS systems Review the problems you might experience with Tivoli Enterprise Monitoring Server on z/ internal tasks. These messages are benign. 166 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 185
a. Run &shilev.INSTLIB. b. On the Installation tool main menu, select Services and utilities > Unlock SMP/E high-level qualifiers. c. Specify Y on the generated in the output. A possible workaround is to manually add the missing parameters. The problem is that the KCITVARS ISPF table is out of - IBM E027SLL-H | Troubleshooting Guide - Page 186
On the ICAT main menu, select Configure products->Services and utilities->Execute a CLIST in the TKANCUS library Monitoring Server on z/OS manual for more information about rte. RKANPAR(KDSSYNSIN) member if IBM Support personnel instructs you to do so. The IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 187
is configured correctly. Refer to the "Configuring Tivoli Enterprise Monitoring Server on z/OS manual for more information about configuring a z/OS Tivoli Enterprise Monitoring Server. 2. If the the Tivoli Enterprise Monitoring Server main menu. Chapter 9. Monitoring server troubleshooting 169 - IBM E027SLL-H | Troubleshooting Guide - Page 188
170 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 189
to a problem you experience with a monitoring agent, refer to the agent-specific user guide. Command-line interface Review the command-line interface troubleshooting descriptions for description of configuration and usage problems and how to resolve them. © Copyright IBM Corp. 2005, 2012 171 - IBM E027SLL-H | Troubleshooting Guide - Page 190
supported. You must specify the exact name of the event log you want to monitor. The Windows Registry Editor lists the event log name as a key in either of two paths: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services the display. When an attribute 172 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 191
IBM Tivoli Monitoring V6.2.2 FP6 IZ98638 does not support reflex actions for OMEGAMON XE agents on z/OS You might encounter a problem using The error message continues to display, even after you have replaced the file with the correct one. This behavior is a Monitoring agent troubleshooting 173 - IBM E027SLL-H | Troubleshooting Guide - Page 192
workspaces entries for a description of problems in the Tivoli Enterprise Portal You might notice that the Agent Management Services (AMS) workspace remains unavailable, even if IBM Tivoli Monitoring for Databases: Oracle Agent's SQL Text workspace. 174 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 193
To solve this problem, replace the view and save the workspace. Local history migration tools move the agent operation logs to multiple agent history locations If you upgrade your Tivoli Enterprise Monitoring Server and Tivoli Enterprise Portal Server after configuring historical data collection in - IBM E027SLL-H | Troubleshooting Guide - Page 194
group to 1 hour to avoid long-term problems. Unable to access History Collection Configuration for any agent You are unable to access History Collection Configuration from the Tivoli Enterprise Portal for to KFWWHISTDA in the portal server, and then 176 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 195
When installing portal server support from images older than v6.2.2, the support installation might fail with To ensure that these monitoring agents are started, manually start them from Manage Tivoli Monitoring Services. Examples of monitoring agents from before v6.2.2 troubleshooting 177 - IBM E027SLL-H | Troubleshooting Guide - Page 196
have a problem with the Service Interface has the following limitations when working with subnode agents: v The Queries Link has been removed and is not supported History views show metrics for all subnodes combined in a single table. Currently, the service IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 197
support to manually start the monitoring server fail with the message: Unable to start tems service. Check IBM Tivoli Monitoring Administration Guide's chapter on Tivoli Agent Management Services for information on how to change the parameters of this file. Chapter 10. Monitoring agent troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 198
Type: CRON Data: * 8-16 * * 1-5 To correct the problem, use the tacmd editcalendarentry command to correct the calendar overlaps so that when deploying an System Service Monitor agent When deploying a System Service Monitor agent using a history. 180 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 199
history data file affects the amount of time it takes the agent to read the file. v The amount of data the agent is trying to send to the monitoring server. v The bandwidth of the communications (for example, a slow data rate). Situations with attributes from more than 1 group not supported Services - IBM E027SLL-H | Troubleshooting Guide - Page 200
v6.2.1, installing a backlevel Windows OS Agent into an existing 6.2.1 environment is not supported, and the Tivoli Enterprise Monitoring Server cannot start. If this is the desired deployment, the backlevel Windows OS Agent should be installed first. 182 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 201
file is missing for IBM Tivoli Monitoring 5.x Endpoint situations A BAROC file does not exist for the IBM Tivoli Monitoring 5.x Integration receiver. The BAROC file for the Integration Agent is no longer supplied with IBM Tivoli Monitoring. Generate the BAROC files as needed by using the BAROC file - IBM E027SLL-H | Troubleshooting Guide - Page 202
Adding ACHAN1 to Window DEPLOY group" tacmd addgroupmember -g Window -t DEPLOY -m achan1.raleigh.ibm.com -p KDYRXA.RXAusername=achan KDYRXA.RXApassword=xxxxx KDYRXA.installDir=C:\data @echo "Issuing group DEPLOY run as the root user ID. To prevent 184 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 203
problem, ensure that the secureMain lock command with the -g option has been previously run. See the "Securing your IBM Tivoli Monitoring installation on Linux or UNIX" appendix in the IBM Tivoli Monitoring Installation and Setup Guide file is no longer supported. Instead you must troubleshooting 185 - IBM E027SLL-H | Troubleshooting Guide - Page 204
root privileges. See the IBM Tivoli Monitoring Installation and Setup Guide for instructions on how to have drive the bus !" or 100 characters of Japanese text. The associated ID string will be of the form 000000000000008D723DE7DFED450F. If the 186 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 205
for any remaining managed system list that is in the distribution list for the situation. Manually restart the situation after the managed system list has been removed. Descriptions are not displayed for create a new workspace to use the new query. Chapter 10. Monitoring agent troubleshooting 187 - IBM E027SLL-H | Troubleshooting Guide - Page 206
in step 1. 5. Edit out all statements except the index you want to recreate. 6. Run the create index statement. 7. Start the Summarization and Pruning Agent. 188 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 207
name can be for some of the DBMS' that are supported by Tivoli Data Warehouse. Agent deployment operations are not completing 16. Resolutions for agent deployment operations that TIMEOUT Problem Resolution KDY0014E message Increase the Tivoli Enterprise Monitoring agent troubleshooting 189 - IBM E027SLL-H | Troubleshooting Guide - Page 208
product node type An agent name truncation problem can occur when the network domain name is included in the network host name portion of the agent name. For example, instead of just the host name myhost1 being used, the resulting host name might 190 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 209
32 characters resulting in the name SERVER1:myhost1.acme.north.prod. The agent name truncation is only a problem if there is more than one monitoring agent on the same system. In this case, the agent name . v For z/OS agents, find the RKANPAR library. Chapter 10. Monitoring agent troubleshooting 191 - IBM E027SLL-H | Troubleshooting Guide - Page 210
for information on name generation. If necessary, contact IBM Software Support. Other symptoms that can be observed when multiple another agent node on the portal client About this task This problem has been seen in monitoring agents that exist on the same IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 211
.00.00.00, activity c638270e4738.22.02.09.2a.15.06.28.a2, 7509.0 These are issued to alert you that some set components are driving RPC requests to a server that is not setup to handle that request. Often, this occurs when the Warehouse Proxy Agent is not Chapter 10. Monitoring - IBM E027SLL-H | Troubleshooting Guide - Page 212
IBM Tivoli Monitoring Installation and Setup Guide (http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/ com.ibm.itm.doc_6.2.3fp1/itm623FP1_install.htm). When upgrading a System Service lose data or have memory problems On distributed systems, the IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 213
Portal, for short term history display, only pulls up that is a potential problem, assuming the persistent ibm.db2.jcc.b.a.a(a.java:373) at com.ibm.db2.jcc.b.a.y(a.java:346) at com.ibm.db2.jcc.b.a.l(a.java:298) at com.ibm.db2.jcc.c.j.c(j.java:234) Chapter 10. Monitoring agent troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 214
ibm.db2.jcc.c.uf.addBatch(uf.java:1348) at com.tivoli.twh.khd.khdxjdbc.addBatch(khdxjdbc.java:1290) Check the ethernet adapter settings on both the client and server. There are problems the kuilistsitVld.xml file to replace the following text: - IBM E027SLL-H | Troubleshooting Guide - Page 215
is to increate the DB2 logging for the warehouse database. See the IBM Tivoli Monitoring for Databases: DB2 Agent User's Guide for more information. Receive a message saying that the statement parameter can Enterprise Portal. Symbol references Chapter 10. Monitoring agent troubleshooting 197 - IBM E027SLL-H | Troubleshooting Guide - Page 216
panel when the workspace is reached through the link. This problem was reported against Tivoli Management Portal v1.9.6 but also applies non-null value cannot be found. In order to support historical queries against summarized data, various agent groups IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 217
of the query text, can be used as a guide to what to provide as filter values. Another symbol to decline it, "3" to print it,"4" to read non-IBM terms, or "99" togo back to the previous screen. versions of the gskit component installed by IBM Tivoli Monitoring. The message is displayed at - IBM E027SLL-H | Troubleshooting Guide - Page 218
for this problem. Either manually set the file permission to history dir, or add the db2 instance user to the root group. See the IBM Tivoli Monitoring Installation and Setup Guide for instructions on how to 256 megabytes of heap space for the JVM. 200 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 219
Chapter 11. Command troubleshooting This chapter describes problems you might experience with commands. On Solaris 8 operating installation directory (for example, C:\IBM\ITM) with either the -d | --directory flag or the group deployment property (KDY.INSTALLDIR=C:\IBM\ITM), the execution of the - IBM E027SLL-H | Troubleshooting Guide - Page 220
system temp directory provides additional information about the problem. tacmd executecommand command run against subnode fails Subnodes running, from the directory where the short-term history files are stored. For more information about the krarloff command IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 221
Primary:CLISOAP:NT,Primary:managed_system_name Problem running the tacmd listsystemlist -d command on Linux systems When running the with a backslash (\). The following example escapes a semicolon (;): ./tacmd listsystemlist -d \; Chapter 11. Command troubleshooting 203 - IBM E027SLL-H | Troubleshooting Guide - Page 222
The tacmd listsit command on AIX systems gives no results as shown below tacmd listsit -m UX Managed System gives no result. Return code = 255 204 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 223
system where the command is issued due to specific AIX memory management. This problem can be solved by setting the environment variable, LDR_CNTRL=MAXDATA=0x80000000, to be large temporary files could present problems that ultimately result in future getfile Chapter 11. Command troubleshooting 205 - IBM E027SLL-H | Troubleshooting Guide - Page 224
specified managed system or managed systems. Problems with Take Action commands and curly brackets replace the following text: with the following text: 206 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 225
message is returned after successful execution: KUICAB022I: The following bundles were successfully added to the C:\IBM\ITM\CMS\depot\depot If the addBundlescommand times out, this message is not returned. Set not valid, so these incorrect values are ignored. Chapter 11. Command troubleshooting 207 - IBM E027SLL-H | Troubleshooting Guide - Page 226
and {hostname}_ lz_klzagent_{timestamp}-XX.log is located in the {CANDLEHOME}/logs directory. Refer to IBM Tivoli Monitoring v 6.2 Problem Determination Guide for more information. The user can verify the agent is stopped by running tacmd listSystems: 208 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 227
trying to use the tacmd maintagent command The tacmd maintAgent command is disabled for the IBM Tivoli Monitoring v6.2.2 release. The command stops and starts situations on individual agents without the fully-qualified host name is itmserver.raleigh.ibm.com). Chapter 11. Command troubleshooting 209 - IBM E027SLL-H | Troubleshooting Guide - Page 228
monitoring server fully-qualified host name during the createnode deployment using the SERVER=itmserver.raleigh.ibm.com property. Take Action commands do not work if unrequired values are left blank The v setOverride v listOverrides v deleteOverride 210 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 229
Procedure This problem has to do with your locale and system configuration, and can be fixed by performing the following procedure: 1. Open file for the virtual system where the tacmd command is being invoked. monitor_control.virtual_rdtsc = false Chapter 11. Command troubleshooting 211 - IBM E027SLL-H | Troubleshooting Guide - Page 230
described in the "Timekeeping in VMWare Virtual Machines" paper on the VMWare website: IBM Service Management Connect (http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf). At this time, there is not a workaround for this issue. 212 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 231
a mistake in typing 1 of the attribute names in the command. If you are using the IBM Tivoli Monitoring command line tacmd createSit function for situation creation, you can use the Situation editor in the installation does not overwrite the existing agent. Chapter 11. Command troubleshooting 213 - IBM E027SLL-H | Troubleshooting Guide - Page 232
IBM/ITM" already exists. OK to use it [ 1-yes, 2-no; "1" is default ]? Select one of the following: 1) Install products to the local host. 2) Install products to depot for remote deployment (requires TEMS). 3) Install TEMS support to install patches. 214 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 233
troubleshooting Review the logging instructions and symptoms and solutions for solving problems with the Tivoli Performance Analyzer. Enabling logging for the agent If you experience any problems IBM Tivoli Monitoring > Manage Tivoli Monitoring Services the log files to support, stop the agent, - IBM E027SLL-H | Troubleshooting Guide - Page 234
the files to itpa_agent_log1.zip and send it to IBM Software Support. Installation and configuration issues Review the problems associated with the installation and configuration of the Performance Performance Analyzer agent for non-linear trending, 216 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 235
Analyzer" in the IBM Tivoli Monitoring Installation and Setup Guide. "Enable SPSS in the IBM Tivoli Monitoring Installation and Setup Guide. Problems after upgrading If click Style. 3. Depending on the view title you are modifying, replace the Kpa:KPAxxxxentry in the header Text box with one of the - IBM E027SLL-H | Troubleshooting Guide - Page 236
be started to service a request, or was terminated as a result of a database system shutdown or a force command. SQLSTATE=55032b This message indicates that DB2 has exhausted all available shared memory segments. To resolve this problem, you must configure your DB2 server to support extended shared - IBM E027SLL-H | Troubleshooting Guide - Page 237
messages to identify the source of the problem. v Is the Summarization and Pruning a distributed system did you install the correct support files on each workstation? The Tivoli Performance agent" section in the IBM Tivoli Monitoring: Installation and Setup Guide. v Check if the troubleshooting 219 - IBM E027SLL-H | Troubleshooting Guide - Page 238
example, for the KPA_GENERIC_D32_NLT_FCAST attribute group in DB2, use the following command: CREATE INDEX DB2INST1.ITMIXFCMEM ON DB2INST1.KPA_GENERIC_D32_NLT_FCAST_PA ("System_Name" ASC) ALLOW REVERSE SCANS 220 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 239
(http://publib.boulder.ibm.com/ infocenter/prodguid/v1r0/clarity/index.html). The Tivoli Enterprise Monitoring Server does not restart after installation of Domain Support This problem may occur on Windows correctly. No data is lost in the process. Chapter 12. Performance Analyzer troubleshooting 221 - IBM E027SLL-H | Troubleshooting Guide - Page 240
222 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 241
Chapter 13. Database troubleshooting Review the common problems and resolutions to prevent data loss and solve problems with the Tivoli Data Warehouse and Warehouse agents. Data loss added during installation for creating the portal server data source. © Copyright IBM Corp. 2005, 2012 223 - IBM E027SLL-H | Troubleshooting Guide - Page 242
The following instructions do not IBM Tivoli Monitoring > Manage Tivoli Enterprise Monitoring Services support multidimensional clustering table creation. Therefore, you can not use the ORGANIZE BY DIMENSIONS(OBJECT) clause required to create the table 224 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 243
supported browser was not present on data source was not defined:[IBM][ODBC Driver Manager] Data source services must also change, otherwise the database does not start because the DB2 processes cannot logon. Use the following steps to resolve this problem: Chapter 13. Database troubleshooting - IBM E027SLL-H | Troubleshooting Guide - Page 244
Start > Control Panel > Administrative Tools > Services. 2. Scan the column on the right for problems and before customizing the monitoring server. The movefrom-m2i.bat is only used to migrate the contents of Microsoft SQL Server to Universal Database. 226 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 245
An error occurs with the SQLTables if the database was created before applying IBM DB2 V8.1 Fix Pack 10 (also known as Version 8.2 Fix Pack blocking all grant public c:\SQLLIB\bnd>DB2 bind @db2cli.lst blocking all grant public c:\SQLLIB\bnd>DB2 bind db2schema.bnd blocking troubleshooting 227 - IBM E027SLL-H | Troubleshooting Guide - Page 246
history file for the tablespace attribute group before you created and distributed the new historical collection that has the filter. The older data would have been exported to the warehouse proxy and shown up in the Tivoli Data Warehouse database. 228 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 247
supported old data automatically, rather than manually. To reduce the data that is History Collection Configuration window in the Tivoli Enterprise Portal" in the IBM Tivoli Monitoring Administrator's Guide for instructions service recognizes the change. Chapter 13. Database troubleshooting 229 - IBM E027SLL-H | Troubleshooting Guide - Page 248
Proxy is installed. 2. Restart the Windows computer so that the Warehouse Proxy windows service recognizes the change. SQL0552N "ITMUSER" does not have the privilege to perform operation -based system user must belong. db2 get dbm cfg | grep SYSADM 230 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 249
14. Event synchronization troubleshooting This section provides descriptions of and resolutions for problems you might experience there is currently no resolution: One or more errors occured during the replacement of files (tecSyncAllFile1) with files (tecSyncAllFile1). Refer to install log for more - IBM E027SLL-H | Troubleshooting Guide - Page 250
IBM Tivoli Monitoring triggers and procedures. Tracing is enabled by editing the get_debug_itmsync procedure in the Netcool/OMNIbus ObjectServer and changing the debug_itmsync flag to 1. The procedure can be edited using Netcool/OMNIbus Administrator. 232 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 251
Resolving problems sending events to Netcool/OMNIbus Cause Resolution IBM Tivoli instructions on how to configure the hub monitoring server, see the "Configuring your monitoring server to forward events" topic of the IBM Tivoli Monitoring Installation and Setup Guide (http://publib.boulder.ibm - IBM E027SLL-H | Troubleshooting Guide - Page 252
problems sending events to Netcool/OMNIbus (continued) Cause Resolution Configuration of the EIF destination is incorrect. Review the configuration of the OMNIbus EIF destination on the IBM OMNIbus Server Default : N Host1 : nswin01:9998 234 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 253
problems the port number information on the IBM Tivoli Monitoring side, you can use the IBM Tivoli Monitoring CLI commands listeventdest and or later, the default port number is 9998. If Tivoli Business Service Manager Version 4.2.1 was used to install the probe, the default troubleshooting 235 - IBM E027SLL-H | Troubleshooting Guide - Page 254
.sh (UNIX) or startSUF.cmd (Windows) $EVENT_SYNC_INSTALLDIR is the directory where the IBM Tivoli Monitoring event synchronization component was installed. Look for Situation Update Forwarder startup errors in /tmp/itmsynch/logs/ synch_trace.log. 236 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 255
the monitoring server user name or password have changed recently, use the sitconfuser command to update the monitoring server's information. See the IBM Tivoli Monitoring Command Reference for details on the syntax of the sitconfuser command. Chapter 14. Event synchronization troubleshooting 237 - IBM E027SLL-H | Troubleshooting Guide - Page 256
topic "Configuring the OMNIbus server for program execution from scripts" in the IBM Tivoli Monitoring Installation and Setup Guide. If the user specified by the PA.Username property is a member of $NCO_PA is the name of the process agent. 238 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 257
attributes set by the IBM Tivoli Monitoring probe rules "Default mapping of situation events to OMNIbus events" in the IBM Tivoli Monitoring Installation and Setup Guide describes how OMNIbus attributes files, you must restart the EIF probe. Chapter 14. Event synchronization troubleshooting 239 - IBM E027SLL-H | Troubleshooting Guide - Page 258
Guide. Verify that the default duplication trigger has been configured to ignore events from IBM Tivoli Monitoring. See the topic "Changing the default deduplication trigger" in the IBM Tivoli Monitoring Installation and Setup Guide for more details. 240 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 259
used when sampled events are deleted or cleared in Netcool/OMNIbus" in the IBM Tivoli Monitoring Installation and Setup Guide for more information. After an event is cleared in Netcool/OMNIbus, the event that is included in the loopback event. Chapter 14. Event synchronization troubleshooting 241 - IBM E027SLL-H | Troubleshooting Guide - Page 260
IBM Tivoli Monitoring Installation and Setup Guide for more details. Tivoli Enterprise Console integration troubleshooting Review the fundamental Tivoli Enterprise Console troubleshooting tasks for guidance with solving integration problems drive on which the rule base resides to import the IBM - IBM E027SLL-H | Troubleshooting Guide - Page 261
timeout_value is the length of the timeout period, in milliseconds. To configure a timeout of 30 seconds, set the timeout_valuevalue to 30000. Chapter 14. Event synchronization troubleshooting 243 - IBM E027SLL-H | Troubleshooting Guide - Page 262
244 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 263
make diagnosis of the underlying problem easier. When troubleshooting Tivoli Common Reporting problems, enable detailed logging as instructed in the Tivoli Common Reporting information center topic, "Using log files for troubleshooting" (http://publib.boulder.ibm.com/infocenter/tivihelp/ v3r1/topic - IBM E027SLL-H | Troubleshooting Guide - Page 264
by adding following options -Xms64m -Xmx768m, or -Xms64m -Xmx1024m if the problem persists. Example of JAVA_ARGS after setting the maximum heap size: JAVA_ARGS="-Dtcr. .ChainHandler [ifElseBirthandler] com.ibm.cognos.birtservice.logic.IfElseBirtHandler 246 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 265
manually delete the *.rtm files located in the cognos/data/cqe/RTModels directory. Displaying data for Situations History Prompted to select report type when installing reports with CLI When you install IBM Tivoli Monitoring OS Agents reports within the CLI, you might be prompted troubleshooting 247 - IBM E027SLL-H | Troubleshooting Guide - Page 266
log files for more information. https://localhost:30343/TCR/Reports/view This problem is occurring because you are missing drivers required by Tivoli Common Reporting. You report.data.oda.jdbc Cannot load JDBC Driver class: com.ibm.db2.jcc.DB2Driver 248 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 267
address this problem for a Guide, SA23-2237 - CEC Base Agent User's Guide, SC23-5239 - HMC Base Agent User's Guide, SA23-2239 - VIOS Premium Agent User's Guide, SA23-2238 v Other base agents: - Monitoring Agent for IBM Tivoli Monitoring 5.x Endpoint User's Guide Common Reporting troubleshooting 249 - IBM E027SLL-H | Troubleshooting Guide - Page 268
extracted do not exist in the database. To view the SQL query that generated this problem and determine what historical data is not being collected, review the log files. See " do the following to correct this problem: 3. Stop Tivoli Common Reporting. 250 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 269
this location: On Windows Systems: C:\Program Files\IBM\tivoli\tip\bin 5. Locate this instruction about half way through the bat file: work with other database managers supported by Tivoli Data Warehouse DW. You could circumvent this problem by collecting Tivoli Data Warehouse troubleshooting 251 - IBM E027SLL-H | Troubleshooting Guide - Page 270
attributes. To determine the cause of this problem you need to enable detailed logging in Tivoli Common Reporting, as described in the IBM Tivoli Common Reporting: User's Guide, and you need to know which agent table (or attribute group) contains the 252 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 271
text at the bottom of a report. Although this error can occur for several reasons, the most common problem is incorrectly entering the Tivoli Data Warehouse database URL into the data source for the report. If some product for the correct URL. Chapter 15. Tivoli Common Reporting troubleshooting 253 - IBM E027SLL-H | Troubleshooting Guide - Page 272
of three reasons: v The report parameters have excluded all the data. v You recently started using IBM Tivoli Monitoring for Energy Management and have not put data into the Tivoli Data Warehouse. v The another reason why no data is being returned. 254 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 273
Facility troubleshooting An auditing facility in IBM Tivoli for the ITM Audit attribute group and distribute the history collection settings to the components you want to have F. ITM Audit log in the IBM Tivoli Monitoring Version 6.2.3 Administrator's Guide. ITM components in a z/OS environment - IBM E027SLL-H | Troubleshooting Guide - Page 274
256 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 275
process names when you want to confirm that a process is indeed running and review the resources used. Table 22. IBM Tivoli Monitoring processes by operating system Component Windows Tivoli Enterprise Monitoring Server cms.exe kdsmain.exe Tivoli Enterprise Portal Server KfwServices - IBM E027SLL-H | Troubleshooting Guide - Page 276
258 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 277
and about the commonly shared components of Tivoli Management Services: v Quick Start Guide Introduces the components of IBM Tivoli Monitoring. v Installation and Setup Guide, GC32-9407 Provides instructions for installing and configuring IBM Tivoli Monitoring components on Windows, Linux, and UNIX - IBM E027SLL-H | Troubleshooting Guide - Page 278
messages generated by all IBM Tivoli Monitoring components and by z/OS-based Tivoli Management Services components (such as Tivoli Enterprise Monitoring Server on z/OS and TMS:Engine). v Troubleshooting Guide, GC32-9458 Provides information to help you troubleshoot problems with the software - IBM E027SLL-H | Troubleshooting Guide - Page 279
documentation You can also obtain technical documentation about IBM Tivoli Monitoring and related products from the following sources: v IBM Integrated Service Management Library http://www.ibm.com/software/brandcatalog/ismlibrary/ IBM Integrated Service Management Library is an online catalog that - IBM E027SLL-H | Troubleshooting Guide - Page 280
Wiki at http://www.ibm.com/developerworks/wikis/display/tivoliomegamon/ Home provides information about the OMEGAMON XE products, NetView for z/OS, Tivoli Monitoring Agent for z/TPF, and other System z monitoring and application management products. 262 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 281
and serviceability tools for problem determination. To install the ISA software, go to http://www.ibm.com/software/support/isa. Troubleshooting Guide For more information about resolving problems, see the product's Troubleshooting Guide. Using IBM Support Assistant The IBM Support Assistant - IBM E027SLL-H | Troubleshooting Guide - Page 282
resolve your problem. To determine which fixes are available for your Tivoli software product, follow these steps: 1. Go to the IBM Software Support website at http://www.ibm.com/software/ support. 2. documents you want to receive. 13. Click Update. 264 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 283
[email protected], describing your problem. By phone Call 1-800-IBM-4You (1-800-426-4968). Contacting IBM Software Support IBM Software Support provides assistance in your country, go to the IBM Software Support website at http://techsupport.services.ibm.com/guides/ contacts.html and click the name - IBM E027SLL-H | Troubleshooting Guide - Page 284
266 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 285
operation of any non-IBM product, program, or service. IBM may have patents IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites - IBM E027SLL-H | Troubleshooting Guide - Page 286
. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those and distribute these sample programs in any form without payment to 268 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 287
platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form - IBM E027SLL-H | Troubleshooting Guide - Page 288
270 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 289
replacement for the Data Encryption Standard. The specification calls for a symmetric algorithm (in which the same key is used for both encryption and decryption), using block encryption of 128 bits and supporting and notify you of common problems with the © Copyright IBM Corp. 2005, 2012 operating - IBM E027SLL-H | Troubleshooting Guide - Page 290
replaced by the Tivoli Enterprise Portal. chart A graphical view of data returned from a monitoring agent. A data point is plotted for each attribute chosen and, for bar and 272 IBM Tivoli Monitoring: Troubleshooting Guide requesting data or services and the server business-support processing - IBM E027SLL-H | Troubleshooting Guide - Page 291
that originated at IBM in 1977 and was adopted by the U.S. Department of Defense. DES supports 72 quadrillion or DES applies a 56-bit key to each 64-bit block of data. Although this is considered strong encryption, many company to host Internet services without allowing unauthorized access. - IBM E027SLL-H | Troubleshooting Guide - Page 292
A database used by the Tivoli Enterprise Monitoring Server that serves as a repository of shared objects for all 274 IBM Tivoli Monitoring: Troubleshooting Guide systems across your enterprise. The EIB stores all persistent data, including situations, policies, user definitions, and managed-object - IBM E027SLL-H | Troubleshooting Guide - Page 293
Service Facility, or ICSF. historical data management The procedures applied to short-term binary history files to invocations of specific Worldwide websites. HTTPS The Secure Hypertext Transport server A proprietary web server developed for IBM Tivoli Monitoring that is installed and configured - IBM E027SLL-H | Troubleshooting Guide - Page 294
they are located without having to understand anything about the server other than the service it performs and its address (called the Interoperable Object Reference, IOR, which comprises a constituent managed system is added or deleted), then all the 276 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 295
system, such as an AIX-based pSeries server, that IBM Tivoli Monitoring is monitoring. A node can also be Monitoring Server, such as those that support other Tivoli products like Netcool/OMNIbus data and methods and responds to requests for services. CORBA defines an object as a combination of - IBM E027SLL-H | Troubleshooting Guide - Page 296
Windows environments. PerfMon 278 IBM Tivoli Monitoring: Troubleshooting Guide monitors network elements such or until the Tivoli Enterprise Portal user manually stops the policy. You can create Monitoring Server. Events are viewed through the Agent Service Interface or can be sent as EIF events or - IBM E027SLL-H | Troubleshooting Guide - Page 297
written to notify you of pure events remain true until they are manually closed or automatically closed by an UNTIL clause. query A particular view remote deployment Using IBM Tivoli Monitoring software, you can deploy agents and other non-agent, Tivoli Management Services-based components to - IBM E027SLL-H | Troubleshooting Guide - Page 298
IBM Tivoli Monitoring: Troubleshooting Guide Enterprise Monitoring Server running under IBM Tivoli Monitoring version 6.1 and subsequent releases. Your site must install this component on the Tivoli Enterprise Console server; for instructions, see the IBM . SSM See System Service Monitors. SSO See - IBM E027SLL-H | Troubleshooting Guide - Page 299
or subsequent. System Service Monitors The IBM Tivoli Netcool/OMNIbus product provides System Service Monitors that support basic system-level response to a situation that has fired (that is, turned true), or manually, as the Tivoli Enterprise Portal operator requires. With Take Action commands, you - IBM E027SLL-H | Troubleshooting Guide - Page 300
monitoring agents. Tivoli Enterprise Web Services An open standards-based interface to IBM Tivoli Monitoring: Troubleshooting Guide of enterprise-wide data for systems-management applications. The software foundation that supports data from their short-term history files to the data warehouse. WAV - IBM E027SLL-H | Troubleshooting Guide - Page 301
session scripts automatically becomes available to all users connected to the same Tivoli Enterprise Portal Server. XML See Extensible Markup Language. z/OS IBM's operating system for its line of mainframe, zSeries computers that can manage large amounts of memory, direct-access storage, and data - IBM E027SLL-H | Troubleshooting Guide - Page 302
284 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 303
application server 272 Application Service Monitors, 281 application support mismatched files 69 Application Support Installer 82 arithmetic Management Workstation 272 CandleManage 196, 206 © Copyright IBM Corp. 2005, 2012 CIM 272 classification of problems 1 clear JAR cache 132 client logon error - IBM E027SLL-H | Troubleshooting Guide - Page 304
problems 65 troubleshooting problems 65 InstallShield error 87 instance, removing remotely 142 integral web server 275 Integrated Cryptographic Service UNIX monitoring process fails to start 123 286 IBM Tivoli Monitoring: Troubleshooting Guide listSit 196, 206 listsystems command 190 listSystems - IBM E027SLL-H | Troubleshooting Guide - Page 305
82 installation 82 UNIX monitoring server application support 89 self-describing capability missing managed system group 83 service console 56 See also IBM Tivoli Monitoring Technology Service Console See also Tivoli Monitoring Service Console blocking access 57 silent installation 95 Simple Network - IBM E027SLL-H | Troubleshooting Guide - Page 306
Support Assistant 263 support portal subscribing 2 viewing 2 sysadmin LDAP 148 SYSIBM:CLI:-805 227 System Modification Program/Extended 273, 281 System Service Add and Remove Programs utility 112 problems 108 288 IBM Tivoli Monitoring: Troubleshooting Guide UNIX installation 89 ksh shell 92 silent - IBM E027SLL-H | Troubleshooting Guide - Page 307
Z z/OS backing up ICAT and CSI environment 107 monitoring server 166 monitoring server support 84 runtime members job 105 z/OS agents reflex automation 173 zOS Tivoli Enterprise Monitoring Server 168 Index 289 - IBM E027SLL-H | Troubleshooting Guide - Page 308
290 IBM Tivoli Monitoring: Troubleshooting Guide - IBM E027SLL-H | Troubleshooting Guide - Page 309
- IBM E027SLL-H | Troubleshooting Guide - Page 310
Printed in USA GC32-9458-05
-
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8
-
9
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
32
-
33
-
34
-
35
-
36
-
37
-
38
-
39
-
40
-
41
-
42
-
43
-
44
-
45
-
46
-
47
-
48
-
49
-
50
-
51
-
52
-
53
-
54
-
55
-
56
-
57
-
58
-
59
-
60
-
61
-
62
-
63
-
64
-
65
-
66
-
67
-
68
-
69
-
70
-
71
-
72
-
73
-
74
-
75
-
76
-
77
-
78
-
79
-
80
-
81
-
82
-
83
-
84
-
85
-
86
-
87
-
88
-
89
-
90
-
91
-
92
-
93
-
94
-
95
-
96
-
97
-
98
-
99
-
100
-
101
-
102
-
103
-
104
-
105
-
106
-
107
-
108
-
109
-
110
-
111
-
112
-
113
-
114
-
115
-
116
-
117
-
118
-
119
-
120
-
121
-
122
-
123
-
124
-
125
-
126
-
127
-
128
-
129
-
130
-
131
-
132
-
133
-
134
-
135
-
136
-
137
-
138
-
139
-
140
-
141
-
142
-
143
-
144
-
145
-
146
-
147
-
148
-
149
-
150
-
151
-
152
-
153
-
154
-
155
-
156
-
157
-
158
-
159
-
160
-
161
-
162
-
163
-
164
-
165
-
166
-
167
-
168
-
169
-
170
-
171
-
172
-
173
-
174
-
175
-
176
-
177
-
178
-
179
-
180
-
181
-
182
-
183
-
184
-
185
-
186
-
187
-
188
-
189
-
190
-
191
-
192
-
193
-
194
-
195
-
196
-
197
-
198
-
199
-
200
-
201
-
202
-
203
-
204
-
205
-
206
-
207
-
208
-
209
-
210
-
211
-
212
-
213
-
214
-
215
-
216
-
217
-
218
-
219
-
220
-
221
-
222
-
223
-
224
-
225
-
226
-
227
-
228
-
229
-
230
-
231
-
232
-
233
-
234
-
235
-
236
-
237
-
238
-
239
-
240
-
241
-
242
-
243
-
244
-
245
-
246
-
247
-
248
-
249
-
250
-
251
-
252
-
253
-
254
-
255
-
256
-
257
-
258
-
259
-
260
-
261
-
262
-
263
-
264
-
265
-
266
-
267
-
268
-
269
-
270
-
271
-
272
-
273
-
274
-
275
-
276
-
277
-
278
-
279
-
280
-
281
-
282
-
283
-
284
-
285
-
286
-
287
-
288
-
289
-
290
-
291
-
292
-
293
-
294
-
295
-
296
-
297
-
298
-
299
-
300
-
301
-
302
-
303
-
304
-
305
-
306
-
307
-
308
-
309
-
310
 |
 |
IBM Tivoli Monitoring
Version 6.2.3 FP1
Troubleshooting Guide
GC32-9458-05
±²³