IDKCS000052
Tool to aid in the collection of DSA logs and settings. WrittenJul 17, 2015

 
Attachments2
  

Product Categories
BAR Software

 Objective
Job and process logs for DSA are distributed between the various components of the backup solution, and also within the DSC repository, which can make their collection either time consuming or haphazard.

So to help in carrying out the task the TGSC BAR group have developed the "dsaextract" tool which is distributed within the "teradata-gsctools" package.

The tool within the "teradata-gsctools" package may not be the latest available so if needed the tool can be installed from the Tarball (dsaextract.tgz) attached to this KA.
ie.
tar -xPzf dsaextract.tgz

The tool needs to be launched on the DSADSC master (The System where the DSC Repository resides)

dsaextract can collect:
 
  • Logs
  • messages (For all components in the system)
  • dsc.log (From the dsadsc master)
  • clienthandler.log (From the Client Handlers)
  • Dsmain logs ( From the Nodes involved in the backup/Restore)
  • Netbackup Job Logs (If still available)
  • activemq.log
  • Properties files
  • dsc.properties
  • clienthandler.properties
  • Process information
  • network information
  • Job information
  • Version information


All of the collected information is combined into a Tarball that can then be easily attached to an incident.
 
Note:
The tool also be run on a clienthandler and also on a  TPA node of a Teradata system. 
 

Environment/Conditions/Configuration
DSA
DSC
NetBAckup


Procedure
dsaextract when run of the DSADSC server, which is always the best place to run the command
 
In its most basic form the tool only requires the -j option.
ie
dsaextract -j "<dsc jobname>"

If the Job has been run more than once, a menu will be displayed so that the operator can select the required job instance.

The tool will collect everything possible with the following options.

dsaextract -j "<dsc jobname>" -L -b -R -x 

However under normal circumstances the suggested options are:      (This is the best set of options to use if unsure of what to collect for a failed job)

dsaextract -j "<dsc jobname>" -b -R -L

If the "-j" option is omitted, general logs and setting will be collected from just the DSADSC Server and the Media Clients. (plus the DSMain logs if the –T option is used)

ie
dsaextract
dsaextract -k
dsaextract -T <tdpid>


The full list of options are:
       -b  collect Netbackup log files
       -c  collect consolidated logs
       -d  <n> Number of days worth of dsmain/clienthandler logs to collect if not the default of 3 (ignored if -R or -r is not used)
       -f  Force collect remote logs/information even if ssh trust relationship is not setup (will not overcome system hardening restrictions)
       -F  Do not collect Failed NetBackup Logs
       -h  Help
       -j  <Jobname> to collect Logs for
       -k  Keep the generated files on the System
       -l  extract Job logs for the selected job. (This can take some time)
       -L  extract Job logs for Backup as well as the selected Restore/Analyze_Validate (This can take some time)
       -m  Skip the collection of Metadata, may be required in the case of backup jobs with over 20,000 objects
       -r  Collect Logs/Config files/Status info' from ClientHandlers (Needs ssh trust relationship set up.)
       -R  Collect Logs/Info' from System Nodes (Needs ssh trust relationship set up.) Note: This also forces the -r option
       -s  include system information
       -S  Do not collect Repository dsmain logs
       -t  <n> Can be used to add <nt> seconds to the time period used to scan NetBackup Logs. (DSA sometimes reports it's completion time before the NetBackup
           Jobs it initiates have completed. This option may be required to overcome the discrepancy.
       -T  Force collect of DSMAIN logs from  <tdpid>, only used if "-j" is not used.
       -v  Version information
       -V  Option for use by sys_val to collect configuration information for local Server/Client.
       -x  Include xml Job logs (This can take some time, only select if requested by TGSC)
       --DM or --Datamover
           Using this flag signifies the name of the Job supplied using the -j option is a Data Mover Job
       --VP or --Viewpoint
           Collect basic information from ViewPoint Server
       --VPfull or --viewpointfull
           Collect information and full logs from ViewPint Server
       -Z  Special option for use with collecting logs from TPA Nodes or ClientHandlers, Operator will be told how/when to use it.
    Note: If the -j option is not used, then dsaextract will collect general
    logs/setting from the DSADSC Master and Media Clients only.
 
dsaextract when run on a ClientHandler
 
If dsaextact is run on a ClientHandler it will only collect logs/information from the ClientHandler.
Valid option that can be used are -d,-Z, -t and -k only, but it's suggested that no options are used. 
The -Z option is a special option that should only be used when instructed to do so by the running of dsaextract on a DSADSC server.
(It's important to enter the command as instructed, as such it's suggested that cut and paste is used.)
 
dsaextract when run on a TPA Node
 
If dsaextract is run on a TPA node, it will collect log/information from the system that is it run on.
Valid options that can be used are -d, -Z, -t and -k only, If no options are used, the tool will collect  the last 3 days worth of logs,
which can be useful as logs can remain on a system for quite sometime, so they can grow quite large.
The -Z option is a special option that should only be used when instructed to do so by the running of dsaextract on a DSADSC server.
(It's important to enter the command as instructed, as such it's suggested that cut and paste is used.)
 

Sample output of the log collection process of a RESTORE job:
 
   # dsaextract -j "RESTORE_MUPPET_PMCPDATA" -L -R -b
   Running inital checks
   Querying Job History
   More than one job has been found:
   
     1)    500010             09/05/16 12:24:24 PM   09/05/16 02:13:06 PM   01:48:41   COMPLETED_SUCCESSFULLY   36        GRP_LTO4_1WK   muppet   
     2)    500011             09/05/16 02:22:00 PM   09/05/16 04:10:54 PM   01:48:54   COMPLETED_SUCCESSFULLY   36        GRP_LTO4_1WK   muppet   
   
   Please enter the number corresponding to the job you wish to query
     Line to query: 2
   
   Extracting the Status for Job: RESTORE_MUPPET_PMCPDATA
   Extracting current definition for Job (May differ from when the job was actually run)
   Extracting the Plan for Job (As if it was run now, not at the time it was actually run)
   Extracting the Logs for Job
   
   There are "1" pages to collect
   Finished extracting from job RESTORE_MUPPET_PMCPDATA
   
   Collecting information about the backup/migrated Job (BKP_MUPPET_PMCPDATA) on which the RESTORE is based
   Extracting the Status for Job: BKP_MUPPET_PMCPDATA
   Extracting current definition for Job (May differ from when the job was actually run)
   Extracting Savesets for Job.
   Extracting the Plan for Job (As if it was run now, not at the time it was actually run)
   Extracting the Logs for Job
   
   There are "1" pages to collect
   Finished extracting from job BKP_MUPPET_PMCPDATA
   
   Extracting Netbackup image IDs
   Number of NBU backup images: 3
   
   Extracting MetaData for job "BKP_MUPPET_PMCPDATA" with ID of "500005"
   
   Capturing NBU Policy Defination
   Gathering system Information
   Extracting System Config information for muppet
   Collecting Consumer information for Media Servers
   Collecting Consumer information for Systems
   Collecting Component information for Media Servers
   Collecting Component information for Systems
   Collecting Component information for Target Groups
   Collecting Target Group Map
   Collecting Component information for NBU  Master
   Collecting Configuration information....
   Configuration information about the repository that is required in the event of a DR has been collected.
   
   Running dsc Pings
   Collecting Migrated Job information......
   Collecting NBU Information
   Collecting NBU Policy Information
   Collecting NBU logs as requested
     Looking for NBU Job "1606" using file "BKP_MUPPET_PMCPDATA_500005_1_file1_data_1473066413618" ...Got it
     Looking for NBU Job "1604" using file "BKP_MUPPET_PMCPDATA_500005_1_file_dict_1473066413618" ...Got it
     Looking for NBU Job "1605" using file "BKP_MUPPET_PMCPDATA_500005_1_file2_data_1473066413618" ...Got it
     Extracting  NBU Job "1599" with BackupID of  "tdbemedia1_1473066597"
     Extracting  NBU Job "1597" with BackupID of  "tdbemedia1_1473066420"
     Extracting  NBU Job "1600" with BackupID of  "tdbemedia2_1473066599"
   
   Extracting NBU Job logs for 2 failed Jobs
     Collecting JobID 1572
     Collecting JobID 1569
   
   Collecting Logs/settings/status information from Media Clients
     Collecting logs/info for Media Server "tdbemedia1" using address "tdbemedia1"
     NOTICE: ssh trust relationships have not been setup, bypassing the collection of remote logs/information
   
     Collecting logs/info for Media Server "tdbemedia2" using address "tdbemedia2"
        This can take a while.
   
   Collecting Logs/settings/status information from Nodes
     Collecting logs/info from "muppet" System using address "muppetcop1"
     NOTICE: ssh trust relationships have not been setup, bypassing the collection of remote logs/information
   
   Kicking off the collection of Consolidated Logs
   
   Creating readable job plans
   
   Packaging extracted files...
   
   ###################################################################################################################
   ###################################################################################################################
   ##                                                                                                               ##
   ## Please upload the "/var/opt/teradata/gsctools/dsaextract/RESTORE_MUPPET_PMCPDATA_500011.tgz"                  ##
   ## file to the incident, not forgetting to remove the file on the system once                                    ##
   ## it's been verified as uploaded.                                                                               ##
   ##                                                                                                               ##
   ## IMPORTANT NOTE: Due to security hardening, logs from other servers has not been collected                     ##
   ## so please log onto each of the systems listed as "root" and collect the additional logs                       ##
   ## which are required to avoid delays in the incident resolution.                                                ##
   ##                                                                                                               ##
   ## Name                 System type      Command to run                                                          ##
   ## ------------------------------------------------------------------------------------------------------------- ##
   ## tdbemedia1           ClientHandler    dsaextract -Z "RESTORE_MUPPET_PMCPDATA_500011"                          ##
   ## muppet               Teradata System  dsaextract -Z "RESTORE_MUPPET_PMCPDATA_500011,1473078120,1473084654"    ##

   ##                                                                                                               ##
   ## ============================================================================================================= ##
   ##                                                                                                               ##
   ## Note: Please ensure that "dsaextract" is at least at version "01.06.05" on the additional Servers             ##
   ## prior to collecting the logs, the version can be checked by running "dsaextract -v"                           ##
   ## If a later version is required please refer in Teradata KCS Article KCS000052                                 ##
   ##                                                                                                               ##
   ###################################################################################################################
   ###################################################################################################################
   
   Finished.
 
Sample output of the log collection without specifying a jobname
 
   # dsaextract
   Running inital checks
   
   No Job name supplied so will collect System setting and general logs
   from the DSADSC Master and Client Handlers.
   
   Gathering system Information
   Collecting Job list.
   Collecting Job history.
   Collecting Consumer information for Media Servers
   Collecting Consumer information for Systems
   Collecting Component information for Media Servers
   Collecting Component information for Systems
   Collecting Component information for Target Groups
   Collecting Target Group Map
   Collecting Component information for NBU  Master
   Collecting Configuration information...
   Running dsc Pings
   Collecting Migrated Job information......
   Collecting NBU Information
   Collecting NBU Policy Information
   
   Collecting Logs/settings/status information from Media Clients
     Collecting logs/info for Media Server "tdbemaster" using address "tdbemaster"
        This can take a while.
     Collecting logs/info for Media Server "tdbemedia1" using address "tdbemedia1"
     NOTICE: ssh trust relationships have not been setup, bypassing the collection of remote logs/information
   
     Collecting logs/info for Media Server "tdbemedia2" using address "tdbemedia2"
        This can take a while.
   
   Creating readable job plans
   
   Packaging extracted files...
   
   ###################################################################################################################
   ###################################################################################################################
   ##                                                                                                               ##
   ## Please upload the "/var/opt/teradata/gsctools/dsaextract/DSCinfo.tgz"                                         ##
   ## file to the incident, not forgetting to remove the file on the system once                                    ##
   ## it's been verified as uploaded.                                                                               ##
   ##                                                                                                               ##
   ## IMPORTANT NOTE: Due to security hardening, logs from other servers have not been collected                    ##
   ## so please log onto each of the systems listed as "root" and collect the additional logs                       ##
   ## which are required to avoid delays in the incident resolution.                                                ##
   ##                                                                                                               ##
   ## Name                 System type      Command to run                                                          ##
 
  ## ------------------------------------------------------------------------------------------------------------- ##
   ## tdbemedia1           ClientHandler    dsaextract -d 3                                                         ##
   ##                                                                                                               ##
   ## ============================================================================================================= ##
   ##                                                                                                               ##
   ## Note: Please ensure that "dsaextract" is at least at version "01.06.05" on the additional Servers             ##
   ## prior to collecting the logs, the version can be checked by running "dsaextract -v"                           ##
   ## If a later version is required please refer in Teradata KCS Article KCS000052                                 ##

   ##                                                                                                               ##
   ###################################################################################################################
   ###################################################################################################################
   
   Finished.

Sample output of the log collection from a clienthandler, as instructed by the output from the log collection on the DSADSC Server.

   # dsaextract -Z "RESTORE_MUPPET_PMCPDATA_500011"
   
   Collecting Logs/settings/status information from Media Clients
   dsaextract is being run on a clienthandler "tdbemedia1" so only information from the client is being collected.
   
   Packaging extracted files...
   
   ###################################################################################################################
   ###################################################################################################################
   ##                                                                                                               ##
   ## Please upload the "/var/opt/teradata/gsctools/dsaextract/RESTORE_MUPPET_PMCPDATA_500011_ch_tdbemedia1.tgz"    ##
   ## file to the incident, not forgetting to remove the file on the system once                                    ##
   ## it's been verified as uploaded.                                                                               ##
   ##                                                                                                               ##
   ###################################################################################################################
   ###################################################################################################################
   
   Finished.

Sample output of the log collection from a system, as instructed by the output from the log collection on the DSADSC Server.

   # dsaextract -Z "RESTORE_MUPPET_PMCPDATA_500011,1473078120,1473084654"
   
   Collecting Logs/settings/status information from Nodes
   dsaextract is being run on a TPA Node of "MUPPET" so only information from the system and dsmain logs will be collected.
        This can take a very long time. (DO NOT ABORT or work files on a TPA Node will not be deleted)
   
   Packaging extracted files...
   
   ###################################################################################################################
   ###################################################################################################################
   ##                                                                                                               ##
   ## Please upload the "/var/opt/teradata/gsctools/dsaextract/RESTORE_MUPPET_PMCPDATA_500011_system_MUPPET.tgz"    ##
   ## file to the incident, not forgetting to remove the file on the system once                                    ##
   ## it's been verified as uploaded.                                                                               ##
   ##                                                                                                               ##
   ###################################################################################################################
   ###################################################################################################################
   
   Finished.
 

Special Considerations
‚ÄčKCS015796   Collect dsaextract in Data Mover Environment

Additional Information
Note: dsaextract does use ssh/scp to collect logs from the other servers in the system, so ssh trust relationships need to have been setup, unless the -f option is used, in which case password prompts will be presented.

See the attach PDF file for a more extensive list of files collected and the method required to collect them manually

Version of dsaextract attached to this KCS Procedure: 4.00.04