IDKCS001011
AIC: Shortage of TD Dump Slots. WrittenFeb 19, 2016

 
Attachments0
  

Product Categories
PDE - Parallel Database Extensions

 Problem
AIC (Auto Incident Creation) reporting Shortage of TD Dump Slots. Some of the error messages you can receive are:
  • STATUS:linux_tddump_chk:NOTICE:0:Teradata dumps found in /var/opt/teradata/tddump
  • Shortage of TD Dump Slots
  • TD Crashdump Limit Reached
     

Environment/Conditions/Configuration
All DBS/PDE versions

Cause
There are either too many unsaved (raw) dumps in the /var/opt/teradata/tddump directory in the or there could be other files/folders that have been placed in this directory

Solution
The GSO Base team should transfer these incidents for Critical Systems Management (CSM) sites to the CSR so that the CSR can work with the customer to get these resolved, otherwise the GSO Base team is expected to resolve these incidents themselves by working with the customer to determine what can be removed or saved to the Crashdumps database.

The steps that can be taken to resolve the incident include:
 
Verify the number of raw dumps is less than the maximum on each node.
 
Verify there are no other files/folders in /var/opt/teradata/tddump.
 
If all the dump files are valid and if the dumps are needed for analysis, then save it using "csp -mode save -force" (There must be enough space in crashdumps db before saving the dump).
By saving the dump using csp, the raw dump will be automatically cleared from /var/opt/teradata/tddump directory.
 
If the dumps are not needed, you may clear the dumps using the "csp -mode clear" command. Note that before deleting the old dumps, you need to make sure there are no active incidents which may require those dumps in the future.
 
If needed, increase maximum dumps to 15.  Note that the Customer's permission is required if you are increasing the dump slots.‚Äč
 
See the steps to perform these actions in the "Additional Information" section below.

Special Considerations
 

Additional Information
Steps to Check and Increase number of raw dumps.
 
List raw dumps:

# csp -mode list
csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump
csp: 3 dumps found, 3 dumps to process
csp: Sel   ID (date-time-token)   Nodes  Event  Instigator         Status
csp: ---  ----------------------  -----  -----  ---------------    ------
csp:  *   2016/05/05-07:47:09-13      2      0  846/11/(31877) 
csp:  *   2016/05/05-07:47:09-12      2      0  782/11/(30305) 
csp:  *   2016/05/05-07:47:09-07      2      0  745/11/(20350) 

Find out many many files/folders are in the directory

# psh "ls -1 `pdepath -D` | wc -l"                                                                
byn001-8 byn011-8 byn016-9 (3):
3

To list and clear all of the raw dumps:
 
# csp -mode list
csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump
csp: 6 dumps found, 5 dumps to process
csp: Sel   ID (date-time-token)   Nodes  Event  Instigator         Status
csp: ---  ----------------------  -----  -----  ---------------    ------
csp:  *   2018/01/23-06:31:16-05      1   3601  30719/13/(26238) 
csp:  *   2018/01/19-04:39:09-04      1   3601  30718/13/(26230) 
csp:  *   2018/01/15-03:19:51-03      1   3601  30719/13/(26238) 
csp:  *   2018/01/14-22:37:06-02      1   3601  30718/13/(26230) 
csp:  *   2018/01/12-02:42:10-01      1   3601  30719/13/(26238) 
csp:      dump_20171128_061321_1-01_1        1         33              dump belongs to a different PDE version

# csp -mode clear
csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump
csp: 6 dumps found, 5 dumps to process
Dump 2018/01/23-06:31:16-05 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump 2018/01/23-06:31:16-05
csp: Dump 2018/01/23-06:31:16-05 cleared

Dump 2018/01/19-04:39:09-04 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump 2018/01/19-04:39:09-04
csp: Dump 2018/01/19-04:39:09-04 cleared

Dump 2018/01/15-03:19:51-03 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump 2018/01/15-03:19:51-03
csp: Dump 2018/01/15-03:19:51-03 cleared

Dump 2018/01/14-22:37:06-02 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump 2018/01/14-22:37:06-02
csp: Dump 2018/01/14-22:37:06-02 cleared

Dump 2018/01/12-02:42:10-01 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump 2018/01/12-02:42:10-01
csp: Dump 2018/01/12-02:42:10-01 cleared

Dump dump_20171128_061321_1-01_1 is about to be cleared
Are you sure? [y,n,q,?] > y
csp: Clearing dump dump_20171128_061321_1-01_1
csp: Dump dump_20171128_061321_1-01_1 cleared
#
 
To increase maximum number of raw dumps

# ctl              
> screen debug
(0) Start DBS:         On            (1) Break Stop:        Off
(2) Start With Logons: All           (3) Start With Debug:  Off
(4) Save Dumps:        Off           (5) Snapshot Crash:    Off
(6) Maximum Dumps:     5             (7) Start PrgTraces:   Off
(8) Restart Dump Type: Selective
> 6 = 15
> write
> screen debug   << confirm change  >>
(0) Start DBS:         On            (1) Break Stop:        Off
(2) Start With Logons: All           (3) Start With Debug:  Off
(4) Save Dumps:        Off           (5) Snapshot Crash:    Off
(6) Maximum Dumps:     15            (7) Start PrgTraces:   Off
(8) Restart Dump Type: Selective
 
> quit